Skip to content

Context tune: 45K live context, sweep data, Codex window fix - #6

Merged
edagher92-coder merged 1 commit into
mainfrom
tune-44k-20261018
Oct 17, 2026
Merged

edagher92-coder merged 1 commit into
mainfrom
tune-44k-20261018

Conversation

@edagher92-coder

Copy link
Copy Markdown
Owner

Measured context tune on the B60 (18 Oct 2026).

Sweep (exact production launch line, governor off, other apps 1.55-1.66 GiB VRAM; bench/ctx-sweep.py, raw data/ctx-sweep-20261018.jsonl):

ctx short (tok/s) 80% full (tok/s) shared GiB
32K 54-57 48.2 0.96 (vision projector)
40K 51-57 46.3 0.96
48K 50-54 43.6 0.96
56K 31-33 22.4 1.90 (spill)

Applied on the PC: governor margin 1.2 GiB and -cram 6144 → live load 45,056 (49,152 when the desktop is light), warm check 45.1 / 50.1 / 47.2 tok/s.

In this PR: tuning-log rows 24-27, changelog entry, agents/codex-qwen.cmd client cap 32,768 → 65,536 with compaction at 90 %, the sweep script and its raw results. No secrets, no user paths.

🤖 Generated with Claude Code

https://claude.ai/code/session_01G4G2wCLwkaZUjYJd6T3uZV

- Context sweep on the production launch line (bench/ctx-sweep.py,
  data/ctx-sweep-20261018.jsonl): speed holds to 48K (43.6 tok/s with
  39K tokens in context), falls off at 56K (22.4 tok/s, ~1 GiB moved to
  system memory). The constant 0.96 GiB "shared" is the vision projector
  in host memory, not a spill.
- Governor margin 2.5 -> 1.2 GiB and -cram 6144 MiB: live load 45,056
  (49,152 when the desktop is light), 45-50 tok/s warm.
- agents/codex-qwen.cmd: client cap 32,768 -> 65,536, compaction at 90%
  of the window.
- Tuning log rows 24-27, including the 17-18 Oct slowdown fixed by a
  Windows restart.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4G2wCLwkaZUjYJd6T3uZV
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 17, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-17T17:14:25.156164Z 9ccf377 PR opened
🔒 Security Review ✅ Completed 2026-10-17T17:12:34.776891Z 9ccf377 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9ccf377ecb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread docs/TUNING-LOG.md
| 23 | Intel graphics services | Graphics Software Service 15 MB RAM, shader distributor 3 MB, PROSet monitoring small; no overlay process was running and none held VRAM. Switching them off is a tidy-up, not a speed-up | observed on the PC |
| 24 | Throughput regression, 17-18 Oct: warm 300-token check fell to 28 tok/s (47 on 15 Oct) with no config change. Checked: ECC **off**, GPU at 2400 MHz and 65 °C under load, no GPU/VRAM/fan tuning, driver unchanged, power plan High performance | a **Windows restart** (uptime was 2 d 13 h) restored it: 42.1 / 44.8 / 46.2 tok/s on the same check. Cause INFERRED: graphics memory manager state after a long uptime | observed on the PC (2026-10-18) |
| 25 | Context sweep: the exact production command line on a spare port, governor off, other apps holding 1.55-1.66 GiB of VRAM. Per size: 3 short code requests, then one request with the context 80% full (code-like filler) | 32K: 54-57 short, **48.2** deep (26K tokens, prompt 456 tok/s). 40K: 51-57 / **46.3** (32.6K, 439). 48K: 50-54 / **43.6** (39K, 421), 1.05 GiB left free. 56K: **31-33 / 22.4**, 1.9 GiB in shared memory. The cliff is between 48K and 56K: 56K would have needed to leave about 0.6 GiB free, and the driver moved about 1 GiB to system memory instead. The 0.96 GiB of "shared" seen at every size is the vision projector held in system memory (`--no-mmproj-offload`, 0.87 GiB file), not a spill | [data/ctx-sweep-20261018.jsonl](../data/ctx-sweep-20261018.jsonl), [bench/ctx-sweep.py](../bench/ctx-sweep.py) |
| 26 | Governor free-VRAM margin 2.5 -> **1.2 GiB** (MAX and BALANCED), host-RAM prompt cache `-cram` 2048 (MAX) / 4096 (BALANCED) -> **6144 MiB** for both, so a 40K agent prompt (about 2.5 GiB of KV) fits in the cache when two agents take turns on the one slot | the governor now loads **45,056** with other apps at 1.64 GiB (1.35 GiB left free), 49,152 when the desktop is light, and steps down when other apps grow. Warm check after the change: 45.1 / 50.1 / 47.2 tok/s | governor.log (2026-10-18 04:04) |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Commit the advertised governor tune

For anyone deploying the governor from this repository, these settings are not actually applied: governor/profiles.json still gives MAX a 1.35 GiB margin and 8192 MiB cache, and BALANCED a 1.5 GiB margin and 4096 MiB cache. Because launch.ps1 reads those values directly, the shipped governor will not use the documented 1.2 GiB/6144 MiB configuration or reliably produce the claimed 45,056/49,152-token loads; update the profile alongside this documentation or mark the result as local-only.

Useful? React with 👍 / 👎.

Comment thread agents/codex-qwen.cmd
set "QWEN_WIN=100000"
for /f "usebackq delims=" %%w in (`powershell -NoProfile -ExecutionPolicy Bypass -File C:\AI\governor\qwen.ps1 window`) do set "QWEN_WIN=%%w"
if "%QWEN_CLIENT_CAP%"=="" set "QWEN_CLIENT_CAP=32768"
if "%QWEN_CLIENT_CAP%"=="" set "QWEN_CLIENT_CAP=65536"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve a safe fallback when window discovery fails

If the PowerShell invocation produces no output—for example because C:\AI\governor\qwen.ps1 is missing or fails before printing—the initial QWEN_WIN=100000 remains in effect, and this increased cap now tells Codex it has 65,536 tokens even when the server may have only a 32K–49K context. Codex then waits until roughly 59K to compact and receives an explicit context-overflow error; initialize the wrapper to the same conservative fallback used by qwen.ps1 or only raise the cap after successful discovery.

Useful? React with 👍 / 👎.

Comment thread bench/ctx-sweep.py
Comment on lines +96 to +98
while not health():
if p.poll() is not None:
raise RuntimeError(f"server exited rc={p.returncode}")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reject an already-occupied benchmark port

If another llama server or compatible service is already listening on port 10011, health() succeeds immediately and this loop never checks whether the newly spawned process exited because it could not bind. The subsequent requests then benchmark the pre-existing server while recording the requested ctx and the failed child's PID, silently corrupting the sweep; perform a preflight port check or verify p.poll() and that the responding server is the process just launched before collecting results.

Useful? React with 👍 / 👎.

@edagher92-coder
edagher92-coder merged commit 6d26bb1 into main Oct 17, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant