Repository navigation
Context tune: 45K live context, sweep data, Codex window fix - #6
Conversation
- Context sweep on the production launch line (bench/ctx-sweep.py, data/ctx-sweep-20261018.jsonl): speed holds to 48K (43.6 tok/s with 39K tokens in context), falls off at 56K (22.4 tok/s, ~1 GiB moved to system memory). The constant 0.96 GiB "shared" is the vision projector in host memory, not a spill. - Governor margin 2.5 -> 1.2 GiB and -cram 6144 MiB: live load 45,056 (49,152 when the desktop is light), 45-50 tok/s warm. - agents/codex-qwen.cmd: client cap 32,768 -> 65,536, compaction at 90% of the window. - Tuning log rows 24-27, including the 17-18 Oct slowdown fixed by a Windows restart. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G4G2wCLwkaZUjYJd6T3uZV
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9ccf377ecb
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| | 23 | Intel graphics services | Graphics Software Service 15 MB RAM, shader distributor 3 MB, PROSet monitoring small; no overlay process was running and none held VRAM. Switching them off is a tidy-up, not a speed-up | observed on the PC | | ||
| | 24 | Throughput regression, 17-18 Oct: warm 300-token check fell to 28 tok/s (47 on 15 Oct) with no config change. Checked: ECC **off**, GPU at 2400 MHz and 65 °C under load, no GPU/VRAM/fan tuning, driver unchanged, power plan High performance | a **Windows restart** (uptime was 2 d 13 h) restored it: 42.1 / 44.8 / 46.2 tok/s on the same check. Cause INFERRED: graphics memory manager state after a long uptime | observed on the PC (2026-10-18) | | ||
| | 25 | Context sweep: the exact production command line on a spare port, governor off, other apps holding 1.55-1.66 GiB of VRAM. Per size: 3 short code requests, then one request with the context 80% full (code-like filler) | 32K: 54-57 short, **48.2** deep (26K tokens, prompt 456 tok/s). 40K: 51-57 / **46.3** (32.6K, 439). 48K: 50-54 / **43.6** (39K, 421), 1.05 GiB left free. 56K: **31-33 / 22.4**, 1.9 GiB in shared memory. The cliff is between 48K and 56K: 56K would have needed to leave about 0.6 GiB free, and the driver moved about 1 GiB to system memory instead. The 0.96 GiB of "shared" seen at every size is the vision projector held in system memory (`--no-mmproj-offload`, 0.87 GiB file), not a spill | [data/ctx-sweep-20261018.jsonl](../data/ctx-sweep-20261018.jsonl), [bench/ctx-sweep.py](../bench/ctx-sweep.py) | | ||
| | 26 | Governor free-VRAM margin 2.5 -> **1.2 GiB** (MAX and BALANCED), host-RAM prompt cache `-cram` 2048 (MAX) / 4096 (BALANCED) -> **6144 MiB** for both, so a 40K agent prompt (about 2.5 GiB of KV) fits in the cache when two agents take turns on the one slot | the governor now loads **45,056** with other apps at 1.64 GiB (1.35 GiB left free), 49,152 when the desktop is light, and steps down when other apps grow. Warm check after the change: 45.1 / 50.1 / 47.2 tok/s | governor.log (2026-10-18 04:04) | |
There was a problem hiding this comment.
Commit the advertised governor tune
For anyone deploying the governor from this repository, these settings are not actually applied: governor/profiles.json still gives MAX a 1.35 GiB margin and 8192 MiB cache, and BALANCED a 1.5 GiB margin and 4096 MiB cache. Because launch.ps1 reads those values directly, the shipped governor will not use the documented 1.2 GiB/6144 MiB configuration or reliably produce the claimed 45,056/49,152-token loads; update the profile alongside this documentation or mark the result as local-only.
Useful? React with 👍 / 👎.
| set "QWEN_WIN=100000" | ||
| for /f "usebackq delims=" %%w in (`powershell -NoProfile -ExecutionPolicy Bypass -File C:\AI\governor\qwen.ps1 window`) do set "QWEN_WIN=%%w" | ||
| if "%QWEN_CLIENT_CAP%"=="" set "QWEN_CLIENT_CAP=32768" | ||
| if "%QWEN_CLIENT_CAP%"=="" set "QWEN_CLIENT_CAP=65536" |
There was a problem hiding this comment.
Preserve a safe fallback when window discovery fails
If the PowerShell invocation produces no output—for example because C:\AI\governor\qwen.ps1 is missing or fails before printing—the initial QWEN_WIN=100000 remains in effect, and this increased cap now tells Codex it has 65,536 tokens even when the server may have only a 32K–49K context. Codex then waits until roughly 59K to compact and receives an explicit context-overflow error; initialize the wrapper to the same conservative fallback used by qwen.ps1 or only raise the cap after successful discovery.
Useful? React with 👍 / 👎.
| while not health(): | ||
| if p.poll() is not None: | ||
| raise RuntimeError(f"server exited rc={p.returncode}") |
There was a problem hiding this comment.
Reject an already-occupied benchmark port
If another llama server or compatible service is already listening on port 10011, health() succeeds immediately and this loop never checks whether the newly spawned process exited because it could not bind. The subsequent requests then benchmark the pre-existing server while recording the requested ctx and the failed child's PID, silently corrupting the sweep; perform a preflight port check or verify p.poll() and that the responding server is the process just launched before collecting results.
Useful? React with 👍 / 👎.
Measured context tune on the B60 (18 Oct 2026).
Sweep (exact production launch line, governor off, other apps 1.55-1.66 GiB VRAM;
bench/ctx-sweep.py, rawdata/ctx-sweep-20261018.jsonl):Applied on the PC: governor margin 1.2 GiB and
-cram 6144→ live load 45,056 (49,152 when the desktop is light), warm check 45.1 / 50.1 / 47.2 tok/s.In this PR: tuning-log rows 24-27, changelog entry,
agents/codex-qwen.cmdclient cap 32,768 → 65,536 with compaction at 90 %, the sweep script and its raw results. No secrets, no user paths.🤖 Generated with Claude Code
https://claude.ai/code/session_01G4G2wCLwkaZUjYJd6T3uZV