Repository navigation
18 Oct field report, governor v4.3, Codex compatibility and engine research - #5
edagher92-coder wants to merge 16 commits into
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 92a2ea7790
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| # a request may have arrived during the (multi-second) save: re-read the slot before letting the caller unload | ||
| $g2 = SlotUnloadGate (SlotGetSlot $base) | ||
| if (-not $g2.ok) { SlotLog "unload postponed: a request started during the slot save ($($g2.reason))"; return $false } |
There was a problem hiding this comment.
Fail closed when the post-save slot check fails
If /slots becomes temporarily unreadable after a multi-second save, SlotGetSlot returns null and SlotUnloadGate treats that unknown state as permission to proceed. Because Unload performed its combined /slots and /metrics check only before the save, a request that starts during the save can therefore be terminated by the immediately following unload. Treat the post-save unknown state as busy, or rerun InFlight immediately before unloading.
Useful? React with 👍 / 👎.
| "id": "qwen3.8-27b-base", | ||
| "port": 10006, | ||
| "model": "C:/AI/models/qwen3.8-27b/Qwen3.8-27B-UD-Q4_K_M.gguf", |
There was a problem hiding this comment.
Register the preserved base model with llama-swap
This adds qwen3.8-27b-base only as a launcher variant, so the advertised preserved base model cannot be requested. A repo-wide search shows that llama-swap/config.yaml still registers only the default, turbo, big, and long 27B models, while $ids and $ports in governor.ps1 also omit this new ID; clients therefore receive an unknown-model response, and the governor could not track it even if it were launched. Add the corresponding swap model and governor mappings.
Useful? React with 👍 / 👎.
| rem codex-qwen-lean: Codex on the local Qwen with the lean CODEX_HOME (3K-token base prompt instead of 12K). | ||
| rem Window = live server ctx - 8K (qwen window), auto-compact at 75% of it. Nothing in %USERPROFILE%\.codex is touched. | ||
| setlocal | ||
| set "CODEX_HOME=C:\AI\codex-local\home" |
There was a problem hiding this comment.
Preserve the sandbox temp override in the lean wrapper
On installations where TEMP or TMP points to an admin-owned C:\Temp, this wrapper inherits that directory and Codex's CodexSandboxUsers account cannot execute normal shell and test operations there. The existing agents/codex-qwen.cmd explicitly redirects both variables to %LOCALAPPDATA%\Temp for this reason, but the new recommended lean entry point omits the workaround. Apply the same overrides before launching Codex.
Useful? React with 👍 / 👎.
| if ($calEnabled) { try { $cal = Get-Content "$dir\calib.json" -Raw -ErrorAction Stop | ConvertFrom-Json; $ce = $cal.PSObject.Properties[$v.id] | ||
| if ($ce -and ($null -ne $ce.Value.overhead_gib)) { $overhead = [math]::Round([math]::Max(0.0, [math]::Min($maxOver, [double]$ce.Value.overhead_gib)), 2) } } catch {} } |
There was a problem hiding this comment.
Invalidate calibration when the model or engine changes
The persisted calibration is looked up only by model ID, so an existing qwen3.8-27b entry is reused even when the underlying runtime changes. This commit changes that ID's GGUF from the base model to ThinkingCap, and the supported qwen engine command can also switch executables without clearing calib.json; a stale overhead of up to 1.5 GiB can then substantially shrink the next context, while a stale underestimate can recreate the spill the calibration is intended to prevent. Record and validate the model file identity and engine/config inputs, or discard the entry when they differ.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a81076cf9e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| } catch { | ||
| SlotLog "SAVE failed ($($_.Exception.Message)); unloading anyway" | ||
| if ($tmp) { Remove-Item -LiteralPath $tmp -Force -ErrorAction SilentlyContinue } | ||
| return $true |
There was a problem hiding this comment.
Fail closed when a slot save throws
When the save request times out or otherwise throws after taking several seconds, a new inference can have started since Unload performed its initial activity check, but this catch returns $true and the caller unloads without another InFlight check. The new successful-save null check does not cover this exception path, so a transient save failure can still terminate an active request; fail closed or recheck activity before allowing the unload.
Useful? React with 👍 / 👎.
| function SlotRestoreTargetOk($slot) { | ||
| if ($null -eq $slot) { return (SlotResult $true 'slot state unknown') } |
There was a problem hiding this comment.
Reject restores when the target slot state is unknown
When GET /slots transiently fails just after the new server becomes healthy, this treats the target as safe and proceeds to restore into slot 0. A user request may already have queued behind the model load or started while the probe failed, so the restore can overwrite or interfere with that request despite this guard's purpose; unknown target state should skip both restore and warm-up.
Useful? React with 👍 / 👎.
| if ($m.loaded -and $s.vram_llama_gib -lt 4 -and $s.vram_used_gib -gt 12) { $s.vram_others_gib = $script:lastOthers; $s.vram_llama_gib = [math]::Round($s.vram_used_gib - $script:lastOthers, 2) } elseif ($m.loaded -or $m.state -eq 'stopped') { $script:lastOthers = $s.vram_others_gib } # never learn 'others' while a model is starting (its counter lags) | ||
| $c2 = ReadJsonFile "$dir\profiles.json"; if ($c2 -and $c2.auto) { $cfg = $c2; $A = $cfg.auto; $script:vramTotalGib = CfgNum $cfg.vram 'total_gib' $script:vramTotalGib } else { LogEvery 'cfg' 30 'profiles.json unreadable: using the last good configuration' } | ||
| $sampleSec = [int](CfgNum $A 'sample_sec' 15); if ($sampleSec -lt 5) { $sampleSec = 5 } | ||
| if ($script:slotLib) { ReapWarmJob; $script:slots = Try1 { SlotCfg $cfg } $script:slots } |
There was a problem hiding this comment.
Honor disabling slots during configuration reload
When profiles.json is hot-reloaded with slots.enabled set to false, SlotCfg legitimately returns null, but Try1 substitutes the previous $script:slots value. The governor therefore continues saving and restoring KV state despite the operator disabling the feature; distinguish a successful null result from an exception so only an unreadable configuration retains the last known slot settings.
Useful? React with 👍 / 👎.
| if __name__ == "__main__": | ||
| d = derive() | ||
| print(json.dumps({k: (round(v, 4) if isinstance(v, float) else v) for k, v in d.items()}, indent=2)) | ||
| print(json.dumps(sustained_runs(), indent=2)) |
There was a problem hiding this comment.
Emit a single JSON document from the timing tool
When the documented reproduction command is redirected to dflash_timing_output.json, this prints a second top-level JSON value, and the later __main__ block prints two more. The checked-in output consequently fails standard JSON parsing with Extra data at line 31, preventing consumers from loading the advertised result file; collect the result sections into one object and serialize it once.
Useful? React with 👍 / 👎.
| $cs = (Get-Counter '\GPU Adapter Memory(*)\Dedicated Usage' -ErrorAction SilentlyContinue).CounterSamples | ||
| $mx = $null | ||
| foreach ($x in @($cs)) { if ($null -ne $x -and $x.Status -eq 0) { $g = [double]$x.CookedValue / 1GB; if (($null -eq $mx) -or ($g -gt $mx)) { $mx = $g } } } | ||
| if ($null -ne $mx) { $samples += $mx } |
There was a problem hiding this comment.
Reject invalid dedicated-memory samples before sizing context
If PDH returns a status-zero but negative or non-finite CookedValue, this accepts it into $samples; the minimum then becomes a negative or invalid $others value, so the context formula later budgets phantom free VRAM and can launch an oversized server. Apply the same finite, nonnegative range validation used by the governor's other VRAM reads before retaining a launch sample.
Useful? React with 👍 / 👎.
| if ($llamaOk) { $llama = GovClamp ([double]$llamaGib) 0.0 $used } | ||
| elseif ($loaded -and $lgOthersNum) { $llama = GovClamp ($used - [double]$lg.others) 0.0 $used } | ||
| elseif ($loaded) { $llama = $used } |
There was a problem hiding this comment.
Mark missing process-memory readings as degraded
When the per-process GPU counter is unavailable while the model is loaded, this reconstructs the model share by subtracting the last known others value from the current adapter usage, but the sample is subsequently left with ok = true and no warning. If another application allocates VRAM during that counter outage, its growth is attributed to llama-server, so profile selection and the pressure detector act on stale others and can suppress the refit intended to prevent spilling; retain the inferred values for display but mark the sample invalid for decisions.
Useful? React with 👍 / 👎.
Imported selectively from three maintainer hand-off archives (Claude and Codex sessions, 15-18 Oct). Measured results, researched-only material and rejected proposals are kept apart and labelled.
docs/FIELD-REPORT-20261018.md): a 57K window spilled 2.90 GiB into WDDM shared memory (20-22 tok/s); the open throughput regression; q8_0 main KV rejected (-53% prefill, -57% generation at 25K depth). Safe window on a busy desktop: 36-40K.docs/CODEX-COMPAT.md) and an opt-in leanCODEX_HOME(start-up prompt 12,235 -> 3,010 tokens).docs/TRIED-AND-REJECTED.md,docs/ROADMAP.md(E0-E5), anddocs/research/(source-level engine study, researched, not run on the card).Not imported: the outside optimisation pack (defective), private machine logs and live configs, duplicate research copies.
Local checks before upload: link checker 0 broken, build_tables --check, build_site (no diff), validate_repo 0 failed, all .ps1 parse, evalsuite selftest 100/100.
🤖 Generated with Claude Code
https://claude.ai/code/session_01DNXKgDTDP7ZzdcjLKTMYRW