Skip to content

18 Oct field report, governor v4.3, Codex compatibility and engine research - #5

Open
edagher92-coder wants to merge 16 commits into
mainfrom
field-report-2026-10-18
Open

edagher92-coder wants to merge 16 commits into
mainfrom
field-report-2026-10-18

Conversation

@edagher92-coder

Copy link
Copy Markdown
Owner

Imported selectively from three maintainer hand-off archives (Claude and Codex sessions, 15-18 Oct). Measured results, researched-only material and rejected proposals are kept apart and labelled.

  • Field report (docs/FIELD-REPORT-20261018.md): a 57K window spilled 2.90 GiB into WDDM shared memory (20-22 tok/s); the open throughput regression; q8_0 main KV rejected (-53% prefill, -57% generation at 25K depth). Safe window on a busy desktop: 36-40K.
  • Governor v3 -> v4.3 with tests (111/0, 7/7, pytest 53/0); MAX/BALANCED margin 1.35 -> 2.5 GiB. Fixes the published v3 loop crash.
  • Codex: compatibility matrix (docs/CODEX-COMPAT.md) and an opt-in lean CODEX_HOME (start-up prompt 12,235 -> 3,010 tokens).
  • docs/TRIED-AND-REJECTED.md, docs/ROADMAP.md (E0-E5), and docs/research/ (source-level engine study, researched, not run on the card).
  • Data: 3 CSVs, raw ctxlab logs (per-process lists redacted), tuning-log rows 24-27. README status note and corrected context guidance.

Not imported: the outside optimisation pack (defective), private machine logs and live configs, duplicate research copies.

Local checks before upload: link checker 0 broken, build_tables --check, build_site (no diff), validate_repo 0 failed, all .ps1 parse, evalsuite selftest 100/100.

🤖 Generated with Claude Code

https://claude.ai/code/session_01DNXKgDTDP7ZzdcjLKTMYRW

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 17, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-17T16:22:34.686218Z a81076c New commits
🔒 Security Review ✅ Completed 2026-10-17T16:12:23.222477Z 92a2ea7 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 92a2ea7790

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread governor/slotlib.ps1
Comment on lines +360 to +362
# a request may have arrived during the (multi-second) save: re-read the slot before letting the caller unload
$g2 = SlotUnloadGate (SlotGetSlot $base)
if (-not $g2.ok) { SlotLog "unload postponed: a request started during the slot save ($($g2.reason))"; return $false }

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Fail closed when the post-save slot check fails

If /slots becomes temporarily unreadable after a multi-second save, SlotGetSlot returns null and SlotUnloadGate treats that unknown state as permission to proceed. Because Unload performed its combined /slots and /metrics check only before the save, a request that starts during the save can therefore be terminated by the immediately following unload. Treat the post-save unknown state as busy, or rerun InFlight immediately before unloading.

Useful? React with 👍 / 👎.

Comment thread governor/profiles.json
Comment on lines +159 to +161
"id": "qwen3.8-27b-base",
"port": 10006,
"model": "C:/AI/models/qwen3.8-27b/Qwen3.8-27B-UD-Q4_K_M.gguf",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Register the preserved base model with llama-swap

This adds qwen3.8-27b-base only as a launcher variant, so the advertised preserved base model cannot be requested. A repo-wide search shows that llama-swap/config.yaml still registers only the default, turbo, big, and long 27B models, while $ids and $ports in governor.ps1 also omit this new ID; clients therefore receive an unknown-model response, and the governor could not track it even if it were launched. Add the corresponding swap model and governor mappings.

Useful? React with 👍 / 👎.

rem codex-qwen-lean: Codex on the local Qwen with the lean CODEX_HOME (3K-token base prompt instead of 12K).
rem Window = live server ctx - 8K (qwen window), auto-compact at 75% of it. Nothing in %USERPROFILE%\.codex is touched.
setlocal
set "CODEX_HOME=C:\AI\codex-local\home"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve the sandbox temp override in the lean wrapper

On installations where TEMP or TMP points to an admin-owned C:\Temp, this wrapper inherits that directory and Codex's CodexSandboxUsers account cannot execute normal shell and test operations there. The existing agents/codex-qwen.cmd explicitly redirects both variables to %LOCALAPPDATA%\Temp for this reason, but the new recommended lean entry point omits the workaround. Apply the same overrides before launching Codex.

Useful? React with 👍 / 👎.

Comment thread governor/launch.ps1 Outdated
Comment on lines +80 to +81
if ($calEnabled) { try { $cal = Get-Content "$dir\calib.json" -Raw -ErrorAction Stop | ConvertFrom-Json; $ce = $cal.PSObject.Properties[$v.id]
if ($ce -and ($null -ne $ce.Value.overhead_gib)) { $overhead = [math]::Round([math]::Max(0.0, [math]::Min($maxOver, [double]$ce.Value.overhead_gib)), 2) } } catch {} }

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Invalidate calibration when the model or engine changes

The persisted calibration is looked up only by model ID, so an existing qwen3.8-27b entry is reused even when the underlying runtime changes. This commit changes that ID's GGUF from the base model to ThinkingCap, and the supported qwen engine command can also switch executables without clearing calib.json; a stale overhead of up to 1.5 GiB can then substantially shrink the next context, while a stale underestimate can recreate the spill the calibration is intended to prevent. Record and validate the model file identity and engine/config inputs, or discard the entry when they differ.

Useful? React with 👍 / 👎.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a81076cf9e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread governor/slotlib.ps1
Comment on lines +366 to +369
} catch {
SlotLog "SAVE failed ($($_.Exception.Message)); unloading anyway"
if ($tmp) { Remove-Item -LiteralPath $tmp -Force -ErrorAction SilentlyContinue }
return $true

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Fail closed when a slot save throws

When the save request times out or otherwise throws after taking several seconds, a new inference can have started since Unload performed its initial activity check, but this catch returns $true and the caller unloads without another InFlight check. The new successful-save null check does not cover this exception path, so a transient save failure can still terminate an active request; fail closed or recheck activity before allowing the unload.

Useful? React with 👍 / 👎.

Comment thread governor/slotlib.ps1
Comment on lines +71 to +72
function SlotRestoreTargetOk($slot) {
if ($null -eq $slot) { return (SlotResult $true 'slot state unknown') }

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Reject restores when the target slot state is unknown

When GET /slots transiently fails just after the new server becomes healthy, this treats the target as safe and proceeds to restore into slot 0. A user request may already have queued behind the model load or started while the probe failed, so the restore can overwrite or interfere with that request despite this guard's purpose; unknown target state should skip both restore and warm-up.

Useful? React with 👍 / 👎.

Comment thread governor/governor.ps1
if ($m.loaded -and $s.vram_llama_gib -lt 4 -and $s.vram_used_gib -gt 12) { $s.vram_others_gib = $script:lastOthers; $s.vram_llama_gib = [math]::Round($s.vram_used_gib - $script:lastOthers, 2) } elseif ($m.loaded -or $m.state -eq 'stopped') { $script:lastOthers = $s.vram_others_gib } # never learn 'others' while a model is starting (its counter lags)
$c2 = ReadJsonFile "$dir\profiles.json"; if ($c2 -and $c2.auto) { $cfg = $c2; $A = $cfg.auto; $script:vramTotalGib = CfgNum $cfg.vram 'total_gib' $script:vramTotalGib } else { LogEvery 'cfg' 30 'profiles.json unreadable: using the last good configuration' }
$sampleSec = [int](CfgNum $A 'sample_sec' 15); if ($sampleSec -lt 5) { $sampleSec = 5 }
if ($script:slotLib) { ReapWarmJob; $script:slots = Try1 { SlotCfg $cfg } $script:slots }

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Honor disabling slots during configuration reload

When profiles.json is hot-reloaded with slots.enabled set to false, SlotCfg legitimately returns null, but Try1 substitutes the previous $script:slots value. The governor therefore continues saving and restoring KV state despite the operator disabling the feature; distinguish a successful null result from an exception so only an unreadable configuration retains the last known slot settings.

Useful? React with 👍 / 👎.

Comment on lines +87 to +90
if __name__ == "__main__":
d = derive()
print(json.dumps({k: (round(v, 4) if isinstance(v, float) else v) for k, v in d.items()}, indent=2))
print(json.dumps(sustained_runs(), indent=2))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Emit a single JSON document from the timing tool

When the documented reproduction command is redirected to dflash_timing_output.json, this prints a second top-level JSON value, and the later __main__ block prints two more. The checked-in output consequently fails standard JSON parsing with Extra data at line 31, preventing consumers from loading the advertised result file; collect the result sections into one object and serialize it once.

Useful? React with 👍 / 👎.

Comment thread governor/launch.ps1
Comment on lines +33 to +36
$cs = (Get-Counter '\GPU Adapter Memory(*)\Dedicated Usage' -ErrorAction SilentlyContinue).CounterSamples
$mx = $null
foreach ($x in @($cs)) { if ($null -ne $x -and $x.Status -eq 0) { $g = [double]$x.CookedValue / 1GB; if (($null -eq $mx) -or ($g -gt $mx)) { $mx = $g } } }
if ($null -ne $mx) { $samples += $mx }

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reject invalid dedicated-memory samples before sizing context

If PDH returns a status-zero but negative or non-finite CookedValue, this accepts it into $samples; the minimum then becomes a negative or invalid $others value, so the context formula later budgets phantom free VRAM and can launch an oversized server. Apply the same finite, nonnegative range validation used by the governor's other VRAM reads before retaining a launch sample.

Useful? React with 👍 / 👎.

Comment thread governor/governor.ps1
Comment on lines +138 to +140
if ($llamaOk) { $llama = GovClamp ([double]$llamaGib) 0.0 $used }
elseif ($loaded -and $lgOthersNum) { $llama = GovClamp ($used - [double]$lg.others) 0.0 $used }
elseif ($loaded) { $llama = $used }

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Mark missing process-memory readings as degraded

When the per-process GPU counter is unavailable while the model is loaded, this reconstructs the model share by subtracting the last known others value from the current adapter usage, but the sample is subsequently left with ok = true and no warning. If another application allocates VRAM during that counter outage, its growth is attributed to llama-server, so profile selection and the pressure detector act on stale others and can suppress the refit intended to prevent spilling; retain the inferred values for display but mark the sample invalid for decisions.

Useful? React with 👍 / 👎.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant