[jun-mercury]

Stacks on #42 and carries its test commit beneath the fix. Read it as the diff from #42.

Why

A server that outlives one build serves the next build from memory. That memory is the unit's HomeUnitEnv in make.hug, its module graph nodes, the HPT entries, and the interpreter's loaded code. Until now the unit id alone decided whether that memory was reused. hasUnit and compareUnits skipped a known id without reading its plan, mergeModuleGraphNodes kept the old ModSummary when a metadata request produced a new one, prepareHmiLoader trusted every HPT entry, and the Interp was never dropped, so a TH splice kept running the first build's code because GHC's getLinkDeps excludes modules it has already loaded. GHC's --make runs checkOldIface before it trusts an interface, and that check compares mi_src_hash, mi_flag_hash from fingerprintDynFlags, and the ABI hashes in mi_usages. The worker dropped that check when it kept state across requests. This change puts it back, at the granularity the worker's state has.

The unit id stays the key. A new UnitFingerprint beside each stored unit is the validity check, the way Bazel's WorkerKey keeps the worker files' hash for validation and not for identity.

Scope

• Types.State.Make gains UnitFingerprint (flags, modules, planFiles) and MakeState.unitFingerprints.
• Internal.State.Make gains evictUnit, linkedInInterp, dropInterpIfLinked, storeUnitFingerprint and ensureInterp, which replaces loadStateCompile.
• Internal.State.withState restores the HUG and module graph, runs the request's setup, then picks the interpreter, so an eviction or reload during setup gives the current session a fresh loader.
• Internal.Cache.Metadata gains flagsFingerprint and validateStoredUnit, stores a fingerprint wherever a unit enters hug, and revalidates known units in loadCachedDepUnits.
• Internal.Cache.Hpt gains readIfaceHeader and interfaceStale, checks them in prepareHmiLoader, and validates the unit in loadHomeUnit.
• Internal.Metadata.prepareMetadataSession evicts a known unit before a metadata request redefines it.
• Internal.Session.withGhcMakeModule drops the interpreter when the target module's old code is loaded.
• ghc-worker/test/StaleUnitTest.hs loses the stillStale wrapper, so the four sequences assert directly, and gains three tests, for the flag fingerprint, the header reader and the fail-closed sidecar check.

The client, buck-proxy and the caps are not touched.

What is checked, and when

At request start, for the requesting unit named by --home-unit, and for its dependency units when the unit is restored, the plan and args files are hashed and compared with the hashes stored at restore time. If they differ, the plan is decoded and the args parsed, and GHC's fingerprintDynFlags, fingerprintOptFlags and fingerprintHpcFlags are computed over the parsed flags, together with the package flags and databases GHC leaves out of them (the worker builds the unit state from those) and the GHC libdir. A different flag fingerprint or module set evicts the unit. Eviction drops its HomeUnitEnv, its module graph nodes, its bytecode locks, its extra-library request, its fingerprint, and the interpreter. The unit is then restored from the plan as if it had never been seen. Two execution roots that differ only in -odir and -hidir hash differently at the byte level and then agree on the fingerprint, so they do not evict each other.

Per interface, in the loader path, a 1 KiB prefix of the .hi gives the header's source hash, which is compared with the entry's mi_src_hash; the .hash sidecar --abi-out wrote is compared with the entry's mi_mod_hash. Either mismatch reloads the entry. The source hash is needed because a value-only edit keeps the ABI hash, and a TH splice reads the value.

Fail closed. No stored fingerprint, an unreadable plan or args file, an unreadable interface, or a missing .hash all count as not trusted and evict or reload. Each eviction, reload and interpreter drop is one ghc-worker: line through the request log with its reason.

Tradeoffs

• A worker-side digest of the plan and args bytes alone was the cheaper check. It evicts when only -odir or -hidir move, which is every pair of execution roots on one host. The bytes are kept as the fast path and GHC's fingerprint decides when they differ.
• A session keyed by the build id (the Gradle daemon's model) discards the warm HPT every build and cannot work on remote execution, where the id would change every action digest.
• Reloading home-unit interfaces from disk on every request and keeping only the package databases in memory would remove the per-module check. The estimated cost is one readBinIface plus initModDetails per dependency per request, against the 1 KiB header read and one sidecar read this change does. It is the alternative worth prototyping if the validation proves fiddly in production.
• Per-module invalidation for a unit whose module set changed would need an :add-style path into the module graph. Whole-unit eviction is coarser and is the only granularity that handles a module added under a known unit today.
• Dropping the whole interpreter instead of unloading one module's linkable. GHC has no partial unload for bytecode; the cost is one relink of the transitive closure the next splice needs, once per recompiled-and-loaded module. Object-code home linkables loaded through the RTS linker are not unloaded; with -fprefer-byte-code none are loaded.

Blast radius

Every compile and metadata request through a long-lived server. When the plan and args bytes match, a request pays two file hashes for the unit and, per dependency in the closure, one 1 KiB read and one sidecar read. When they do not match, one plan decode and one args parse. A unit evicted in error is restored from its plan, which is what a fresh server does for every unit, so the failure mode of a wrong fingerprint is a slower build and not a stale one. Without the fix a server that survives a build hands the next build stale interfaces, stale flags and stale TH results, as #42 shows.

Verification

The test suite ran as WT=<worktree> TEST_PATTERN=ALL nix build --impure -f test-stale.nix against lab's GHC 9.10.3. The builder is a scratch file that overrides buck-worker-types across the package set so the new MakeState field is seen.

    stale unit state across builds
      source changes, same args: the second build exports the new binding:                     OK (0.28s)
      unit args change: -DFOO added, the second build exports the CPP-gated binding:           OK (0.39s)
      TH splice reads a changed module: the second build's splice sees the new value:          OK (0.71s)
      module added to a known unit: the importer's second build sees the new module's binding: OK (0.52s)
      flag fingerprint ignores output dirs but tracks a preprocessor define:                   OK (0.17s)
      interface header source hash agrees with the full interface:                             OK (0.15s)
      interface check fails closed on a missing sidecar and passes on a matching one:          OK (0.16s)
All 20 tests passed (26.44s)

The four sequences from #42 assert directly. The flag-hash and fail-closed cases are unit tests of flagsFingerprint and interfaceStale, because the in-process harness compiles without --abi-out and never writes a .hash sidecar, and a two-root plan-flow sequence needs a real server. The shell harnesses below provide that server.

run-stale-seq.sh drives one server through six builds of unit u and compares the ABI hash of each produced .dyn_hi with a one-shot ghc -c of the same tree. Upstream 85f41f9 had builds C, E and F wrong, see #42.

build A (v1)  K 204f229a89ff8f012c1c463c95de01f3 SAME   M d4c9cb533c8a01dc130d2b958fcc8f4d SAME
build B (v2)  K 5c8d00a4b583857287fb22e49768ab62 SAME   M 059051a07b4e956b173ab2c50900ec26 SAME
build C (v3)  K compiled outside the server, .hash present; M 584ed63e7c54fdd2d7e1564c96b01670 SAME, records K as 21a66ca0... SAME
build D (v2)  K compiled outside, .hash removed; M 059051a07b4e956b173ab2c50900ec26 SAME
build E (v4)  K2 added, M imports it; M 584ed63e7c54fdd2d7e1564c96b01670 SAME (twice: after K2 outside, after K2 through the server)
build F (v5)  args gain -DFOO; K cea784688a532d4e3092b8d02495cfbe SAME, exports [k kFoo T{T U}] SAME
legacy -M (no --dep-units-static), fresh server: A then B, K and M SAME both times; the compiled K says k = 2

The client's stderr carried ghc-worker: evict unit u: redefined by a metadata request once, in build B of the legacy run. That eviction is what the four #42 cases depend on.

run-th-stale.sh gives mth = I# 1# in build A and mth = I# 7# in build B, after k = 7, -c K and -c MTH through the same server. The one-shot reference is I# 7#, and upstream said 1#. The client's stderr carried ghc-worker: drop interpreter: K is loaded and about to change.

cards-ab.sh sends the cards unit src/Mercury/Sentry/Constants through one server in cases A to D and compares the Fingerprints.dyn_hi ABI with one-shot. The run used a scratch copy that strips a stray -DSENTRY_V2 an earlier run had left in the checkout's args file. Upstream's earlier pass on case C was that leak.

A  799fb96f2da558e01dbdfc163e3bdcec SAME   exports [default_]
B  861d0cff9fa9c3b9a5c191cbcd16f38a SAME   exports [defaultV2 default_]
C  0a5b2f81ec91df624761fedde0de16b3 SAME   exports [defaultV2 defaultV3 default_]   (upstream: 861d0cff... DIFF, no defaultV3)
D  Extra afc6cbdfae64c928d27efa3e5863f8a0 SAME; Fingerprints 0a5b2f81ec91df624761fedde0de16b3 SAME   (upstream: panic "Could not find ModSummary" for Extra)

The client's stderr carried ghc-worker: evict unit root-src-Mercury-Sentry-Constants-Constants: module set changed: +Mercury.Sentry.Constants.Extra. Case C evicted nothing, because the cards -M runs with --dep-units-static, so the unit is transient and the -c restores it from the plan with the new args.

Per-request cost comes from the same cards sequence on an idle machine, two rounds each, with the fork's client behind a timing wrapper. Warm -c requests: 124 ms → 132 ms mean, 14 requests each, spread 88 to 174 ms. -M requests: 115 ms → 111 ms. A fast-path request reads two file hashes for the unit (plan and args) and, per dependency in the closure, a 1 KiB interface prefix and the .hash sidecar. A miss decodes the plan and parses the args once.

The remote pool check is out of scope for this PR, because the pool would need a republished closure and a new pool for this binary, both shared state. The local cards check stands in for it.

Unverified at HEAD

The HEAD lineage (fix/validate-unit-state-across-builds, on #42's test/stale-unit-across-builds) is the cherry-pick of the 85f41f9 commit with conflicts resolved by hand and no build, since HEAD needs two GHC patches lab's 9.10.3 lacks. The resolution changed three things. evictUnit takes useIncrModGraph and resets moduleGraphState to emptyEModuleGraph before rebuildModuleGraph, because the incremental reachability index only adds keys. loadCachedHomeUnit keeps HEAD's node-based store and its extra Bool. The import lists are the union of both sides. Everything tested above ran on fix/validate-unit-state-across-builds-85f41f93 (same commit on 85f41f9, pushed to the fork for reference).

A known limit stays. A compile node's stored ModSummary (legacy -M graph with fixedNodesCache = False) can still hold an old source buffer when no metadata request reaches the state between builds; production uses fixed nodes, which re-summarise from disk.

Assisted-by: Claude:claude-fable-5-1