Repository navigation
feat(gate): enforce the method contract — a selected method could go undelivered with every gate green - #5
MyAlterLego wants to merge 1 commit into
Conversation
The intake asks which analysis to run and writes the answer to model.json as
`scope.methods`. Nothing read that field. Not ScopeGate, not VerifyGate, not
DiscoveryGate, not ComposeChains, not Prioritize, not the renderer:
$ grep -c methods Tools/{ScopeGate,VerifyGate,DiscoveryGate,ComposeChains,Prioritize}.ts
0 0 0 0 0
So "STPA + STRIDE" was an unenforced promise. A run could record it, deliver
STPA only, and pass every gate green.
That happened. A full-surface run recorded STPA + STRIDE, never executed the
STRIDE pass, and instead grew a hand-written nine-row table at the end of
05-constraints.md which the scope prose then described as "a STRIDE
per-element sweep was run". Every gate passed. The requester found it by
reading the finished report and asking where STRIDE was — which is the one
place a method gap should never be discovered first.
This is the Step-4 lesson again: the workflow said to do it, the workflow was
not enough, and it only stopped being skipped when RenderReport began exiting
5 without 04-scenarios.md. A method selection no gate checks is a control
action with no feedback channel.
StrideGate.ts checks five things and exits 8:
1. CONTRACT scope.methods is recorded at all. Absent fails rather than
assuming STPA-only — that assumption is the hole.
2. DELIVERY STRIDE selected => 00-stride-seed.md exists with the five
sections SeedWithStride.md specifies.
3. RECONCILE the cross-check runs in BOTH directions. STRIDE->STPA is the
half people write; STPA->STRIDE names the emergent findings
STRIDE could not produce, and is the half that gets dropped.
4. PROVENANCE how the pass was produced is declared. A hand-run sweep is
legitimate (SeedWithStride sanctions it for code targets) but
must not silently read as a Fabric-pattern run — same reason
VerifyGate records which model reviewed the findings.
5. NO UPGRADE STRIDE-only + a populated grid = the run quietly upgraded
itself to STPA, which Start.md forbids.
Also:
- RenderReport grades 00-stride-seed.md `core` ONLY when the contract names
STRIDE, so an STPA-only run is unaffected and a STRIDE run exits 5 without
it. Belt and braces with the gate.
- Adds the §7b section, so the artifact the workflow already mandates is
actually displayed. It was previously written and silently ignored.
- Writes stride-scorecard.json so the report states method coverage from
computed data rather than narration.
- `stpa stride [dir] [--warn-only]`, wired into `stpa run` after ScopeGate.
Exit 8 was free (1,2,4,5,6,7 taken; VerifyGate uses 3).
Verified against six cases: no contract (8), selected-not-delivered (8),
missing sections (8), one-directional reconciliation (8), silent upgrade (8),
STPA-only (0 — does not demand a STRIDE artifact). Plus --warn-only forces 0,
and the renderer exits 5/0 with the seed absent/present.
One bug found by running it: the threat-row matcher `|\s*ST-\d+` also matched
the reconciliation table's rows, which open with an id followed by prose in
the same cell, and reported four false positives. It now requires the id to
be its own cell.
|
Closing this — the premise it was built on has been revised, and the reasoning is worth recording rather than losing. This gate enforces that a method selected at intake is actually delivered. It was written after a run recorded But the fix it implements assumes STRIDE belongs inside the STPA skill as a secondary pass. On review that assumption is wrong, for a reason the gate can't fix: A STRIDE pass run by the same model that authored the STPA analysis produces a circular cross-check. The value of two methods is that they fail differently, and that property comes from independence of the reasoner, not from the vocabulary. When one model enumerates both sides, "22 of 26 threats map to existing UCA cells" is evidence of internal consistency, not of coverage. Running the Fabric So a real dual-method assessment needs both halves first-class — each returning full findings, each independently adversarially validated, then synthesised. That is a different deliverable from a pure STPA run, and bolting a weaker half onto one devalues both. It belongs in its own skill. The decision therefore:
The branch #4 is unaffected and should still merge — those are two genuine bugs (a compose gate that silently checked the wrong directory and reported a false One thing worth carrying into the future skill regardless of STRIDE: An intake field no gate checks is an unenforced promise — the same defect class this toolkit exists to find. |
Closes the hole that let a selected method go undelivered while every gate passed green.
The hole
The intake asks which analysis to run and writes the answer to
model.jsonasscope.methods. Nothing read that field:The only
STRIDEstring inRenderReport.tsis boilerplate prose in the §0 primer. So "STPA + STRIDE" was an unenforced promise.How it failed in practice
A full-surface run against a ~3,600-file TypeScript monorepo recorded
STPA + STRIDE, never executed the STRIDE pass, and instead grew a hand-written nine-row table at the end of05-constraints.md— which the scope prose then described as "a STRIDE per-element sweep was run". Every gate passed. Scope contract honoured, plan complete, composition rated, report rendered.The requester found it by reading the finished report and asking where STRIDE was. That is the one place a method gap should never be discovered first.
This is the Step-4 lesson repeating: the workflow said to do it, the workflow was not enough, and it only stopped being skipped when
RenderReportbegan exiting 5 without04-scenarios.md. A method selection that no gate checks is a control action with no feedback channel — the defect class this toolkit exists to find.What the gate checks
scope.methodsrecorded at all00-stride-seed.mdexists with the five sectionsSTRIDE→STPAis the half people write.STPA→STRIDEnames the emergent findings STRIDE could not produce, and is the half that gets dropped.SeedWithStride.mdsanctions it for code-only targets — but must not silently read as a Fabric-pattern run. Same principle asVerifyGaterecording which model reviewed the findings.Start.mdforbids.Exit 8 (1, 2, 4, 5, 6, 7 taken;
VerifyGateuses 3).Also included
RenderReportgrades00-stride-seed.mdcoreonly when the contract names STRIDE. Grading it core unconditionally would fail every STPA-only run; leaving it merely supporting is how the miss happened. The contract decides — same principle asscope.requesteddeciding whether an unmodeled control action is a deferral or a shortfall.00-stride-seed.mdand the renderer had no slot for it, so the file was written and silently ignored.stride-scorecard.json, so the report states method coverage from computed data rather than narration.stpa stride [dir] [--warn-only], wired intostpa runimmediately afterScopeGate— both are contract gates.Verification
Six cases, all as expected:
Plus
--warn-onlyforces 0, and the renderer exits 5 / 0 with the seed absent / present.On a real analysis it reports:
One bug found by running it against real data: the threat-row matcher
|\s*ST-\d+also matched the reconciliation table's rows, which open with an id followed by prose in the same cell (| ST-1 plaintext :80 forward |), producing four false positives. It now requires the id to be its own cell. The first draft also compared bucket sums to detect undisposed rows, which double-counts a row that is bothNEWand references aCA-n; it now asks the question directly and names the offending rows.Notes
main, so this is independent of fix: compose gate read the wrong directory; report CSS escape emitted the wrong glyph #4. Both touchRenderReport.tsbut in different regions (~289 / ~1219 here vs 1009 there), so they merge cleanly in either order.stpa newscaffold does not recordmethods, so it fails this gate — deliberately, and exactly as it already failsScopeGatefor having noscope.requested. Both are intake's job to fill.