Skip to content

docs: FIELD-NOTES.md — the failure mode behind every gate - #7

Merged
MyAlterLego merged 1 commit into
mainfrom
docs/field-notes
Jul 25, 2026
Merged

MyAlterLego merged 1 commit into
mainfrom
docs/field-notes

Conversation

@MyAlterLego

Copy link
Copy Markdown
Contributor

The gates in #6 each exist because an analysis shipped a specific error, but that reasoning lived only in a local SKILL.md — not in this repo, and on an ephemeral filesystem. The checks would have outlived the explanation, and the first person to hit a red gate would have no way to judge whether it was telling them something true.

Seven failure modes, each with the evidence and the gate that now catches it:

  1. Four false absence claims — asserting a control was absent without reading the code implementing it. One rated band-1 R0 and recorded as CONFIRMED. Same mechanism every time: read the use, infer the default, never read the declaration or the mount. The bias runs one way, so it always inflates severity.
  2. 2,508 guard candidates the analysis never consumed — including a WWW-Authenticate header inside the middleware a finding declared did not exist.
  3. A belief credited as a working control because sourcedFrom named a hop, not an origin — invalidating six remediations that were placebos against a forged token.
  4. Two generic RPC bridges that survived two passes and an adversarial review, because a :param route collapses into one REST control action.
  5. 0 upgrades in 96 verdicts — a property of the brief, not evidence of completeness.
  6. A verdict filed but reverted by the apply step, while the scorecard claimed it had been checked.
  7. A hand-typed count that drifted from the grid.

It also records the three false positives these gates produced on first run, because a gate that cries wolf gets disabled — that is a failure of the gate, not a cosmetic issue. The generalisable one: the evidence standard must match what the analysis was derived from. A codebase analysis cites code; a design-document analysis cites the document.

Docs only — no tool changes.

The gates added in #6 each exist because a specific analysis shipped a specific
error. That reasoning lived only in a local SKILL.md, which is not in this repo and
sat on an ephemeral filesystem — so the checks would have outlived the explanation
of why they are worth their friction, and the first person to hit a red gate would
have had no way to judge whether it was telling them something true.

Records seven failure modes with the evidence: four false absence claims (one rated
band-1 and recorded as CONFIRMED); 2,508 guard candidates the analysis never
consumed; a belief credited as a working control because sourcedFrom named a hop
rather than an origin, invalidating six remediations; two generic RPC bridges that
survived two passes and a review; a one-directional review returning 0 upgrades in
96 verdicts; a verdict filed but reverted by the apply step; and a hand-typed count
that drifted from the grid.

Also records the three false positives these gates produced on first run, because a
gate that cries wolf gets disabled — that is a failure of the gate, not a cosmetic
issue.
@MyAlterLego
MyAlterLego merged commit ec7cbbe into main Jul 25, 2026
2 checks passed
@MyAlterLego
MyAlterLego deleted the docs/field-notes branch July 25, 2026 23:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants