Skip to content

test: pin audit-gate's allowlist to SECURITY.md table rows in both directions - #1345

Merged
mrbobbytables merged 1 commit into
mainfrom
quality/test-audit-gate-security-md-parity
Oct 12, 2026
Merged

mrbobbytables merged 1 commit into
mainfrom
quality/test-audit-gate-security-md-parity

Conversation

@hivecommons-hive

Copy link
Copy Markdown
Contributor

Test Improvement

tests/audit-gate.test.mjs held scripts/audit-gate.mjs's ALLOWLISTED_ADVISORIES
and SECURITY.md's "Known unpatched dependency advisories" table together with a
whole-file securityMd.includes(id). That left two holes, both reproduced by
mutation before the fix:

  1. A prose mention satisfies it. SECURITY.md names GHSA-ch52-4w7c-c8xp at
    lines 54 and 63 — in the paragraph recording that the entry was removed
    after upstream published http-cache-semantics 4.3.0. Adding that id back to
    ALLOWLISTED_ADVISORIES left the parity test passing.
  2. The reverse direction was unchecked. unusedAllowlistEntries() forces the
    allowlist side of a removal; nothing forced the table side, so an orphaned row
    could tell a reader the gate accepts an advisory it would in fact block.
    Adding a table row for a non-allowlisted id left the suite green at 30/30.

This PR adds documentedAdvisoryIds(), which reads the GHSA ids that have a row
inside that ## section, and three tests over it:

  • every allowlisted id has a table row (catches hole 1)
  • every table-row id is allowlisted (catches hole 2)
  • the parser cannot pass vacuously — a reformatted table that the row matcher
    stops recognising fails instead of iterating over an empty set, the same
    silent-pass failure mode audit-gate.mjs already refuses for an unreadable
    npm audit report

The existing includes() test is kept: it still catches an allowlisted id absent
from the file entirely, with a simpler message. Tests only — scripts/audit-gate.mjs
and SECURITY.md are unchanged.

Verification

Run at fb75c1a (this branch's base), node v26.10.0:

  • TZ=UTC node --test tests/audit-gate.test.mjs — 30 pass / 0 fail
  • npm run test:unit:coverage:check — exit 0; src files 100.00% lines /
    100.00% regions, harness files 99.63% regions (unchanged). tests/audit-gate.test.mjs
    region coverage goes 85.71% → 86.96%; its one uncovered region is pre-existing
    (reported at lines 316-318 before, 398-400 after the insertion).
  • npx prettier --check tests/audit-gate.test.mjs — clean
  • Mutation checks, each caught by the intended new test and by nothing else:
    deleted table row → 3 fail; prose-only id re-allowlisted → caught by the new
    table-row test while the old includes() test still passed; orphaned table row
    → 29 pass / 1 fail, the new reverse check alone.

Related Issue

Closes #1344

Overlap

Touches only tests/audit-gate.test.mjs, covering the scripts/audit-gate.mjs
↔ SECURITY.md allowlist cluster. Disjoint from the open hold-gated PRs
(#1334 mdx-active-content, #1339 devcontainer Features, #1330 DirectoryFreshness,
#1332 ci.yml trace upload, #1337 docs-contract.test.mjs broken-link policy,
#1341 e2e coverage Node major, #1343 check:links gate placement) — none of them
touch this file, scripts/audit-gate.mjs, or SECURITY.md.


Filed by quality agent (hold-gated mode). Human review required.

— hive: agent=quality backend=copilot model=claude-opus-5 copilot=1.0.88

…rections

The only link between scripts/audit-gate.mjs's ALLOWLISTED_ADVISORIES and
SECURITY.md's "Known unpatched dependency advisories" table was a whole-file
substring search, which leaves two holes.

A prose mention satisfies it. SECURITY.md names GHSA-ch52-4w7c-c8xp twice, in
the paragraph recording that the entry was removed once upstream published
http-cache-semantics 4.3.0; includes() cannot tell that apart from a live
suppression. Re-adding that id to the allowlist leaves the test passing.

The reverse direction was unchecked. unusedAllowlistEntries() forces the
allowlist side of a removal, but nothing forced the table side, so a row could
outlive its entry and tell a reader the gate accepts an advisory it blocks.

Parse the ids that have a row in that section and assert parity both ways, plus
a guard so a reformatted table fails rather than passing over an empty set --
the silent-pass failure mode audit-gate.mjs already refuses for an unreadable
npm audit report. The existing includes() test stays: it still catches an id
absent from the file entirely.

Signed-off-by: quality <quality@hive.kubestellar.io>
@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

Important

Held for human review by the hive's ACMM level gate.

This PR was opened by the "quality" agent while Hive policy required a human checkpoint for that agent. Non-outreach agents are held at ACMM L3–L5; the outreach agent is always held because it publishes project-facing communication.

Hive will keep the hold label until a human removes it. Operators can make a deliberate one-off release during an ACMM level change with release_level_holds=true, but level changes never release this hold automatically.

@mrbobbytables
mrbobbytables added this pull request to the merge queue Oct 12, 2026
Merged via the queue into main with commit 4fd76d8 Oct 12, 2026
8 checks passed
@mrbobbytables
mrbobbytables deleted the quality/test-audit-gate-security-md-parity branch October 16, 2026 01:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[quality] audit-gate's SECURITY.md parity test passes on a prose mention and never checks the reverse direction

1 participant