Skip to content

test: pin CONTRIBUTING.md's e2e coverage threshold values to ci.yml's gate - #1323

Merged
mrbobbytables merged 1 commit into
mainfrom
quality/test-e2e-docs-threshold-parity
Oct 11, 2026
Merged

mrbobbytables merged 1 commit into
mainfrom
quality/test-e2e-docs-threshold-parity

Conversation

@hivecommons-hive

Copy link
Copy Markdown
Contributor

Test Improvement

Adds tests/e2e-docs-threshold-parity.test.mjs, which pins the e2e coverage
threshold values printed in CONTRIBUTING.md's fenced
npm run report:e2e:coverage recipe to the values the e2e-coverage job in
.github/workflows/ci.yml actually enforces.

tests/docs-threshold-parity.test.mjs closed this drift class for the unit
lane (#1193). The e2e lane was left open, and the two contracts that look like
they cover it both stop short:

  • tests/e2e-coverage-gate.test.mjs holds floors under ci.yml's flags, so
    it fails on a weakened gate and passes on a ratcheted one — the direction
    that strands the docs.
  • tests/contributing-e2e-setup.test.mjs compares the documented recipe to
    ci.yml by flag name only, so a documented --check-source-regions 90
    against a gate of 96 satisfies it.

Between them, a contributor who follows the recipe renders a report that passes
locally and fails in CI — the one thing the recipe exists to rule out.

The new test parses the --check-* flags and their values out of both sides
and asserts the maps are deeply equal. It fails loudly, rather than vacuously,
when either side loses the invocation, drops the coverage job, or stops passing
any threshold. Eight fixtures drive the parsers directly so every arm is proven
to go red rather than being asserted on faith.

Also corrects the stale 95% figure at the top of
tests/e2e-coverage-report-file-region-floor.test.mjs, left behind when the
gate moved to --check-source-regions 96. That stale number is the live proof
the drift happens.

Files claimed

  • tests/e2e-docs-threshold-parity.test.mjs (new)
  • tests/e2e-coverage-report-file-region-floor.test.mjs (header comment only)

No production code, no package.json change, and no workflow change — disjoint
from the harness-floor ratchet in #1319 and the just build parity contract in
#1321.

Verification

  • TZ=UTC node --test tests/e2e-docs-threshold-parity.test.mjs — 9/9 pass.
  • Temporarily editing CONTRIBUTING.md to document
    --check-source-regions 95 turns the live assertion red, confirming the
    contract binds the real files and not just its fixtures.
  • npm run test:unit:coverage:check — exit 0. The new file reports
    100.00% lines / 100.00% regions, so it clears
    --check-harness-file-regions without moving any floor.

Related Issue

Fixes #1322


Filed by quality agent (hold-gated mode). Human review required.

— hive: agent=quality backend=copilot model=claude-opus-5 copilot=1.0.88

… gate

tests/docs-threshold-parity.test.mjs pins the unit lane's documented
thresholds to the values package.json enforces (#1193). The e2e lane has
the same split with nothing tying it together: the enforced numbers live
in the report:e2e:coverage invocation inside ci.yml's e2e-coverage job,
the documented numbers live in CONTRIBUTING.md's fenced recipe.

The neighbouring contracts stop short of the values.
tests/e2e-coverage-gate.test.mjs holds floors under ci.yml's flags, so it
passes on a ratchet - the direction that strands the docs.
tests/contributing-e2e-setup.test.mjs compares the recipe to ci.yml by
flag name only, so a documented --check-source-regions 90 against a gate
of 96 satisfies it. A contributor following the recipe therefore renders
a report that passes locally and fails in CI.

tests/e2e-docs-threshold-parity.test.mjs parses the --check-* flags and
their values from both sides and asserts the maps are equal, failing
loudly rather than vacuously when either side loses the invocation or
stops passing a threshold. Fixtures drive the parsers so every arm is
proven to go red.

Also corrects the stale '95%' in
tests/e2e-coverage-report-file-region-floor.test.mjs, which the gate left
behind when it moved to --check-source-regions 96.

Signed-off-by: quality <quality@hive.kubestellar.io>
@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

Important

Held for human review by the hive's ACMM level gate.

This PR was opened by the "quality" agent while Hive policy required a human checkpoint for that agent. Non-outreach agents are held at ACMM L3–L5; the outreach agent is always held because it publishes project-facing communication.

Hive will keep the hold label until a human removes it. Operators can make a deliberate one-off release during an ACMM level change with release_level_holds=true, but level changes never release this hold automatically.

@mrbobbytables
mrbobbytables added this pull request to the merge queue Oct 11, 2026
Merged via the queue into main with commit ce5e499 Oct 11, 2026
8 checks passed
@mrbobbytables
mrbobbytables deleted the quality/test-e2e-docs-threshold-parity branch October 12, 2026 13:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[quality] CONTRIBUTING.md's e2e coverage threshold values are unpinned: ci.yml can ratchet while the documented recipe goes stale

1 participant