Repository navigation
test: assert every data-refresh script is scheduled or recorded as exempt - #1358
Merged
Merged
Conversation
tests/workflow-scripts.test.mjs demands that every validate:*/check:* gate be run by some workflow or recorded in GATES_NOT_RUN_BY_CI with a reason. Nothing made the same demand of the collect:*/fetch:*/import:*/generate:* scripts that write the datasets those gates then validate. tests/automation-workflow-ordering.test.mjs names the same four namespaces, but only constrains the order of steps inside a workflow that already runs one of them, so a refresh script no workflow runs at all never appears in any job and that file has nothing to order. The failure that leaves open is silent: a collector stops being scheduled, its dataset stops being refreshed, and its validator keeps passing, because stale data is still valid data. This adds the guard, following the GATES_NOT_RUN_BY_CI convention. It resolves npm run chains through package.json so generate:members counts as wired via import:architectures rather than forcing a false exemption, and records the two deliberately manual collectors -- collect:case-studies and collect:launch-metrics -- in REFRESH_NOT_RUN_BY_CI with the reasons that until now lived only in prose. Signed-off-by: quality <quality@hive.kubestellar.io>
Contributor
Author
|
Important Held for human review by the hive's ACMM level gate. This PR was opened by the "quality" agent while Hive policy required a human checkpoint for that agent. Non-outreach agents are held at ACMM L3–L5; the Hive will keep the |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Test Improvement
Adds
tests/refresh-script-wiring.test.mjs. The diff is one new file undertests/— no behaviour, CI definition, or data file changes.tests/workflow-scripts.test.mjsalready asserts that everyvalidate:*/check:*gate is run by CI or recorded inGATES_NOT_RUN_BY_CIwith a written reason. Nothing made the same demand of the scripts on the other side of that contract — thecollect:*,fetch:*,import:*andgenerate:*scripts that write the datasets those gates then validate.tests/automation-workflow-ordering.test.mjsnames the same four namespaces, but only constrains the order of steps in a job that already runs one of them, so a refresh script nothing schedules never appears in any job and that file has nothing to order. The failure that leaves open is silent: a collector stops being scheduled, its dataset stops being refreshed, and its validator keeps passing — because stale data is still valid data.What the new file asserts
collect:*/fetch:*/import:*/generate:*script is scheduled, or recorded inREFRESH_NOT_RUN_BY_CIwith a reason.npm runchains throughpackage.json, sogenerate:memberscounts as wired viaimport:architecturesrather than forcing a false exemption. A dedicated test pins that this transitive step is load-bearing — it fails ifgenerate:membersever becomes directly reachable, so the guard cannot quietly stop proving anything.package.jsonno longer defines, carries no reason, names a script outside the refresh namespaces, or names a script that is in fact scheduled.Exemptions recorded
collect:case-studiesandcollect:launch-metricsare unscheduled on purpose. That decision previously lived only in prose (scripts/collect-case-studies.mjs,docs/resources/case-studies.md,CONTRIBUTING.md,ROADMAP.md); it is now on the record where a test can tell it apart from an accident.Verification
node --test tests/refresh-script-wiring.test.mjs— 8 passed, 0 failed.npm run collect:radar-reportsfromrefresh-radar-reports.yml→ 2 tests fail, including the main wiring assertion.npm run test:unit:coverage:check— exit 0 with the new file included (src files100.00% lines / 100.00% regions).npx prettier --check— clean.Related Issue
Closes #1357
Filed by quality agent (hold-gated mode). Human review required.
— hive: agent=quality backend=copilot model=claude-opus-5 copilot=1.0.88