Repository navigation
Description
Description
The DevHub GitHub Action gains the ability to surface the two new comparison figures — page states newly scanned and page states no longer scanned — alongside what it reports today. This runs on the Action's own release schedule and gates nothing else in the epic.
The Action currently reads the page-state difference and reports it as "N more page states" / "N fewer page states" / "No change in page state count", and exposes it as a workflow output that customers wire into their own CI steps. That output must keep working exactly as it does.
Be careful with the existing numeric handling. The Action reads the difference figure with jq -r and compares it numerically. An absent field yields the string null, which fails that comparison outright rather than degrading quietly. Any new figure read the same way needs a default before it reaches a numeric test.
Contract
| Triggering action | The Action runs in a customer workflow against a backend publishing the new comparison figures. |
| Observable result | The Action reports both new figures alongside what it reports today. Where the figures are absent from the response it falls back to today's reporting without erroring and never prints null. Every existing workflow output keeps its name, type and meaning, and a currently released Action runs unchanged against the updated backend. |
| Proving test | A test that runs the Action against a response carrying the new figures and asserts both are reported, paired with one against a response lacking them that asserts today's summary and no null. Nothing in the automation suite drives the Action today, so this needs a new harness capability — see Resources. |
Non-Goals
- Computing or publishing the figures — walnut#16559 and walnut#16561.
- Changing any existing workflow output's name, type or meaning. Additive only.
- Gating the rest of the epic. This trails the backend and is deliberately excluded from the QA ticket's blocking dependencies — the feature is testable in DevHub without it.
Interface Surface
Inputs
| Input | Valid values | Invalid value | Absent | Malformed | Empty result |
|---|---|---|---|---|---|
| The two new comparison figures in the backend response | Any integer, including zero and negative | — | Falls back to today's reporting without erroring | A non-numeric value must not reach a numeric comparison — default it first | Zero is a real value and is reported as such, never treated as missing |
Outputs
| Output | Complete field or header list | Null / empty / omitted cases |
|---|---|---|
| The Action's workflow outputs | Every output it publishes today, plus any new one for the two figures | Every existing output keeps its name, type and meaning. No output ever carries the string null |
| The Action's step summary | Today's page-state reporting, plus both new figures when present | With the figures absent, today's summary exactly |
Stated rules, credentials and references
| Rule or access case | What proves it |
|---|---|
| "Every existing workflow output keeps its name, type and meaning" | Each existing output asserted by name and type against the updated backend |
"Never print null, and never fail the step because an optional figure was missing" |
A run against a response with the figures absent |
| Credential absent / malformed / valid-but-unauthorized | Unchanged by this work; the Action's existing handling applies |
| Repeat call | Unchanged; the Action is a single-shot step |
| Equivalence with what this replaces | A currently released Action run against a backend with both flags on, compared against its behavior today |
| Scale | Not applicable — one comparison per run |
Surfaces this ships on: the Action's step summary and its workflow outputs. One scenario asserts the difference figure the Action reads equals newly scanned − no longer scanned shown in the interface for the same commit.
Axes that vary independently: figures present or absent × difference positive, negative or zero. The absent case changes what the caller receives, so it gets its own scenario; the three difference signs are the same code path at this surface and are table-driven.
Unchanged Behavior
- Every existing workflow output — same name, same type, same meaning. Customers wire these into their own CI steps.
- A currently released Action — runs unchanged against a backend with both flags on.
- Today's page-state reporting wording — still produced where the new figures are absent.
- The build-failing threshold behavior — unchanged by this ticket, though the underlying numbers move because of walnut#16552.
Acceptance Criteria
- Given an unchanged version of the Action, when it runs against the updated backend, then it behaves exactly as it does today
- Given the updated Action, when the new figures are present in the response, then it reports both
- Given the updated Action, when the new figures are absent from the response, then it falls back to today's reporting without erroring and never prints
null - Given a figure of zero, when the Action reports, then zero is reported as a value rather than treated as missing
- Given the same commit, when the difference figure the Action reads is compared against the interface, then it equals
newly scanned − no longer scannedshown there - Every existing workflow output keeps its name, type and meaning
- The README's outputs table documents any new outputs
- I wrote the proving test first, ran it, and saw it fail for the reason it should — with the failure output recorded in the PR
- The full relevant suite runs green on my machine, not only in CI, and the passing tally is recorded in the PR
- If my work changed anything the epic's QA ticket describes, I have updated it
- I have double-checked the Definition of Done for this ticket
Standards that apply
Resources
- Full spec:
prds/url-matching-workplan.md— Chunk 7 - Epic: EPIC: Unique URLs for web projects
- Harness gap: nothing in
dequelabs/axe-developer-hub-automationdrives the Action against a deployed backend today. Running it is a new capability the epic's QA ticket also depends on
Activity