Skip to content

Surface the new comparison figures in the DevHub GitHub Action #195

Closed

Description

@dequejenn

Description

The DevHub GitHub Action gains the ability to surface the two new comparison figures — page states newly scanned and page states no longer scanned — alongside what it reports today. This runs on the Action's own release schedule and gates nothing else in the epic.

The Action currently reads the page-state difference and reports it as "N more page states" / "N fewer page states" / "No change in page state count", and exposes it as a workflow output that customers wire into their own CI steps. That output must keep working exactly as it does.

Be careful with the existing numeric handling. The Action reads the difference figure with jq -r and compares it numerically. An absent field yields the string null, which fails that comparison outright rather than degrading quietly. Any new figure read the same way needs a default before it reaches a numeric test.

Contract

Triggering action The Action runs in a customer workflow against a backend publishing the new comparison figures.
Observable result The Action reports both new figures alongside what it reports today. Where the figures are absent from the response it falls back to today's reporting without erroring and never prints null. Every existing workflow output keeps its name, type and meaning, and a currently released Action runs unchanged against the updated backend.
Proving test A test that runs the Action against a response carrying the new figures and asserts both are reported, paired with one against a response lacking them that asserts today's summary and no null. Nothing in the automation suite drives the Action today, so this needs a new harness capability — see Resources.

Non-Goals

  • Computing or publishing the figures — walnut#16559 and walnut#16561.
  • Changing any existing workflow output's name, type or meaning. Additive only.
  • Gating the rest of the epic. This trails the backend and is deliberately excluded from the QA ticket's blocking dependencies — the feature is testable in DevHub without it.

Interface Surface

Inputs

Input Valid values Invalid value Absent Malformed Empty result
The two new comparison figures in the backend response Any integer, including zero and negative — Falls back to today's reporting without erroring A non-numeric value must not reach a numeric comparison — default it first Zero is a real value and is reported as such, never treated as missing

Outputs

Output Complete field or header list Null / empty / omitted cases
The Action's workflow outputs Every output it publishes today, plus any new one for the two figures Every existing output keeps its name, type and meaning. No output ever carries the string null
The Action's step summary Today's page-state reporting, plus both new figures when present With the figures absent, today's summary exactly

Stated rules, credentials and references

Rule or access case What proves it
"Every existing workflow output keeps its name, type and meaning" Each existing output asserted by name and type against the updated backend
"Never print null, and never fail the step because an optional figure was missing" A run against a response with the figures absent
Credential absent / malformed / valid-but-unauthorized Unchanged by this work; the Action's existing handling applies
Repeat call Unchanged; the Action is a single-shot step
Equivalence with what this replaces A currently released Action run against a backend with both flags on, compared against its behavior today
Scale Not applicable — one comparison per run

Surfaces this ships on: the Action's step summary and its workflow outputs. One scenario asserts the difference figure the Action reads equals newly scanned − no longer scanned shown in the interface for the same commit.

Axes that vary independently: figures present or absent × difference positive, negative or zero. The absent case changes what the caller receives, so it gets its own scenario; the three difference signs are the same code path at this surface and are table-driven.

Unchanged Behavior

  • Every existing workflow output — same name, same type, same meaning. Customers wire these into their own CI steps.
  • A currently released Action — runs unchanged against a backend with both flags on.
  • Today's page-state reporting wording — still produced where the new figures are absent.
  • The build-failing threshold behavior — unchanged by this ticket, though the underlying numbers move because of walnut#16552.

Acceptance Criteria

  • Given an unchanged version of the Action, when it runs against the updated backend, then it behaves exactly as it does today
  • Given the updated Action, when the new figures are present in the response, then it reports both
  • Given the updated Action, when the new figures are absent from the response, then it falls back to today's reporting without erroring and never prints null
  • Given a figure of zero, when the Action reports, then zero is reported as a value rather than treated as missing
  • Given the same commit, when the difference figure the Action reads is compared against the interface, then it equals newly scanned − no longer scanned shown there
  • Every existing workflow output keeps its name, type and meaning
  • The README's outputs table documents any new outputs
  • I wrote the proving test first, ran it, and saw it fail for the reason it should — with the failure output recorded in the PR
  • The full relevant suite runs green on my machine, not only in CI, and the passing tally is recorded in the PR
  • If my work changed anything the epic's QA ticket describes, I have updated it
  • I have double-checked the Definition of Done for this ticket

Standards that apply

Resources

Activity

  1. added
    QA: requiredThis ticket requires QA signoff
    DesignSignoff: noneThis ticket does not require design signoff
    Docs: noneThis ticket does not require docs changes/additions
    on Sep 3, 2026
  2. added theissue type on Sep 3, 2026
  3. added
    QA: hold for epicTicket cannot be tested alone, needs the rest of the epic to be done first
    and removed
    QA: requiredThis ticket requires QA signoff
    on Oct 12, 2026
  4. self-assigned this
    on Nov 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

DesignSignoff: noneThis ticket does not require design signoffDocs: noneThis ticket does not require docs changes/additionsQA: hold for epicTicket cannot be tested alone, needs the rest of the epic to be done firstteam-resultsResults team ticket

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions