Skip to content

Show each scenario's totals in the coverage comment - #38

Merged
NullVoxPopuli merged 1 commit into
mainfrom
nvp/coverage-comment-tables
Aug 12, 2026
Merged

Show each scenario's totals in the coverage comment#38
NullVoxPopuli merged 1 commit into
mainfrom
nvp/coverage-comment-tables

Conversation

@NullVoxPopuli

Copy link
Copy Markdown
Owner

Note

Made with Claude Code. Apologies if I've stepped on how you wanted this to read — the layout is a guess, and the security reasoning below is the part most worth a second pair of eyes.

The comment linked the artifact zips but said nothing about what was in them, so seeing whether a PR moved coverage meant downloading and unzipping up to four files. This inlines each scenario's totals, collapsed behind <details> so the comment stays short, with the artifact table kept underneath.

Rendered from real scenario output:

v2-addon-js
Metric % Covered / Total
Lines █████████████████░░░ 85.41% 41 / 48
Statements █████████████████░░░ 85.41% 41 / 48
Functions ████████████░░░░░░░░ 60.00% 3 / 5
Branches ████████████████████ 100.00% 6 / 6
vite-app-js
Metric % Covered / Total
Lines ███████████████████░ 94.24% 180 / 191
Statements ███████████████████░ 94.24% 180 / 191
Functions █████████████░░░░░░░ 62.50% 5 / 8
Branches █████████████░░░░░░░ 65.00% 13 / 20

The security bit

The header comment on this workflow makes a specific promise — workflow_run hands it a write-capable token, and that's only safe because it "executes no code from the PR — no checkout, only GitHub API calls". Reading the numbers means touching PR-produced data, so I wanted to be careful not to quietly weaken that.

What it now does: downloads the coverage artifacts (actions/download-artifact with run-id, so still no checkout) and JSON.parses them. That's data handling, not code execution — nothing from the PR is installed, built, or run.

The parsed JSON is still untrusted, so only numbers are taken out of it, each coerced through:

const num = (value) => {
  const n = Number(value);
  return Number.isFinite(n) && n >= 0 ? n : 0;
};

No string from coverage-summary.json reaches the comment body — scenario names come from the artifact names, which CI controls. So a fork can't smuggle markdown, a link, or a --> through a crafted summary file. The bar is clamped to 0–20 characters too, so a wild pct can't produce a giant row.

I updated the header comment to say all this rather than leaving the old "only GitHub API calls" wording, which would no longer be strictly accurate.

Also

Adds an artifact for coverage-parallel. The parallel scenario reuses the vite-app-js app but writes to its own folder, so it wasn't being uploaded at all — meaning the one scenario that exercises multi-browser merging had no downloadable report.

Testing

I dry-ran the rendering against real coverage-summary.json files from two scenarios (that's the output above). A scenario whose artifact has no summary — failed before writing one, or expired — falls through to just its zip link rather than erroring. oxlint + oxfmt clean; both workflow files parse.

What I can't test locally is the workflow_run trigger itself, so the download step's run-id wiring is only exercised once this is on main.

The comment linked the artifact zips but said nothing about what was in them,
so seeing whether a PR moved coverage meant downloading and unzipping four
files. Inline each scenario's totals as a table, collapsed behind <details> so
the comment stays short, with the artifact links kept underneath.

The workflow still executes nothing from the PR: it downloads the coverage
artifacts and JSON.parses them, which is data handling rather than code
execution. Every value read out of that JSON is coerced to a finite,
non-negative number before it reaches the comment, so a fork cannot inject
markdown through a crafted coverage-summary.json.

Also uploads the parallel scenario's coverage-parallel folder, which had no
artifact of its own.
@github-actions

Copy link
Copy Markdown
Contributor

Coverage reports

Scenario Artifact
v2-addon-js coverage-v2-addon-js.zip
vite-app-js coverage-vite-app-js.zip
vite-app-js-parallel coverage-vite-app-js-parallel.zip
vite-app-using-v2-addon-js coverage-vite-app-using-v2-addon-js.zip

Each artifact is the scenario's coverage/ folder (HTML report, text/JSON summaries, raw V8 snapshot). Unzip and open index.html.

From this CI run for 7dc5891.

@NullVoxPopuli
NullVoxPopuli merged commit fe7c609 into main Aug 12, 2026
5 checks passed
@NullVoxPopuli
NullVoxPopuli deleted the nvp/coverage-comment-tables branch August 12, 2026 18:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant