feat(reporter): print a dashboard link per failed test - #429
Open
PhenX wants to merge 5 commits into
Open
Conversation
Add GET /test-runs/:id/locate so a link can be built from what the reporter knows before execution ids exist (run id, spec file, title, retry, project) and redirect to /test-run-cases/:id. An unknown test renders a readable 404 page; a signed-out visitor is sent through /login and back to the link. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0145ws9TJpSi3vjZfzdBmB4c
Print one `✗ <title> → <url>` line per test whose final attempt failed, as soon as the run id is known (immediately while streaming, after the submit in batch mode). The link targets the dashboard's locate route so it needs no execution id. GitHub Actions gets the same list in the job summary (capped at 20) and a piwi_failed_count step output; the GitLab dotenv and the JSON output file carry the count and the failures too. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0145ws9TJpSi3vjZfzdBmB4c
testInfo.attach(name, { body }) attachments were skipped silently because
only path-backed ones were collected. Stage a body as a temp file under
os.tmpdir() for the upload (removed when the run ends) and apply one size
ceiling to inline and path-backed attachments alike, warning once per
attachment when it is skipped.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0145ws9TJpSi3vjZfzdBmB4c
…head Slack and email run notifications linked each failure to the test's history page even though the payload carries the execution id; link to /test-run-cases/:id and fall back to the history page only without one. The excerpt embedded in notifications, cluster alerts and pull-request comments was the raw first characters of the error, so a timeout read as a bare "Timeout 30000ms exceeded" with the call log cut off. Build it from the message head shared with the fingerprint, and append the last call-log state line when the head is only a timeout. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0145ws9TJpSi3vjZfzdBmB4c
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0145ws9TJpSi3vjZfzdBmB4c
Contributor
Coverage Report for Reporter (./packages/reporter)
File Coverage
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Contributor
Coverage Report for Application (./apps/application)
File Coverage
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
The reporter-and-alerts slice of the failing-run data audit (quick wins 8 and 9): a developer whose test failed should get a link to that exact execution where they already are — the terminal, the CI summary, the Slack/email alert, the pull-request comment — and the quoted error should say what Playwright was waiting on, not just that a timeout elapsed.
8a — Per-failure links from the reporter
apps/application/server/routes/test-runs/[id]/locate.get.ts(new) —GET /test-runs/:id/locate?file=…&title=…&retry=…&browser=…resolves an execution from what the reporter knows before ids exist and redirects to/test-run-cases/:id.retry/browserare preferences (exact attempt when persisted, else the failing one, else the latest), an unknown test renders a readable 404 page with a link to the run, and a signed-out visitor goes through/loginand back. The events route returns no execution ids to the reporter, so this is the deterministic-link approach the audit suggested. It lives underserver/routes/(notserver/api/), so the demo router check needs no mirror.apps/application/app/pages/login.vue— honours a same-origin?redirect=after sign-in so those links survive an auth-enabled dashboard.packages/reporter/src/internal/support/failure-links.ts(new),src/public/reporter.ts,src/internal/submit/run-submitter.ts— one[Piwi Dashboard] ✗ <title> → <url>line per test whose final attempt failed (result.retry >= test.retries), printed the moment the streaming run id is known, or after the submit in batch mode; never twice.apps/application/tests/run-locate.spec.ts(new, projectrun-locate-testregistered inshared/test-project-names.ts),packages/reporter/tests/failure-links.spec.ts(new).8b — GitHub step summary and outputs
packages/reporter/src/internal/support/ci-output.ts—$GITHUB_STEP_SUMMARYlists the failed tests with their links under the existing "Piwi test run" heading (20 max, then+N more);piwi_failed_countjoins the step outputs. The GitLab dotenv gainsPIWI_FAILED_COUNT, and the JSON output file gainsfailedCount+failuresso any CI can read the same list.packages/reporter/tests/ci-output.spec.tsextended.8c — Body-only attachments
packages/reporter/src/internal/files/file-handler.ts—testInfo.attach(name, { body })attachments are staged as temp files underos.tmpdir()(one file per attachment, reused across the repeated lookups the upload paths make, removed inonEnd). One size ceiling (500 MB, the dashboard's default multipart limit) applies to inline and path-backed attachments alike; an oversized one is skipped with a single warning naming it instead of failing the whole upload. Applying the cap to path-backed files too is a small behaviour change: before, a >500 MB video made the multipart upload fail with a 413.packages/reporter/tests/file-handler.spec.ts(new).9a — Alerts link to the execution
apps/application/shared/notification-events.ts—failureTargetPath():/test-run-cases/<executionId>, falling back to/test-cases/<testCaseId>.server/utils/notifications/dispatch.ts(Slack) andserver/utils/email.ts(run email, HTML + text) use it. The digest renderers link per event to the run, which is unchanged and correct for a digest line.9b — Excerpts quote the message head
shared/error-fingerprint.tsexportsextractMessageHead;shared/notification-events.tsaddserrorExcerpt(): the head (beforeCall log:and the first stack frame, ≤5 lines), with the lastwaiting for …/locator resolved to …call-log line appended when the head is only a bare timeout, ANSI stripped, capped.buildTopFailuresand thecluster.newsample excerpt use it.server/utils/scm/pr-feedback.tsuses the same helper withPR_EXCERPT_MAX(200, exported fromshared/pr-feedback.ts) so the pull-request comment quotes failures the same way.tests/unit/notification-events.test.ts,tests/unit/pr-feedback.test.tsextended.Docs (same commits)
apps/docs/ci.md(terminal lines, step summary, new outputs, JSON file shape),apps/docs/reporter.md(streaming step, body attachments and the size ceiling),apps/docs/notifications.md(errorExcerptsemantics and links),packages/reporter/README.md,packages/reporter/ARCHITECTURE.md.Nothing was left out. The audit document itself is not on the branch.
How was it tested?
packages/reporter/):reporter:build,reporter:typecheck,reporter:lint,reporter:format:check,reporter:test— 46 files, 674 tests passing.apps/application/):app:typecheck,app:lint,app:format:check,app:test:unit(120 files, 1622 tests),app:check:demo.tests/run-locate.spec.ts,tests/notifications.spec.ts,tests/email-notifications.spec.ts,tests/browser-notifications.spec.tsagainst a dev server — 12 passed, the 46 auth-enabled cases skip outside CI as configured.npx commitlint --from main --to HEADpasses for every commit.Checklist
type(scope): subject)apps/docs/, README, or reporter README)🤖 Generated with Claude Code
https://claude.ai/code/session_0145ws9TJpSi3vjZfzdBmB4c
Generated by Claude Code