review-runs-tracking: 2026-04

[prql-bot]

Monthly tracking issue for below-threshold findings from review-runs analysis. Each run appends findings as a comment. Future runs read these to build cumulative evidence.

Do not close manually — a new issue is created each month.

[prql-bot]

Run $GITHUB_RUN_ID — 2026-04-09

Period: past 12 hours (2026-04-08T20:18Z to 2026-04-09T08:18Z)
Runs found: 28 total, 9 with session artifacts

Token usage

Workflow	Runs	Input	Output	Cache-Create	Cache-Read
tend-nightly	1	3.5K	13.3K	228K	2.6M
tend-review	3	87	7.8K	156.1K	1.7M
tend-notifications	5	77	5.4K	164.5K	1M
Total	9	3.7K	26.6K	548.7K	5.4M

Analyzed runs

Run ID	Workflow	Conclusion	Turns	Notes
24165406677	tend-review	failure	—	Checkout failure (infrastructure)
24165602166	tend-review	failure	9	API rate limit hit during pre-flight
24166017174	tend-review	success	32	Approved PR #5785 correctly
24166078115	tend-mention	failure	—	GH_TOKEN empty in verify step
24176715649	tend-nightly	success	103	Created PR #5786, commented on #5784
24176900893	tend-review	success	24	Reviewed self-authored PR #5786 (no self-approval, correct)

Finding: tend-mention GH_TOKEN empty on bot's own review events

• Evidence level: High (Critical — clearly wrong outcome)
• Occurrences this run: 1
• Structural/stochastic: Structural — pull_request_review events from the bot always trigger tend-mention, and the verify step always needs a token
• Run ID: 24166078115
• Detail: When the bot approved PR chore: fix tend 0.0.9 workflow drift and update skill #5785, the pull_request_review (submitted) event triggered tend-mention. The verify step uses secrets.PRQL_BOT_GITHUB_TOKEN as GH_TOKEN, which was empty, causing gh api calls to fail.
• Status: Already addressed by PR chore: update tend workflows #5786 (created by nightly run 24176715649). That PR adds bot-login filtering at the job if: level and downgrades the verify step token to github.token. CI passes. No additional action needed.

Finding: Nightly uvx not in PATH

• Evidence level: Medium
• Occurrences this run: 1
• Structural/stochastic: Structural — uvx is not installed on the runner
• Run ID: 24176715649
• Detail: uvx tend@latest init failed (exit 127). The nightly fell back to pip install tend successfully. Not a blocker, but adds noise and a wasted attempt on every run.
• Proposed change type: Upstream (tend plugin) — not actionable at repo level.

Finding: Nightly file survey token proportionality

• Evidence level: Low
• Occurrences this run: 1
• Structural/stochastic: Stochastic — depends on what the survey finds
• Run ID: 24176715649
• Detail: Three Explore subagents read 22 files totaling ~514K cache-read tokens. Found only pre-existing minor debt (.unwrap() in anchor.rs, dead code in Output.jsx, stale comment in test_all.py). The nightly correctly chose not to act. Not actionable at repo level — inherent to the nightly survey design.

Run 24278200312 — 2026-04-11

Period: past 24 hours (2026-04-10T08:03Z to 2026-04-11T08:03Z)
Runs found: 41 completed (1 nightly, 3 review, 3 ci-fix skipped, ~32 notifications, 1 review-runs)
Session artifacts analyzed: 9 (nightly, 3 review, 4 notifications, 1 review-runs)

Token usage

Workflow	Runs	Input	Output	Cache-Create	Cache-Read
tend-nightly	1	233	66.6K	583.8K	12.5M
tend-review	3	168	46.6K	353.6K	3.5M
tend-notifications	4	168	24.2K	351K	2.7M
tend-review-runs	1	84	18.3K	155.4K	2.5M
Total	9	653	155.8K	1.4M	21.3M

Outcomes

• Nightly 24276866429 — 28 min, 160 turns. Reviewed commits, closed no issues (correctly deferred), delegated file survey to subagent (22 Reads / 5 Greps / 1 Bash), created three PRs: chore: update tend workflows #5786 (tend workflow regen as new commit on existing PR branch), docs: fix grammar in ranges reference #5790 (ranges docs typos), refine: fix typo in positional_mapping comment #5791 (positional_mapping comment typo). All three still open, all CI green.
• Review runs for chore: update tend workflows #5786 — all three correctly stayed silent on self-authored PR. The larger review session (24276986866, 44 turns) did the full context load; the other two exited early after checking author+bot login.
• Review-runs 24233554789 — completed Step 3 analysis but never wrote out Step 6/7. The single assistant text block at session end is a corrupted token stream (560 chars of garbled base64-like output).
• Notifications — 4 sessions inspected, all no-op (no actionable items); the 20+ others produced no artifacts (same pattern).

Finding: nightly worktree recovery when existing PR targets the same drift

• Evidence level: Medium (1 occurrence, structural under a niche condition)
• Occurrences this run: 1 (new)
• Structural/stochastic: Structural — nightly Step 5 assumes it can create a fresh tend/update-workflows branch, but if a prior PR (e.g. chore: update tend workflows #5786) already exists with a different approach on chore/update-tend-workflows, the bot has to switch to updating that existing PR instead
• Run ID: 24276866429
• Detail: After uvx tend@latest init produced drift, bot found existing open PR chore: update tend workflows #5786 on chore/update-tend-workflows. It attempted gh pr checkout 5786 from the main checkout, which hit sandbox write restrictions on .git/config, leaving the working tree in a half-switched state. Recovery cost ~10 extra bash commands: stash, switch main, try git worktree add --detach, checkout -B from origin, merge origin/main, rerun uvx tend@latest init in the fresh worktree, commit, push. Eventually succeeded, but noisy.
• Gate assessment: Doesn't pass Gate 1 (need 2+ structural occurrences for a targeted fix to tend skill). Upstream issue — not actionable at repo level.

Finding: uvx not in PATH on nightly (recurrence)

• Evidence level: Medium (2 occurrences: prior 2026-04-09 + this run)
• Occurrences this run: 1
• Structural/stochastic: Structural — uv is not installed on the GitHub Actions runner; nightly has to pip install --target=$TMPDIR/pip-uv uv as a fallback every run
• Run ID: 24276866429
• Detail: Same as prior tracking entry. The bot now handles this inline with a pip-fallback detour (~4 extra commands). Still not actionable at repo level — this belongs upstream in the tend plugin's runner setup.

Finding: previous review-runs text output corruption

• Evidence level: Medium — single observation, but severe when it happens
• Occurrences this run: 1
• Structural/stochastic: Stochastic — model output glitch, unclear root cause
• Run ID: 24233554789
• Detail: Previous review-runs session has only one assistant text block in its entire JSONL — and its content is a 560-char stream of garbled tokens (<inviscsi:'AL4eycAll6SoI=1,pelm,s://ASIAI3RO41V44/...). Thinking blocks are normal and the 39 Bash commands all executed successfully, but no Step 6/7 summary was ever produced. Because the bot was almost entirely driven through thinking+tool-use (rather than user-visible text), the session still "succeeded" from the runner's perspective. Not acting — one-off stochastic lapse; no repo-level fix makes sense. If this recurs, investigate whether thinking-mode + tool-use can drop the final summary text block without a visible failure signal.

Finding: nightly wasted ScheduleWakeup call

• Evidence level: Low (1 occurrence, self-corrected)
• Occurrences this run: 1
• Structural/stochastic: Stochastic
• Run ID: 24276866429
• Detail: Bot called ScheduleWakeup mid-nightly, then immediately reasoned "actually that wakeup isn't right — I'm not in /loop mode". Self-corrected within the same turn. Not actionable — system prompt already makes the /loop constraint explicit.

Assessment

All findings below threshold for PR/issue creation. Nightly delivered its real work (3 useful PRs) despite the worktree friction; review runs were clean; notifications are steady-state noise. No repo-local skill or workflow changes warranted this cycle.

Run 24302125103 — 2026-04-12T08:17:33Z

Period: past 24 hours (2026-04-11T08:09Z to 2026-04-12T08:09Z)
Runs found: 52 completed (1 nightly, 1 review, 1 triage, 4 ci-fix skipped, 1 mention skipped, ~38 notifications, 1 review-runs [this run])
Session artifacts analyzed: 3 (nightly with 6 subagents, review, triage)

Token usage

Workflow	Runs	Input	Output	Cache-Create	Cache-Read
tend-notifications	7	151	20.9K	290.2K	2.3M
tend-nightly	1	0	0	0	0
tend-review	1	0	0	0	0
tend-triage	1	0	0	0	0
Total	10	151	20.9K	290.2K	2.3M

Note: nightly, review, and triage show 0 tokens in the report because the token-report script could not parse their session artifacts (likely a format mismatch). The nightly ran 25 min with 6 subagents; review ran 3 min; triage ran 2 min.

Outcomes

• Nightly 24300833294 — 25 min, success. Created PR #5792 (Annotate display fix + typos + stale docs) and issue #5793 (dead code findings for maintainer review). Correct restraint on issue Bot temporarily unavailable #5784 (no close without authorization). Clean, well-structured run.
• Review 24301240279 — 3 min, failure (false negative). Reviewed PR fix: correct Annotate display, fix typos, update stale docs #5792 correctly, verified all three changes, correctly skipped self-approval, monitored CI. Claude session completed successfully.
• Triage 24301252595 — 2 min, failure (false negative). Verified all three dead-code claims in issue Dead code in PL IR: unused fold_func_param, write-only prev_columns, unreachable display_ids #5793, posted correct triage comment. Claude session completed successfully.
• Notifications — all succeeded. 4 sessions had meaningful work (processing GitHub notifications); rest were no-op.
• CI-fix — 4 runs, all skipped (CI green on main).

Finding: RUN_STARTED_AT empty in tend action post-session cleanup (2 occurrences)

• Evidence level: Critical (clearly wrong outcome — runs marked as failed despite successful sessions)
• Occurrences this run: 2 (review run 24301240279, triage run 24301252595)
• Structural/stochastic: Structural — the tend action's post-session notification cleanup step uses $RUN_STARTED_AT in a jq filter, but the variable is empty, producing jq: parse error: Invalid numeric literal at line 1, column 2 and exit code 5.
• Detail: Both the tend-review and tend-triage runs completed their Claude sessions successfully (correct reviews, correct triage comments posted). The failure occurred in a post-session step that marks GitHub notifications as read. The RUN_STARTED_AT env var is shown as empty in the runner logs for both runs. The nightly run also has RUN_STARTED_AT empty but didn't hit this code path (likely no notification context for schedule-triggered runs).
• Impact: PR fix: correct Annotate display, fix typos, update stale docs #5792's review status check shows as FAILURE, which may block auto-merge or confuse maintainers.
• Gate assessment: Critical + structural = passes Gate 1 with 1 occurrence. However, this is an upstream bug in max-sixty/tend@v1, not fixable via repo-local skills or workflows. Not actionable at repo level.

Finding: Triage bot /tmp/claude/ directory miss (1 occurrence)

• Evidence level: Medium (single occurrence)
• Occurrences this run: 1
• Structural/stochastic: Stochastic — the bot tried cat > /tmp/claude/comment-5793.md before creating /tmp/claude/, then recovered with mkdir -p + Write tool. The comment was posted successfully.
• Detail: Minor stochastic lapse. The bot's recovery path worked correctly, and the triage outcome was unaffected.
• Gate assessment: Does not pass Gate 1 (need 5+ occurrences for stochastic). No action.

Assessment

No repo-local skill or workflow changes warranted. Both failures are upstream tend action bugs (the RUN_STARTED_AT issue). The nightly run quality was good — real bug fix (Annotate display), correct dead-code triage, appropriate restraint on issue management.

---

Run 24333536109 — 2026-04-13T08:33:37Z

Period: past 24 hours (2026-04-12T08:30Z to 2026-04-13T08:33Z)
Runs found: 43 total (37 tend-notifications no-ops, 3 tend-ci-fix skipped, 3 tend-review, 1 tend-nightly, 1 tend-mention). Sessions analyzed: 4.

Token usage

Per-run token report returned zeros across the board — token-usage.json files in session artifacts contained all zeros for every run. The session JSONL files no longer emit type: "result" records (only assistant/user/attachment/queue-operation/last-prompt/pr-link), so the post-job jq query that sums result.usage finds nothing. This is the same upstream tend issue: the usage summary is stale after the session log schema changed.

Analyzed runs

Run ID	Workflow	Conclusion	Duration	Notes
24330252250	tend-nightly	success	7 min	Created PR #5795 fixing real emacs/nano grammar bugs (already merged as 17695574)
24330464957	tend-review	success	4 min	Reviewed self-authored PR #5795; correctly skipped self-approval, verified fix against lexer source, added +1 reaction
24306309406	tend-review	failure (false negative)	3 min	Approved PR #5794 docs change correctly, CI monitored; post-job jq parse error
24306354969	tend-mention	failure (false negative)	1 min	Correctly exited silently (bot's own approval review); post-job jq parse error
24306303579	tend-review	cancelled	—	Superseded by 24306309406; normal

Bot behavior quality

• Nightly: Excellent. Surveyed 33 files, found real bugs in editor grammar regex & keyword lists, created single-concern PR fix(grammars): correct emacs regex and align keywords with lexer #5795 with all fixes grounded in lexer source (prqlc/prqlc-parser/src/lexer/mod.rs:197-206), CI green, merged.
• Review of PR fix(grammars): correct emacs regex and align keywords with lexer #5795 (self-authored bot PR): Correct restraint — verified all three changes against source, added +1 reaction, stayed silent on approval since self-authored.
• Review of PR docs: Add Kakoune integration to CHANGELOG #5794 (human docs PR): Correct approval of trivial CHANGELOG addition, CI monitored to completion.
• Mention (bot's own approval of docs: Add Kakoune integration to CHANGELOG #5794): Correctly identified self-conversation loop, exited silently.

Finding: jq parse error in tend post-job notification-cleanup (2 occurrences, recurring)

• Evidence level: Critical (runs marked failed despite successful sessions)
• Occurrences this run: 2 (runs 24306309406, 24306354969) — same root cause as 2 occurrences in the 2026-04-10 entry (runs 24301240279, 24301252595)
• Structural/stochastic: Structural — upstream max-sixty/tend@v1 post-job step runs gh api notifications | jq -r --arg started "$RUN_STARTED_AT" ..., and jq fails with Invalid numeric literal at line 1, column 2 / exit 5. RUN_STARTED_AT is logged as empty in the step env both times.
• Cumulative occurrences: 4 across two review runs. Pattern is reliable and predictable.
• Gate assessment: Passes Gate 1 (critical, structural, repeated). Does not pass Gate 2 for any repo-local change — the bug is in tend action's post-job script, not reachable from .claude/skills/, .config/tend.toml, or CLAUDE.md. Not actionable at repo level. Already documented in this tracking issue on 2026-04-10; carrying forward as evidence until tend upstream resolves.

Finding: token usage artifacts return zeros (7 occurrences this window)

• Evidence level: Medium (diagnostic-only impact; no wrong outcome)
• Occurrences this run: 7 (every run with an artifact this window)
• Structural/stochastic: Structural — session JSONL files contain no type: "result" records, so tend's post-job jq 'select(.type == "result")' always sums to 0 tokens / $0.00 cost.
• Gate assessment: Also an upstream tend/claude-code-action schema mismatch. Not actionable at repo level. The review-runs token report section will be misleading until upstream catches up.

Assessment

No repo-local skill/workflow changes warranted. Bot decision quality was uniformly good — one real bug-fix PR merged, correct restraint on self-review, correct silent exit on self-conversation trigger. Both "failures" are the known upstream post-job jq issue. Recording both findings here for cumulative evidence; no PRs or issues to open against this repo.

Run 24388549499 — 2026-04-14T08:25:22Z

Period: past 24 hours ending 2026-04-14T08:25:22Z
Runs found: 52 total (mostly tend-notifications polling), 7 with real activity reviewed

Token usage

Workflow	Runs	Input	Output	Cache-Create	Cache-Read	Cost
tend-nightly	1	49	11.7K	184.5K	2.5M	$2.73
tend-mention	5	83	4.5K	91.1K	954.5K	$2.36
tend-review	5	21	689	1.7K	363.8K	$3.77
tend-notifications	2	22	2.4K	73K	282.9K	$0.66
Total	13	175	19.4K	350.5K	4.1M	$9.52

Token reporting is working again this window — prior runs (2026-04-10 through 2026-04-13) had observed all zeros due to upstream schema mismatch. No additional note needed; upstream appears resolved.

Bot behavior quality

• Nightly (run 24388549499 → 24385422030): 50 turns, clean. Surveyed 29 files, checked open issues, regenerated tend workflows (no diff). Correctly declined to open PRs for judgment-call items (UnOp::Add→Pos TODO, stale README date, commented-out ClickHouse/GlareDB runners verified still pending upstream).
• Review of chore: pre-commit autoupdate #5796 (pre-commit autoupdate, 24356338125): 2-turn approve + CI monitor. Correct.
• Review of chore: bump the patch group with 2 updates #5797 (libc/tokio patch bump, 24360076677): 35-min run. Approved correctly, then monitored CI through a concurrency-cascade replacement. Handled cancellation-cascade pattern correctly per running-in-ci guidance.
• Review of chore: bump mysql from 27.0.0 to 28.0.0 #5798 (mysql 27→28, 24362786554): 21-min run. Diagnosed CI failure, bot-authored fix commits verified, approved. Clean analysis grounded in protocol.rs:61.
• Notifications polls: Both real-work runs recognized merged PRs as no-op and marked-read correctly.

Finding: tend-mention and another bot run raced to push the same fix on #5798

• Evidence level: Low
• Occurrences this run: 1 (24362049764)
• Structural/stochastic: Partially structural — when a tend-review leaves a "I'll push a fix" review, a tend-mention is triggered by the pull_request_review event and tries to fulfil it. If the review's own workflow (or another run) pushes the fix first, the mention's push gets fetch first rejected.
• Detail: Mention run took 13 min and $1.47 producing an identical commit (chore: bump connector_arrow to 0.11.0 and rusqlite to 0.39.0) to one already at 36390168 when it tried to push. The bot detected the rejection, verified the remote commit matched its intent, and exited silently without opening a duplicate PR or comment. Outcome was correct, just wasteful.
• Gate assessment: Fails Gate 1 (only 1 occurrence, low evidence). No fix proposed; recording for cumulative evidence.

Finding: long tend-mention queue delays (up to 7 min)

• Evidence level: Low (observational)
• Occurrences this run: 3 (24360153231 444s, 24360724965 311s, 24362972753 265s)
• Structural/stochastic: Structural — GitHub Actions queue depth + concurrency groups. Not a bot-decision issue.
• Detail: The running-in-ci preamble prompt tells the bot when it's been queued; the bot then correctly re-checks for already-addressed triggers and exits silently. Queue delay itself is not a behavior problem.
• Gate assessment: Not actionable at repo level (platform-level queuing). Recording for awareness only.

Assessment

All clear. No repo-local skill/workflow changes warranted. Bot decision quality was consistent across all sampled runs: correct approvals on trivial deps, accurate CI-failure diagnosis on #5798 with a bot-authored fix that merged, correct silent exits on duplicate-trigger mentions, clean nightly sweep. The one observed waste (duplicate-push race on #5798) ended with the correct silent-exit outcome; a single occurrence is below the evidence threshold for action.