Repository navigation
ci-status never fails in a merge group, holding the queue until timeout #1682
Description
Merge queue entries whose checks have already failed stay AWAITING_CHECKS instead of being ejected, because the ci-status aggregate check never reports a failure.
What happens
Observed on the queue with #1645 at position 1 and #1649 at position 2. Both merge group SHAs had three checks concluded failure (coverage, lint-xtask, test-inference) ~40 minutes earlier, yet ci-status was still in_progress on both and neither entry had been ejected.
Cause
.github/workflows/ci-status.yaml polls sibling check runs in a while (true) loop. The loop has exactly one non-exceptional exit — every check green. On failure it warns and loops:
if (failed.length > 0) {
for (const cr of failed) {
core.warning(`${cr.name} concluded with: ${cr.conclusion} — waiting for re-run`);
}
core.info('Re-run the failed job(s) and ci-status will pick up the result automatically.');
await new Promise(r => setTimeout(r, 30000));
continue;
}There is no core.setFailed() anywhere in the script.
Waiting for a re-run is a deliberate choice for pull_request: a human sees the red job, re-runs it, and ci-status picks up the new conclusion without needing a re-run of its own. The premise does not hold for merge_group. The gh-readonly-queue/* ref is ephemeral, nobody is watching it, and no re-run is coming. The workflow handles both events identically.
Impact
ci-status is the sole aggregate required status check in the main ruleset, and the merge queue rule sets check_response_timeout_minutes: 60. So GitHub abandons the entry after 60 minutes while the job itself keeps running to its timeout-minutes: 180. A doomed entry costs ~60 minutes of queue time instead of failing at ~18 minutes, when the first check reported.
This is recurring, not a one-off. Merge-group ci-status run history shows a clear signature — healthy runs finish in ~28-30 min, wedged ones land on exactly 3h00m, the timeout-minutes guillotine:
10-09 10:02:42 -> 13:02:55 cancelled pr-1648 (3h00m)
10-09 11:34:02 -> 14:34:18 cancelled pr-1509 (3h00m)
10-09 13:19:51 -> 16:20:08 cancelled pr-1648 (3h00m)
10-09 23:11:15 -> 02:11:19 cancelled pr-1648 (3h00m)
...
10-12 13:11:14 -> 13:41:03 success pr-1645 (0h30m)
#1648 burned four of those slots in one day.
Expected
In a merge group, the first failed check should fail ci-status immediately so GitHub ejects the pull request and starts building the next entry. The pull_request wait-for-re-run behavior should be left alone.
Note on the triggering failure
The failure that exposed this was legitimate — the merge queue was doing its job. #1645 bumps hardcoded inventory counts in tests/utils/src/inference_fixture/coverage.rs, but another PR landed the same 69 -> 70 bump while it sat in review, so rebased it should read 71. A semantic merge conflict, invisible to that PR's own checks. #1649 fails only because it is speculatively stacked behind #1645. That is separate from this bug.
Activity
fix(ci): fail ci-status immediately when a merge group has failed che…