ci(qemu): skip bench tests under TCG instead of capping them a third time - #208
Open
lunarthegrey wants to merge 1 commit into
Open
ci(qemu): skip bench tests under TCG instead of capping them a third time#208lunarthegrey wants to merge 1 commit into
lunarthegrey wants to merge 1 commit into
Conversation
…aningful and safe Third round against the 5.15 soft-lockup flake (PR #206 2026-08-25, PR #207 2026-08-26: watchdog splat storm from bench_custom_fib_* timing out the job). The first two rounds capped PACKETFRAME_BENCH_QUICK harder (#82 per-syscall repeat, then a ~400-execution total budget) and slow TCG runners still straddled the 22 s watchdog, because bpf_test_run is uninterruptible and hosted-runner TCG speed varies too much for any fixed budget to hold. Bench numbers under emulation were never meaningful, so stop running them there: a PACKETFRAME_BENCH_SKIP=1 knob (same skip-print idiom as the BPF-stub gates) set only in qemu-verifier.yml's in-VM command. The benches keep running full-mode where they mean something — ci.yml's native-speed sudo step and the hardware-artifacts SAFE suite on routers. Belt-and-braces, the guest now boots with watchdog_thresh=60 so a stalled fixture test_run can't reignite the splat storm. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The qemu-verifier 5.15 leg soft-lockup flake struck twice in two days (2026-08-25 on #206, 2026-08-26 on #207):
bpf_prog_test_runexecutes uninterruptibly, and under TCGbench_custom_fib_*stalls a vCPU past the 22 s watchdog; the per-fire ~35-line splat storm on the emulated serial console then times the job out at 20 min. Each hit costs a manual rerun.This is the third round against this flake. #82 capped per-syscall repeat under
PACKETFRAME_BENCH_QUICK; the follow-up capped total volume to ~400 executions per bench. Slow hosted runners still trip the watchdog, because TCG per-execution cost varies too much across runners for any fixed budget to be both meaningful and safe.What
PACKETFRAME_BENCH_SKIP=1: the three bench tests early-return with a skip print (same idiom as the BPF-stub gates). Set only in qemu-verifier.yml's in-VM command — bench numbers under emulation were never meaningful.QUICKset) and the hardware-artifacts SAFE suite on routers (run-tests.shincludesbench, no skip env).watchdog_thresh=60(virtme-ng --append), since any uninterruptibletest_runin the fixture suites can stall a TCG vCPU too.PACKETFRAME_BENCH_QUICKstays for constrained-but-native environments.Verification
--workspace --all-targets --all-features -D warnings) green on all 4 published targets withPACKETFRAME_SKIP_BPF_BUILD=1;cargo fmt --checkclean.🤖 Generated with Claude Code