Repository navigation
Commit f05a9fc
authored
ci: post recursion guest cycle+accelerator diff (main vs PR) on /bench-verify (#808)
* feat(cli): report keccak/ecsm accelerator call counts under `execute --cycles`
The cycle counter only counted retired instructions, so a keccak or ecsm
accelerator call — one ecall that runs a whole permutation / scalar-mul in a
single cycle — was invisible. `execute --cycles` now also prints `Keccak calls`
and `Ecsm calls`, classified exactly as the prover's CpuOperation::from_log
does (a7 == KECCAK/ECSM syscall number on an EcallEbreak), tallied in the
memory-safe streaming loop.
* feat(bench): deterministic recursion-guest cycle/accelerator comparison
Add scripts/bench_recursion_cycles.sh. The in-VM recursion verifier's cost
for a fixed (guest ELF, input blob) is fully deterministic, so comparing two
git refs is two EXACT integer readings — total guest cycles, Keccak
accelerator calls, Ecsm accelerator calls — with no A/B/B/A interleaving.
For each ref the script: adds an isolated worktree at the ref's SHA, builds
the recursion guest via `make compile-recursion-elfs` (auto-detecting
recursion-<PRESET>.elf, falling back to recursion.elf on refs that build a
single unnamed guest), generates that ref's own input blob via the ignored
test_dump_recursion_input, and measures it with a measuring CLI built once
from this branch (`execute --cycles`, which reports the keccak/ecsm counts).
Results are cached per ref SHA so re-runs are fast; worktrees are kept so a
re-measure is a cargo no-op. It fails with a clear, ref-named message when a
ref lacks the dump test or a recursion guest artifact.
Prints an exact comparison table (negative delta = PR fewer = better,
matching bench_verify's sign convention) plus a machine-parseable raw-number
block for downstream tooling.
Usage: scripts/bench_recursion_cycles.sh REF_A [REF_B=origin/main] [PRESET=min]
* ci(bench-verify): post recursion guest cycle+accelerator diff (main vs PR)
After the verifier benchmark, run scripts/bench_recursion_cycles.sh and
append its deterministic recursion-guest cycle/keccak/ecsm comparison to
the same /bench-verify PR comment (same marker comment; not a second one).
The step is additive and isolated from the verifier bench: continue-on-error
plus `if: !cancelled()` keep a recursion-bench failure from failing the job
or clobbering the verifier verdict, and it still runs if the verifier step
failed. Bump the job timeout to 90 min for the cold recursion guest builds
(~10-20 min/ref the first time; the measurement itself is ~1 min).
* bench(recursion-cycles): harden result cache + workflow timeout/build-sharing
Address review findings on the recursion-guest cycle bench.
Script (scripts/bench_recursion_cycles.sh):
- Corrupt-cache correctness: write result files atomically (tmp + mv) and
validate every cached record on read (numeric keys integer, elf non-empty)
via a new valid_result(); an incomplete/partial cache is now discarded and
re-measured, and the final table/RAW emit is guarded so a truncated record
fails loudly instead of posting a bogus -100%.
- Cache key now folds in the MEASURE_CLI source SHA so the two ref sides can
never be compared across different counters.
- Bounded, self-limiting worktree cache: keep the newest PRUNE_KEEP (default
10) ref worktrees, prune older ones + their results/blobs/logs at startup
(touch-on-use tracks recency), and remove a worktree whose guest build fails
mid-run so partial trees never linger.
- Docs: correct the determinism claim (each ref dumps its own blob, so a
proof-format change conflates guest-code and proof-structure deltas); explain
that main->recursion.elf vs PR->recursion-min.elf is the expected comparison
(both under min proof options) and surface the differing artifacts prominently
in the output; reconcile the MEASURE_CLI "built from main in CI" comment.
Workflow (.github/workflows/bench-verify.yml):
- Add a step-level timeout-minutes: 70 to the recursion step (fail-fast well
under the 90-min job cap; continue-on-error absorbs it so the verifier
verdict is still posted).
- Set GUEST_TARGET_DIR / HOST_TARGET_DIR so the RV64 build-std and host cargo
target are shared across ref worktrees (cuts cold time and per-worktree disk).
- Update the timeout comments to reflect the real cold cost (MEASURE_CLI +
per-ref guest build + per-ref prover-test build for the blob dump).
- Bound the success-path raw-log fallback in the Post-result script the same
way the failure branch is bounded; note each ref dumps its own blob.1 parent a81c44e commit f05a9fc
2 files changed
Lines changed: 413 additions & 4 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
26 | 26 | | |
27 | 27 | | |
28 | 28 | | |
29 | | - | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
30 | 37 | | |
31 | 38 | | |
32 | 39 | | |
| |||
85 | 92 | | |
86 | 93 | | |
87 | 94 | | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
88 | 126 | | |
89 | 127 | | |
90 | 128 | | |
91 | 129 | | |
92 | 130 | | |
93 | 131 | | |
94 | 132 | | |
| 133 | + | |
95 | 134 | | |
96 | 135 | | |
97 | 136 | | |
98 | 137 | | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
99 | 141 | | |
100 | 142 | | |
101 | 143 | | |
102 | | - | |
| 144 | + | |
103 | 145 | | |
104 | 146 | | |
105 | 147 | | |
106 | 148 | | |
107 | | - | |
108 | | - | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
109 | 169 | | |
110 | 170 | | |
111 | 171 | | |
| |||
0 commit comments