Skip to content

perf(programs): build the ethrex guest with thin LTO - #861

Merged
MauroToscano merged 5 commits into
mainfrom
perf/ethrex-guest-thin-lto
Jul 31, 2026
Merged

perf(programs): build the ethrex guest with thin LTO#861
MauroToscano merged 5 commits into
mainfrom
perf/ethrex-guest-thin-lto

Conversation

@MauroToscano

Copy link
Copy Markdown
Contributor

What

Add a [profile.release] with lto = "thin" to the ethrex guest crate
(executor/programs/rust/ethrex/Cargo.toml). The crate previously had no
[profile.release] at all, so it built with cargo's release defaults
(lto = false, codegen-units = 16).

Why

Thin LTO measurably lowers the number of instructions the VM executes for a
real ethrex block, at no runtime cost — execution in this VM is priced per
executed instruction, so the small ELF-size increase is free.

Measured on the committed fixtures (same toolchain, same inputs):

fixture baseline (cycles) thin-LTO (cycles) Δ cycles Δ % Keccak Ecsm
ethrex_bench_4.bin 3,841,262 3,750,291 −90,971 −2.37% 174 16
ethrex_bench_16.bin 11,616,950 11,352,127 −264,823 −2.28% 356 64

The syscall invariants are identical before and after (Keccak 174 / Ecsm 16 for
_4, Keccak 356 / Ecsm 64 for _16) — the guest does exactly the same work;
LTO just compiles it into fewer executed instructions.

Methodology

Deterministic A/B on a dedicated 32-core rig, same toolchain
(nightly-2026-02-01, -Z build-std, riscv64im-lambda-vm-elf), cycles read
via cli execute --cycles on the committed fixtures. The measurement noise
floor is exactly 0: the pristine baseline was built twice in separate
CARGO_TARGET_DIRs and produced byte-identical ELFs, so the deltas above are
the whole signal.

Size

Guest ELF grows 3.64 MB → 3.82 MB (+4.9%). This is free here: the VM charges
per executed instruction, not by ELF size.

Follow-up (out of scope)

A tuned variant — thin LTO plus -C llvm-args=-inline-threshold=2500 /
-unroll-threshold=2000 — measured a further ~2× improvement
(ethrex_bench_16 11,035,843, −5.00% total; ethrex_bench_4 3,616,998,
−5.84% total) at +66% ELF size. That inline/unroll tuning is deliberately left
out of this PR and can land as a separate change.

The ethrex guest crate had no [profile.release], so it built with cargo's
release defaults (lto = false, codegen-units = 16). Adding thin LTO lowers the
number of instructions the VM executes for a real ethrex block, at no runtime
cost — execution is priced per executed instruction, so the small ELF-size
increase (3.64 MB -> 3.82 MB) is free.

Measured on the committed fixtures (same toolchain, same inputs):

  ethrex_bench_4.bin:   3,841,262 -> 3,750,291 cycles (-90,971, -2.37%)
  ethrex_bench_16.bin: 11,616,950 -> 11,352,127 cycles (-264,823, -2.28%)

Syscall invariants are identical before and after (Keccak 174 / Ecsm 16 for _4,
Keccak 356 / Ecsm 64 for _16): the guest does exactly the same work, LTO just
compiles it into fewer executed instructions.
@MauroToscano

Copy link
Copy Markdown
Contributor Author

/bench

@github-actions

github-actions Bot commented Jul 24, 2026

Copy link
Copy Markdown

Benchmark — ethrex 20 transfers (median of 3)

Table parallelism: auto (cores / 3)

Metric main PR Δ
Peak heap 51123 MB 49915 MB -1208 MB (-2.4%) ⚪
Prove time 25.974s 25.616s -0.358s (-1.4%) ⚪

✅ No significant change.

🔬 Looks like a small speedup (-1.4%) — below what 3 runs can confirm. Comment /bench-abba to run the drift-free ABBA tiebreaker (paired-t CI + exact Wilcoxon). Note: it occupies the bench server for ~30–40 min.
Optional pair count: /bench-abba 32 (20 resolves ~1%, 32 for ~0.6%).

✅ Low variance (time: 1.7%, heap: 0.9%)

Commit: e36794d · Baseline: cached · Runner: self-hosted bench

@diegokingston

Copy link
Copy Markdown
Collaborator

/bench

@diegokingston

Copy link
Copy Markdown
Collaborator

/bench

@diegokingston

Copy link
Copy Markdown
Collaborator

/bench

@MauroToscano
MauroToscano added this pull request to the merge queue Jul 31, 2026
Merged via the queue into main with commit 9ccdaf2 Jul 31, 2026
15 checks passed
@MauroToscano
MauroToscano deleted the perf/ethrex-guest-thin-lto branch July 31, 2026 17:34
MauroToscano added a commit that referenced this pull request Jul 31, 2026
The branch predated #861 (thin LTO on the ethrex guest) and #863, so every cycle
count and derived cost in these docs described a guest that no longer exists.
Re-measured on this branch at the merge, guest rebuilt from source:

  real block   74,819,518 -> 50,781,557 cycles
  synthetic    9,063,727  ->  8,734,622 cycles
  ratio             ~7x   ->      ~5.8x

so the CPU prove drops ~6 min -> ~4.5-4.8 min and the monolithic-heap figure
~330 GB -> ~240 GB (re-derived from the same measured growth fit via a same-ELF
cycle ratio, not rescaled by hand). Accelerator call counts are unchanged at
10,478 keccak / 116 ECSM, as expected — they follow the workload, not codegen.

Every count now names the ELF it came from. Two things move them and both have bitten
this doc already: guest optimisation (#861), and the clang major on PATH, worth ~2% —
the Makefile pins the guest's target flags but not its compiler, so `cc` takes whatever
clang is installed. The 50,713,534 the RTX 5090 box measured for this block on the same
main commit is that effect, not a regression: clang 18 there, clang 21 here, 0.13%
apart.

GPU is no longer unmeasured: 59.87 s wall at --epoch-size-log2 22 on an RTX 5090,
which is the recommended GPU epoch because VRAM binds — 2^22 leaves 28.9% headroom and
2^23 does not fit a 32 GiB card at all. CPU is ~4.5-4.8x that wall time. The
CPU-server epoch recommendation is left as "pending calibration" rather than guessed
from the GPU figure.

The two alternate candidate blocks keep their PRE-LTO numbers, explicitly labelled and
marked for re-measurement. Removing them would lose the selection evidence; presenting
them next to a main-vintage number without a label would be exactly the vintage-mixing
error the rest of this change is about.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants