Skip to content

[parallel] Add job size benchmark, raise job to 48 as best-performing - #8495

Merged
TomasVotruba merged 3 commits into
mainfrom
tv-jobsize
Oct 1, 2026
Merged

TomasVotruba merged 3 commits into
mainfrom
tv-jobsize

Conversation

@TomasVotruba

@TomasVotruba TomasVotruba commented Sep 14, 2026 •

Copy link
Copy Markdown
Member

Adds a benchmark that measures Rector wall time across parallel job sizes, plus a manual CI workflow to run it.

Script

scripts/rector-jobsize-benchmark.php runs cold Rector dry-runs across a list of job sizes and prints a markdown table of wall time, slowdown vs fastest, exit code, and file counts. It is a standalone dev tool and is excluded from PHPStan.

CI workflow

.github/workflows/jobsize_benchmark.yaml runs on workflow_dispatch only (no per-PR cost). Sizes default to 50-200 by 25 and are overridable from the run dialog. Results go to the job summary and an uploaded artifact.

Measurements

Both runs analysed the same 2620 files, every run cold, one sample per size.

Remote (GitHub runner, 4 cores)

jobSize wall vs fastest
16 48.8 s 1.40x
50 39.3 s 1.13x
100 34.9 s 1.00x
150 35.6 s 1.02x
300 36.2 s 1.04x

Local (24 cores)

jobSize wall vs fastest
50 49.1 s 1.16x
100 44.7 s 1.06x
130 42.2 s 1.00x
200 47.9 s 1.13x
300 64.2 s 1.52x
400 71.5 s 1.69x

Side by side, shared sizes

jobSize local (24c) remote (4c)
50 49.1 s 39.3 s
100 44.7 s 34.9 s
150 49.8 s 35.6 s
300 64.2 s 36.2 s

Notes:

  • On 24 cores, wall time climbs past job size ~250 (300 is 1.5x the fastest, 400 is 1.7x) as large chunks starve cores and the last worker drags the tail. The sweet spot sits around 60-130.
  • On 4 cores the curve is flat from 100 upward, because with few workers the chunk size matters much less.
  • The local box was under concurrent load during the sweep, so its absolute numbers are noisier and higher than the runner here; the shape of the curve is the point, not the absolute seconds.

@TomasVotruba TomasVotruba changed the title Add job size benchmark script + on-demand CI workflow Add job size benchmark script + on-demand CI workflow, bump to 48 Oct 1, 2026
@TomasVotruba

Copy link
Copy Markdown
Member Author

It seems the CI and local is passing well with values 50-100.

Next time we'll get time out, we have to figure out what is the dynamic function there. The project size and loc might be huge trigger. If so, we'll include it in dynamic computation to avoid mid-size projects run faster while big one running as well.

@TomasVotruba
TomasVotruba merged commit c5254e7 into main Oct 1, 2026
50 checks passed
@TomasVotruba
TomasVotruba deleted the tv-jobsize branch October 1, 2026 08:07
@TomasVotruba TomasVotruba changed the title Add job size benchmark script + on-demand CI workflow, bump to 48 [parallel] Add job size benchmark, raise job to 48 Oct 1, 2026
@TomasVotruba TomasVotruba changed the title [parallel] Add job size benchmark, raise job to 48 [parallel] Add job size benchmark, raise job to 48 as best-performing Oct 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant