Skip to content

Add --profile to record a CPU profile of every sample - #132

Merged
NullVoxPopuli merged 3 commits into
NullVoxPopuli:mainfrom
NullVoxPopuli-ai-agent:nvp/runner-profile
Oct 4, 2026
Merged

NullVoxPopuli merged 3 commits into
NullVoxPopuli:mainfrom
NullVoxPopuli-ai-agent:nvp/runner-profile

Conversation

@NullVoxPopuli-ai-agent

@NullVoxPopuli-ai-agent NullVoxPopuli-ai-agent commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

pnpm bench --profile=<dir> records a CPU profile of every sample, cut to the time between :start and :done. pnpm profile:report shows where that time went, or what changed against another profiled run.

pnpm bench --framework=ember --bench=all --profile=profiles/ember-main
pnpm profile:report profiles/ember-main
pnpm profile:report profiles/ember-pr --base=profiles/ember-main

How it records

  • Each sample runs inside a Chrome trace with only blink.user_timing and the V8 CPU profiler on. The trace has the marks and the profiler samples on one clock, so the cut at :start and :done is exact. The Profiler domain alone has no clock in common with performance.mark.
  • Only the page's main thread counts. Worker threads and other renderers are left out.
  • Per bench, the directory gets <bench>.jsonl (one line per sample: self time per frame, and time per stack) and <bench>.trace.json.gz (the full trace of the first sample, for DevTools or Perfetto).
  • The app's source maps are copied into <dir>/<framework>/maps, because the next build replaces dist.
  • The result file records PROFILE in its args. Profiling slows the page down, so a profiled run is only comparable with other profiled runs.
  • Profiling also changes the difference between two builds, not only the times. On the ember apps at 8x, it added 10 to 20 points to one build's gain on the sync benches. Use unprofiled runs to compare times, and profiled runs to find where the time goes.

What the report shows

Per bench: self time per package, self time per function, and inclusive time per function. With --base, it shows the largest changes for each instead.

Frames go through the source maps. ember-source ships no maps of its own, but its dist keeps real names, so the report reads the function name from the original source line. It also drops the content hash from chunk names (tracked-JcbMdnul.js becomes tracked.js), so two different ember-source builds line up by function and file.

Example, ember on 1 item, 100k updates at 8x, 3 samples:

self time by function
      27.0 ms  21.5%  set ember-source/shared-chunks/tracked.js
      26.3 ms  21.0%  dirtyTagFor ember-source/shared-chunks/meta.js
      19.0 ms  15.1%  MS common/src/tests/one-item.js
      15.4 ms  12.2%  (garbage collector)
      7.12 ms   5.7%  setter ember-source/shared-chunks/tracked-value.js

Size

At 8x, a trace has a sample about every 180 µs. A 100 ms sample is about 300 KB of trace and 12 KB of JSON lines. Only the first trace per bench is kept.

🤖 Generated with Claude Code

NullVoxPopuli and others added 2 commits October 3, 2026 03:06
`pnpm bench --profile=<dir>` records a Chrome trace of each sample and
keeps the CPU profile of the page's main thread between :start and
:done. A trace has the user timing marks and the profiler samples on
one clock, so the cut is exact.

Per bench, the directory gets one JSON line per sample (self time per
frame, and time per stack) and the full trace of the first sample. The
app's source maps are copied as well, because the next build replaces
dist.

`pnpm profile:report <dir> [--base=<dir>]` prints self time per package
and per function, and inclusive time per function, or the largest
changes against a base run. Frames go through the source maps, and the
name comes from the original source line, so two builds line up by
function and file.

The result file records PROFILE in its args, because a profiled run is
only comparable with other profiled runs.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
When the report finds no name in the original source, it used the
minified name, which changes with every build. Two runs of different
builds then showed the same harness function as one removed and one
added frame. An unnamed function now shows as (anonymous) with its
original file and line.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@vercel

vercel Bot commented Oct 3, 2026

Copy link
Copy Markdown

@NullVoxPopuli is attempting to deploy a commit to the NullVoxPopuli's projects Team on Vercel.

A member of the Team first needs to authorize it.

@vercel

vercel Bot commented Oct 3, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
rere-benchmark Ready Ready Preview Oct 3, 2026 11:03pm UTC

On the ember apps at 8x, profiled runs added 10 to 20 points to one
build's gain on the sync benches, compared with unprofiled runs of the
same two builds.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@NullVoxPopuli
NullVoxPopuli merged commit 047c8a3 into NullVoxPopuli:main Oct 4, 2026
5 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants