Skip to content

[test] Fix flaky build CLI output capture - #97900

Merged
sokra merged 1 commit into
canaryfrom
codex/fix-debug-prerender-output-flake
Aug 26, 2026
Merged

[test] Fix flaky build CLI output capture#97900
sokra merged 1 commit into
canaryfrom
codex/fix-debug-prerender-output-flake

Conversation

@sokra

@sokra sokra commented Aug 25, 2026

Copy link
Copy Markdown
Member

What?

Stabilize NextStartInstance.build() CLI output capture so failed production builds return all trailing stdout/stderr diagnostics.

The existing sync-io-blocks-root behavioral assertions remain unchanged, including the route-specific Date.now() diagnostic, timeout/retry guard, and exit-code check.

Why?

The --debug-prerender cases intermittently lost the detailed Date.now() diagnostic from result.cliOutput in CI even though builds exited normally. Across the reported CI run and two retries, the missing route set varied from 4 to 2 to 3 routes, consistent with an output-drain race rather than a rendering or resource failure.

On latest canary with Node 20, NEXT_TEST_MODE=start, and IS_WEBPACK_TEST=1, 5 baseline campaigns completed 24 builds (20 debug-prerender) without reproducing the intermittent failure locally. A controlled Node 20 reproduction established the lifecycle race directly: the child exit event observed no trailing output, while close observed the trailing diagnostic.

How?

Wait for the child process close event before snapshotting cliOutput. Unlike exit, close is emitted after the piped stdout and stderr streams close, so diagnostics emitted near process shutdown are fully drained before the build result is returned.

This is intentionally limited to the affected start-mode build harness and is independent of the mmap persistence PR stack (#97873, #97888, #97889).

Verification

  • Baseline on Node 20.20.2 / start mode / Webpack: 5 campaigns, 24 builds total, 20 debug-prerender; 0 failures and no affected local routes
  • Deterministic child-process lifecycle reproduction: exit captured empty output; close captured the trailing diagnostic
  • Post-fix focused test: 5/5 runs passed, covering 20/20 debug-prerender cases
  • Post-fix complete test: 3/3 runs passed, covering 24/24 cases (12 debug-prerender and 12 normal production)
  • pnpm build-all
  • Prettier and ESLint for test/lib/next-modes/next-start.ts
  • pnpm types

Co-authored-by: Tobias Koppers <1365881+sokra@users.noreply.github.com>
@github-actions

github-actions Bot commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Stats from current PR

🟢 1 improvement

Metric Canary PR Change Trend
Webpack Build Time 25.187s 24.655s 🟢 532ms (-2%) ▁▁▁█▁
📊 All Metrics
📖 Metrics Glossary

Dev Server Metrics:

  • Listen = TCP port starts accepting connections
  • First Request = HTTP server returns successful response
  • Cold = Fresh build (no cache)
  • Warm = With cached build artifacts

Build Metrics:

  • Fresh = Clean build (no .next directory)
  • Cached = With existing .next directory

Change Thresholds:

  • Time: Changes < 50ms AND < 10%, OR < 2% are insignificant
  • Size: Changes < 1KB AND < 1% are insignificant
  • All other changes are flagged to catch regressions

⚡ Dev Server

Metric Canary PR Change Trend
Cold (Listen) 814ms 813ms ███▁█
Cold (Ready in log) 813ms 804ms ▇▃▃▁▅
Cold (First Request) 1.306s 1.290s ▆▁▁▁▂
Warm (Listen) 814ms 814ms ███▁█
Warm (Ready in log) 806ms 806ms █▃▃▁▆
Warm (First Request) 1.291s 1.291s █▁▂▁▃
📦 Dev Server (Webpack) (Legacy)

📦 Dev Server (Webpack)

Metric Canary PR Change Trend
Cold (Listen) 813ms 813ms ▂▂▂▇▂
Cold (Ready in log) 801ms 803ms ▁▁▁█▁
Cold (First Request) 3.403s 3.443s ▁▁▁▇▁
Warm (Listen) 813ms 812ms ▂▂▂▇▂
Warm (Ready in log) 798ms 813ms ▁▁▂▇▁
Warm (First Request) 3.462s 3.484s ▁▁▁▇▁

⚡ Production Builds

Metric Canary PR Change Trend
Fresh Build 5.870s 5.664s █▂▁▁▁
Cached Build 2.962s 2.964s █▁▁▁▁
📦 Production Builds (Webpack) (Legacy)

📦 Production Builds (Webpack)

Metric Canary PR Change Trend
Fresh Build 25.187s 24.655s 🟢 532ms (-2%) ▁▁▁█▁
Cached Build 24.792s 24.802s ▂▁▁█▁
node_modules Size 553 MB 553 MB █████
📦 Bundle Sizes

Bundle Sizes

⚡ Turbopack

Client

Main Bundles
Canary PR Change
017_4moyv5m34.js gzip 117 B 117 B
05jocc85-b73_.js gzip 341 B N/A -
096twxhe86hwd.js gzip 7.53 kB N/A -
0cz1d0mv5g_q7.js gzip 39.4 kB 39.4 kB
0e5lgz-xcf6ln.js gzip 273 B 273 B
0enhddgqs2dl-.js gzip 265 B 265 B
0h34mejwa_7pd.js gzip 354 B 354 B
0mg5-nyl2xcjy.js gzip 65.5 kB N/A -
0w853zxni3ro0.js gzip 1.36 kB N/A -
13lq2aw3_0bgw.js gzip 3.52 kB N/A -
17wjm_wc16a_w.js gzip 71.5 kB N/A -
1drifx6-oqf3q.js gzip 5.7 kB N/A -
1elt1qium-r2m.css gzip 115 B 115 B
1m5vdnlf0deiv.js gzip 411 B 411 B
1nigpy_mr0xvj.js gzip 339 B N/A -
1p0l8mn2wz5f2.js gzip 1.97 kB N/A -
1sqoyy39jpgkq.js gzip 121 B 121 B
1tf1phijqlx9j.js gzip 220 B 220 B
2-54v5vj7orwp.js gzip 46.8 kB N/A -
26upjmii0ixt5.js gzip 153 B N/A -
2b05_424u1zq-.js gzip 120 B N/A -
2ca40xittx95r.js gzip 8.75 kB N/A -
2kci0vnt-49o-.js gzip 113 B 113 B
2rtriq9lc6zj-.js gzip 116 B N/A -
2sxgl_rcrrn_p.js gzip 399 B N/A -
2vt62i17bhbdu.js gzip 2.35 kB N/A -
342di-crrg9kq.js gzip 119 B N/A -
36hfc2_979aw2.js gzip 119 B N/A -
38wfp6jjrrdfm.js gzip 131 B 131 B
3dl1ilk37-30g.js gzip 117 B N/A -
3h0us7yy7bd_o.js gzip 13.1 kB N/A -
3kvxagoobts7h.js gzip 117 B N/A -
3p99hd9vwe6ae.js gzip 13 kB N/A -
3r0475panhk6m.js gzip 123 B N/A -
3s0i-weykgzrt.js gzip 125 B N/A -
3vjosdbddykpw.js gzip 5.88 kB N/A -
3wijk2ywqgsv9.js gzip 2.52 kB N/A -
41u5s3oe2-erp.js gzip 2.29 kB N/A -
turbopack-0d..9ejy.js gzip 3.74 kB 3.74 kB
02zc7sqlpqxn3.js gzip N/A 119 B -
06pzxqp5qiozm.js gzip N/A 5.7 kB -
0dug3edczk4gl.js gzip N/A 71.5 kB -
0e28eh3gbwmp0.js gzip N/A 3.51 kB -
0e53csrecof4e.js gzip N/A 65.5 kB -
0jwho9fkrb_t3.js gzip N/A 2.29 kB -
0longconn5mlm.js gzip N/A 123 B -
0p4l3ozbez2cz.js gzip N/A 2.35 kB -
0sbh_15ejop4o.js gzip N/A 13 kB -
0skdm-9rpid8g.js gzip N/A 117 B -
0wv-h56sz83wn.js gzip N/A 120 B -
137xup2t69ovc.js gzip N/A 1.97 kB -
1ezuqvubf86h1.js gzip N/A 5.88 kB -
1fk10s3m1oecx.js gzip N/A 400 B -
1hut5u-lz2yif.js gzip N/A 117 B -
20qhkyyj2_tla.js gzip N/A 116 B -
2imyufe18bsb7.js gzip N/A 1.36 kB -
2p4n3-8-xixh_.js gzip N/A 341 B -
2vqdfieehz5do.js gzip N/A 8.75 kB -
2x99jdcpr5n8x.js gzip N/A 7.53 kB -
2xn85ykzw2v1e.js gzip N/A 13.1 kB -
34knadd6wu9qg.js gzip N/A 119 B -
39jq2b6t6z6up.js gzip N/A 339 B -
3a9a9tvkag_-j.js gzip N/A 2.52 kB -
3b660b4ak0547.js gzip N/A 153 B -
3d61tgick8g6b.js gzip N/A 46.8 kB -
3t121tj-h0kbm.js gzip N/A 125 B -
Total 299 kB 299 kB ⚠️ +3 B

Server

Middleware
Canary PR Change
middleware-b..fest.js gzip 1.02 kB 1.03 kB
Total 1.02 kB 1.03 kB ⚠️ +1 B
Build Details
Build Manifests
Canary PR Change
_buildManifest.js gzip 846 B 848 B
Total 846 B 848 B ⚠️ +2 B
Build Cache
Canary PR Change
00000001.sst gzip 15.3 MB 13.2 MB 🟢 2.09 MB (-14%)
00000002.sst gzip 12.9 MB 12.5 MB 🟢 398 kB (-3%)
00000003.sst gzip 12.1 MB 13.9 MB 🔴 +1.76 MB (+14%)
00000004.sst gzip 14.6 MB 15.3 MB 🔴 +708 kB (+5%)
00000005.sst gzip 2.75 MB 10.1 MB 🔴 +7.34 MB (+267%)
00000006.sst gzip 10.1 MB 2.75 MB 🟢 7.33 MB (-73%)
00000007.sst gzip 59 B 59 B
00000008.meta gzip 89 B 89 B
00000009.meta gzip 297 kB 297 kB
00000010.meta gzip 297 kB 297 kB
00000011.meta gzip 297 kB 297 kB
00000012.sst gzip 53.3 kB 52.6 kB 🟢 732 B (-1%)
00000013.sst gzip 1.78 MB 1.76 MB 🟢 22.9 kB (-1%)
00000014.sst gzip 59 B 59 B
00000015.meta gzip 116 B 116 B
00000016.meta gzip 327 kB 327 kB
00000017.meta gzip 404 kB 403 kB
00000018.sst gzip 54 kB 53.3 kB 🟢 676 B (-1%)
00000019.sst gzip 1.36 MB 1.37 MB 🔴 +4.94 kB (+0%)
00000020.sst gzip 59 B 59 B
00000021.meta gzip 116 B 116 B
00000022.meta gzip 327 kB 327 kB
00000023.meta gzip 372 kB 372 kB
00000024.sst gzip 54.1 kB 53.4 kB 🟢 684 B (-1%)
00000025.sst gzip 1.36 MB 1.37 MB 🔴 +6.61 kB (+0%)
00000026.sst gzip 59 B 59 B
00000027.meta gzip 116 B 116 B
00000028.meta gzip 327 kB 327 kB
00000029.meta gzip 372 kB 372 kB
00000030.sst gzip 53.4 kB 52.8 kB 🟢 619 B (-1%)
00000031.sst gzip 1.36 MB 1.37 MB 🔴 +6.85 kB (+1%)
00000032.sst gzip 59 B 59 B
00000033.meta gzip 116 B 116 B
00000034.meta gzip 327 kB 327 kB
00000035.meta gzip 372 kB 372 kB
00000036.sst gzip 54 kB 53.4 kB 🟢 620 B (-1%)
00000037.sst gzip 1.36 MB 1.37 MB 🔴 +7.03 kB (+1%)
00000038.sst gzip 59 B 59 B
00000039.meta gzip 116 B 116 B
00000040.meta gzip 327 kB 327 kB
00000041.meta gzip 372 kB 372 kB
CURRENT gzip 94 B 93 B 🟢 1 B (-1%)
LOG gzip 672 B 676 B
Total 79.6 MB 79.6 MB ✅ -23.3 kB

📦 Webpack

Client

Main Bundles
Canary PR Change
3322-HASH.js gzip 65.7 kB N/A -
4191.HASH.js gzip 169 B N/A -
9784-HASH.js gzip 5.6 kB N/A -
b1ad9f4c-HASH.js gzip 63.5 kB N/A -
framework-HASH.js gzip 59.7 kB 59.7 kB
main-app-HASH.js gzip 251 B 252 B
main-HASH.js gzip 39.8 kB 39.7 kB
webpack-HASH.js gzip 1.68 kB 1.68 kB
3577.HASH.js gzip N/A 168 B -
578-HASH.js gzip N/A 65.8 kB -
8590-HASH.js gzip N/A 5.57 kB -
a8984546-HASH.js gzip N/A 63.5 kB -
Total 236 kB 236 kB ⚠️ +107 B
Polyfills
Canary PR Change
polyfills-HASH.js gzip 39.4 kB 39.4 kB
Total 39.4 kB 39.4 kB
Pages
Canary PR Change
_app-HASH.js gzip 194 B 193 B
_error-HASH.js gzip 181 B 182 B
css-HASH.js gzip 334 B 331 B
dynamic-HASH.js gzip 1.81 kB 1.81 kB
edge-ssr-HASH.js gzip 255 B 253 B
head-HASH.js gzip 349 B 351 B
hooks-HASH.js gzip 382 B 384 B
image-HASH.js gzip 5.05 kB 5.05 kB
index-HASH.js gzip 260 B 259 B
link-HASH.js gzip 2.48 kB 2.48 kB
routerDirect..HASH.js gzip 317 B 318 B
script-HASH.js gzip 384 B 386 B
withRouter-HASH.js gzip 316 B 315 B
1afbb74e6ecf..834.css gzip 106 B 106 B
Total 12.4 kB 12.4 kB ✅ -6 B

Server

Edge SSR
Canary PR Change
edge-ssr.js gzip 128 kB 128 kB
page.js gzip 293 kB 293 kB
Total 421 kB 421 kB ✅ -325 B
Middleware
Canary PR Change
middleware-b..fest.js gzip 601 B 602 B
middleware-r..fest.js gzip 156 B 156 B
middleware.js gzip 45.6 kB 45.2 kB
edge-runtime..pack.js gzip 842 B 842 B
Total 47.2 kB 46.8 kB ✅ -385 B
Build Details
Build Manifests
Canary PR Change
_buildManifest.js gzip 699 B 699 B
Total 699 B 699 B
Build Cache
Canary PR Change
0.pack gzip 4.8 MB 4.79 MB 🟢 8.03 kB (0%)
index.pack gzip 124 kB 123 kB
index.pack.old gzip 124 kB 123 kB
Total 5.04 MB 5.04 MB ✅ -8.88 kB

🔄 Shared (bundler-independent)

Runtimes
Canary PR Change
app-page-exp...dev.js gzip 375 kB 375 kB
app-page-exp..prod.js gzip 206 kB 206 kB
app-page-tur...dev.js gzip 374 kB 374 kB
app-page-tur..prod.js gzip 206 kB 206 kB
app-page-tur...dev.js gzip 370 kB 370 kB
app-page-tur..prod.js gzip 204 kB 204 kB
app-page.run...dev.js gzip 371 kB 371 kB
app-page.run..prod.js gzip 204 kB 204 kB
app-route-ex...dev.js gzip 82.6 kB 82.6 kB
app-route-ex..prod.js gzip 55.7 kB 55.7 kB
app-route-tu...dev.js gzip 82.6 kB 82.6 kB
app-route-tu..prod.js gzip 55.7 kB 55.7 kB
app-route-tu...dev.js gzip 82.2 kB 82.2 kB
app-route-tu..prod.js gzip 55.4 kB 55.4 kB
app-route.ru...dev.js gzip 82.2 kB 82.2 kB
app-route.ru..prod.js gzip 55.4 kB 55.4 kB
dev-validati...dev.js gzip 134 kB 134 kB
dev-validati...dev.js gzip 134 kB 134 kB
dev-validati...dev.js gzip 132 kB 132 kB
dev-validati...dev.js gzip 132 kB 132 kB
dist_client_...dev.js gzip 324 B 324 B
dist_client_...dev.js gzip 326 B 326 B
dist_client_...dev.js gzip 318 B 318 B
dist_client_...dev.js gzip 317 B 317 B
pages-api-tu...dev.js gzip 46.3 kB 46.3 kB
pages-api-tu..prod.js gzip 34.3 kB 34.3 kB
pages-api.ru...dev.js gzip 46.2 kB 46.2 kB
pages-api.ru..prod.js gzip 34.3 kB 34.3 kB
pages-turbo....dev.js gzip 54.7 kB 54.7 kB
pages-turbo...prod.js gzip 39.7 kB 39.7 kB
pages.runtim...dev.js gzip 54.7 kB 54.7 kB
pages.runtim..prod.js gzip 39.6 kB 39.6 kB
server.runti..prod.js gzip 65.9 kB 65.9 kB
use-cache-pr...dev.js gzip 73.3 kB 73.3 kB
use-cache-pr...dev.js gzip 73.3 kB 73.3 kB
use-cache-pr...dev.js gzip 71.5 kB 71.5 kB
use-cache-pr...dev.js gzip 71.5 kB 71.5 kB
Total 4.1 MB 4.1 MB
📎 Tarball URL
https://vercel-packages.vercel.app/next/commits/5df331ccd6862e468bc96718a8fa1d96e2d37e71/next

Commit: 5df331c

@github-actions

github-actions Bot commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Tests Passed

Commit: 5df331c

@sokra
sokra marked this pull request as ready for review August 25, 2026 22:00
@sokra
sokra requested a review from bgw August 25, 2026 22:00
@eps1lon eps1lon changed the title Fix flaky build CLI output capture [test] Fix flaky build CLI output capture Aug 26, 2026
@sokra
sokra merged commit 2059206 into canary Aug 26, 2026
232 of 234 checks passed
@sokra
sokra deleted the codex/fix-debug-prerender-output-flake branch August 26, 2026 06:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants