Skip to content

fix(ci): harden workerd smoke readiness (false-ready race) #313

Description

@cameronapak

Context

Follow-up from PR #216. The workerd linkedom smoke gate is green and correctly catches build/call/transform failures, but readiness has a narrow race, and the documented mechanism for build-time failures is wrong.

Do not block #216 on this.

Load-bearing fact (do not delete)

The fetch catch is the only thing catching build failures today. child.exitCode never fires on a failed build — wrangler stays alive. That handling needs a comment saying so, or it reads as redundant error plumbing and gets deleted. Removing it would silently un-cover build failures.

Measured behavior (build failure)

With a genuine esbuild failure (domino with in ESM):

  • wrangler dev does not exit — child.exitCode stays null
  • Lines 65–66 (wrangler exited early) therefore never fire
  • Wrangler still prints an Explorer API banner containing http://localhost:
  • waitForReady() returns on that banner
  • fetch throws ECONNREFUSED → catch sets exitCode = 1

Race on success path (Bumble's timings)

Current readiness regex in packages/core/scripts/workerd-linkedom-smoke.mjs:

/Ready on|Local:|http:\/\/127\.0\.0\.1:|http:\/\/localhost:/i

Timed on a clean successful run:

Event ms
readiness regex first matched 936
"Ready on http" logged 1002
first successful HTTP response 1040
race window ~104

waitForReady() can return before the worker serves; the runner then does a single fetch with no retry. The 250ms poll often absorbs it (CI passed at 29s), but a required check should not depend on that.

Do not take regex-only (measured)

Tightening to /Ready on http/ alone was confirmed against a real failed build (domino):

Check Result
Does a failed build ever print Ready on http? No — 0 matches
Loose regex matches on the same failed build 10
Tightened regex, full runner loop TIMED OUT after 90,047 ms

Regex-only turns today's ~1s ECONNREFUSED red into a measured 90,047 ms timeout on every build failure. Do not ship that.

Chosen fix

Leave the regex. Wrap the smoke fetch in a short retry loop. Closes the 104ms race and does not touch the failure path.

Also add a comment on the fetch catch: it is load-bearing for stuck failed builds (child.exitCode never fires because wrangler stays alive).

Optional later: fast-fail + tight regex

If the 90,047 ms case starts mattering: Bumble verified the build error text is already present in the captured output while waitForReady is polling, so this detector works without further proof:

if (/Build failed/i.test(output)) throw new Error(`Build failed\n${output}`);

That gets build failures red in ~1s and would let the regex be tightened safely. More code than the retry; not required for this issue.

File

packages/core/scripts/workerd-linkedom-smoke.mjs

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions