fix: recover the workflow server across suspend and wake

[osolmaz]

Summary

Every laptop wake used to print warning storms in every open Pi session.
After sleep the workflow server killed itself because its lease looked expired, its half-dead process still held the lock file, and clients never retried a failed replacement, so each session warned Workflow server did not become ready: A workflow server is already running with PID X with a new PID every wake.
This change keeps the server serving across suspend, makes lock takeover require proof that the holder is not serving, and makes clients re-spawn failed replacements within seconds.

Plan: docs/plans/2026-09-27-wake-recovery-plan.md.

What Changed

The root cause was a chain, and each link got its own fix.

• renewServer in src/server/state.ts re-arms the authenticated owner's expired lease by matching the full recorded identity (epoch, server id, token hash, pid, process start identity) instead of requiring expires_at > now. Expiry now gates takeover; it never kills the owner. A superseded epoch or a wrong token still fails the renewal and stops the server, so fencing is unchanged.
• acquireServerLock moved to src/server/lock.ts and probes the recorded holder's socket with a bounded hello check before reporting already running. A live serving holder keeps the error; a dead or shutdown-bound holder loses the lock and the new server takes over. The epoch claim row stays the fencing authority, so a probe misfire costs one wasted start, never two serving servers.
• ensureAvailable in src/client/client.ts re-spawns a replacement when the spawned child exits without becoming ready, with a short grace for a competing holder and a cap of three attempts inside the existing 10-second window. The thrown error contract is unchanged.
• docs/WORKFLOWS.md documents the wake recovery behavior, including that expires_at answers whether the lease is valid, not whether the owner is alive.
• src/extension/index.ts is unchanged; the warning text stays accurate.
• A review round found a takeover race between concurrent replacements: the awaited probe let several starters pass the lock check, and a claim loser could unlink the winner's bound socket. The lock now revalidates the holder after the probe and converges on a newer takeover, and the server removes the socket file only when it actually bound it (commit 4bb797a). A second review round found the reverse race: the superseded holder exits unlinks the socket path its own listener bound, which the replacement now serves. The replacement re-creates the socket file within one poll tick while its claim still names it, so the handover cannot strand the new server (commit 29d78b8).

Testing

The repository checks pass on this Mac with a short temp root because the default macOS TMPDIR makes unix socket paths exceed the 103-byte limit.

• TMPDIR=/tmp/e2e npm run check: format, lint, typecheck, and build pass; vitest reports 1433 passing tests with 15 failures that a detached base worktree at bf32dc5 reproduces as a superset of 19 across 10 files, so this change adds no new failures.
• TMPDIR=/tmp/e2e npm run test:e2e: 3 files / 14 tests pass against the real Pi runtime.
• npx slophammer-ts@latest dry . and the dependency-boundary check pass.
• New tests cover lease re-arm, superseded/wrong-token refusal, real-server heartbeat keep-serving and superseded-stop, lock takeover against missing/dead/serving/deaf holders with real socket probes, and client re-spawn success plus the three-attempt cap.

Not tested here: the real-model live E2E from AGENTS.md runs as a separate manual validation. CI covers the unit suite in its normal environment.

Risks

A serving holder with a temporarily blocked event loop can fail the 500 ms probe and lose the lock. The epoch claim fences the outcome: the robber exits while the holder's lease is live, or the holder exits when its renewal fails, so the worst case is one wasted spawn, never two serving servers. An expired lease no longer implies a dead owner, so expires_at readers were audited to keep their lease-validity meaning. A short handover window can have two listening sockets, which is the pre-existing lease-granularity behavior.

Follow-ups

Adopt the fixed release in OnurPi (~/repos/onurpi pins @osolmaz/pi-workflows 0.17.4) so running sessions pick it up. Later, claim-only fencing with a serving-proof heartbeat and recovery telemetry can remove the lock file entirely.