Change blueprint format to git and enable receiving blueprint updates

[kentonv]

A blueprint now encodes a git packfile instead of a Yjs snapshot.

When you update a blueprint, people who have created gadgets from the old blueprint are now given the option to update it. For now, this does not happen automatically -- you just see a dot on the blueprints button and from there can choose to update. (We can consider implementing auto-update later perhaps.)

If local changes have been made since the blueprint was first instantiated, an agent is spawned to review the merge and fix any conflicts.

You can also choose to switch blueprints when updating. This allows you to e.g. switch from an original blueprint to some other user's customized version of it.

Update dialog:

Agent fixing merge conflicts:

[ask-bonk]

Review: 2 findings.

Posted 2 actionable inline comments in a single review.

github run

[github-actions]

Preview: pr659-kenton-git-blueprints

https://pr659-kenton-git-blueprints-router.cloudflare-os-previews.workers.dev

Dashboard · deleted when this PR closes

[ask-bonk]

Review: 1 finding.

Posted 1 new actionable inline comment in a single review. Existing findings were not duplicated.

github run

[kentonv]

Added a second round to this ("part 2" in the plan doc) to fix the way merges are done -- both for blueprint updates and when merging changes in a chat back to the workspace's mainline.

Previously we would encode the entire diff as OTs in a "changes" message, which could be very expensive to construct (it was a character-by-character diff) and required hacks to split it across multiple messages if the payload was too large for one row.

Part 2 of the PR makes it so we instead just introduce the changes in the form of a new commit, which subsequent OTs are based off of. Much cleaner.

I decided to add it to this PR rather than later because it allowed removing some complexity that had been added to the storage format earlier in the PR -- before anyone actually stored data in that format.

[ask-bonk]

LGTM!

github run

[ask-bonk]

LGTM!

github run

[github-actions]

Eval results

Verdict: ⚪ Unchanged. No task moved beyond what 10 runs can tell apart from noise.

Task	Score	Δ score	Fisher test	Cache hits	Avg min	Avg steps
change-calendar	80% → 100%	+20 pp	p = 0.47	87% 0 pp	3.3 → 3.2	22.5 → 25.4
chess	90%	0 pp	p = 1.00	97% 0 pp	8.9 → 9.0	59.6 → 68.4
incident-desk	90% → 100%	+10 pp	p = 1.00	94% → 95% +1 pp	3.9	29.0 → 34.1
worker-logs	90% → 80%	−10 pp	p = 1.00	93% → 91% −2 pp	3.9 → 2.9	26.7 → 20.4

Failed checks

Task	Check	Failed
change-calendar	t2 plan-is-a-document-listing-exactly-the-weeks-windows	1/10 → 0/10
change-calendar	t4 names-the-booked-windows-the-new-rules-reject	1/9 → 0/10
chess	t1 agrees-with-the-oracle-on-the-hard-positions	1/10
chess	t1 agrees-with-the-oracle-on-perft-positions	1/10 → 0/10
chess	t1 agrees-with-the-oracle-through-random-games	1/10 → 0/10
incident-desk	t1 opens-acknowledges-and-resolves-in-order	1/10 → 0/10
incident-desk	t1 simultaneous-opens-of-one-id-admit-exactly-one	1/10 → 0/10
worker-logs	t1 hourly-buckets-cover-every-hour-including-empty-ones	0/10 → 2/10
worker-logs	t1 ranges-are-half-open-and-filter-by-worker	0/10 → 2/10
worker-logs	t1 ingests-resets-and-summarises-per-worker	0/10 → 1/10
worker-logs	t3 latency-rises-match-the-reference	1/10 → 0/8
worker-logs	t3 latency-rises-count-late-events	1/10 → 0/8

Run · trajectories and raw results

[ask-bonk]

🔬 Eval runs review

Performance

Across 10 runs per task, pass rates were calendar 80% → 100%, chess 90% → 90%, incident desk 90% → 100%, and worker logs 90% → 80%; comparison.json labels these changes unchanged, not beyond noise. Chess remained the largest expense, averaging about $0.053 → $0.066 and 59.6 → 68.4 model steps, with PGN implementation and exact-text editing retries driving detours; worker logs fell from about $0.023 → $0.019 and 26.7 → 20.4 steps, partly because two candidate runs stopped after turn 1. Cache hits stayed broadly similar, and chess’s significant reduction in cache breaks does not demonstrate fewer misses; the PR’s new merge-conflict guidance also prompted unnecessary conflict searches in freshly built gadgets.

⚪ VERDICT: NO REGRESSION FROM THIS PR

Neither the candidate’s generated-code failures nor its cache and cost changes establish a beyond-noise regression caused by the blueprint-format or merge changes.

Triage

Failure modes

• Events lost before hourly reporting · worker-logs 2/10 · model error · this PR: no — In trial 2, turn 1’s writeFile(server.js) validated route but omitted it from normalizeEvent’s return, so ingestion violated the NOT NULL constraint and subsequent queries were empty. In trial 9, the same turn’s server write grouped timestamps using division without explicit flooring; the final executeCode smoke test returned zero for an hour containing an event, but the agent still claimed success. The task supplied the necessary event and bucket contracts; neither defect follows from the diff.
• Rook capture preserves castling rights · chess 1/10 · model error · this PR: no — Trial 6, turn 1’s writeFile(server.js) compared the captured piece with castling-right letters (K/Q/k/q) rather than rook pieces (R/r). After gxh1=N, move() returned FEN retaining K; later smoke tests covered promotion and castling separately, not their interaction. Main also failed one chess run, but with broader legal-move errors.

Tool errors

• editFile: Validation failed · change-calendar 1 → 4, chess 3 → 6, incident-desk 5 → 5, worker-logs 5 → 4 · model error — Calls omitted required fields or invented parameter names. Candidate calendar trial 2 repeated .replacement despite the error naming replacement; main chess trial 1 omitted filename and then corrected it.
• editFile: No matching text was found in the file · chess 10 → 16, incident-desk 2 → 4, worker-logs 1 → 0 · model error — Agents quoted text that differed from the file, especially backslash escaping during PGN edits. Candidate chess trial 2, turn 2 issued six mismatching edits together, then used grep to recover; main chess trial 2 similarly recovered with corrected escaping.
• readFile: File does not exist · incident-desk 2 → 6, worker-logs 4 → 1 · model error — Agents tried reading server.js or client.js immediately after creating an empty gadget. Candidate incident-desk trial 3 and worker-logs trial 2 then created the files with writeFile; main incident-desk trial 9 made the same unnecessary reads.
• editFile: Multiple matches were found · change-calendar 0 → 1, chess 3 → 0, incident-desk 2 → 3, worker-logs 1 → 0 · model error — The schema explicitly requires exactly one match, but agents supplied repeated helper calls or SELECT fragments. Candidate calendar trial 7, turn 1 recovered by adding surrounding context; incident-desk trial 7, turn 3 similarly expanded its SELECT match.
• executeCode: Failed to start Worker · chess 2 → 1 · model error — Main trials 1 and 10 supplied bare numeric source (24 and 3); candidate trial 7, turn 3 supplied 44. Each lacked the required default export and succeeded after retrying with an exported function, rather than exposing a Workshop startup defect.

Prompt cache

• chess · cache hits 96.55% → 97.28% · breaks 0.87% → 0.49% · this PR: unclear — Break-rate p = 0.023, but mean steps increased from 59.6 to 68.4. Both trial-1 transcripts lose most history cache reads at the start of turn 2, after gadget creation changes the system prompt’s gadget/file listing: main reads 6,948 tokens and writes 16,492; candidate reads 7,174 and writes 16,225. Both reuse the accumulated history at turn 3. The candidate’s additional 226 initial prompt tokens match the new conflict guidance, but no inspected step demonstrates fewer misses; additional cached steps can lower the break rate.

What to do

• In packages/workshop-backend/src/agent.ts, scope the new merge-conflict guidance explicitly to an actual merge or reported conflict, so agents need not scan freshly created gadgets before finishing. This addresses observed extra work, not a demonstrated regression.
• Optional: add a brief correctness-check reminder to the gadget-writing guidance in packages/workshop-backend/src/agent.ts: exercise ingestion and a nonempty hourly bucket, inspect returned values, and test interacting chess special moves. The observed failures are model errors, not defects this PR must fix.

github run

[Maximo-Guk]

🤖 comment ( feel free to refute! )

1. P2 File/directory conflicts abort blueprint updates
   Location: packages/workshop-backend/src/overseer.ts:3445–3450
   If a gadget locally modifies file lib while an upstream release replaces it with lib/helper.js, the flattened merge result contains both paths. Writing that result throws conflicting file paths at: lib/helper.js before creating the proposal chat, so neither the user nor the agent can resolve the conflict through the update workflow.
   Reproduced both an upstream file-to-directory refactor and concurrent additions. These conflicts need a representable resolution state before writing the proposal commit.

[kentonv]

@Maximo-Guk Known pre-existing issue: OTs don't detect when a name has been created both as a file and a directory; an error is raised only when you try to commit. Worth fixing eventually but not in this PR I think.

[ask-bonk]

Review: 1 finding.

Posted 1 new actionable inline comment in a single review. Existing findings were not duplicated.

github run