Skip to content

Hard cut connected replicas to certified authoritative HOT state - #1674

Open
samuelstroschein wants to merge 16 commits into
mainfrom
codex/hot-state-authority-hard-cut
Open

Hard cut connected replicas to certified authoritative HOT state#1674
samuelstroschein wants to merge 16 commits into
mainfrom
codex/hot-state-authority-hard-cut

Conversation

@samuelstroschein

@samuelstroschein samuelstroschein commented Aug 30, 2026

Copy link
Copy Markdown
Member

Summary

  • make connected sync a certified serving cache behind the normal Lix handle instead of a second, partial engine API
  • keep current lix_file reads and one-argument lix_diff(relation) local and zero-round-trip after certification
  • route mutations, transactions, observations, branch operations, history, cold diff, coherent cold batches, and snapshot export to the authority inside the Rust handle
  • fence every successful authority mutation into the local certified HOT publication before returning
  • remove the JavaScript composite binding; JavaScript and Rust now use the same Rust routing implementation
  • hard-cut incompatible peers to sync protocol v6

Correctness contract

A connected replica serves only an atomic authority-published prefix. Its HOT rows, branch coordinates, working-diff epoch, blob payloads, and publication cursor are certified together; it never publishes replica-authored or mixed-epoch state. A remote concurrent write may make the prefix lag the authority, which is unavoidable without a read round trip, but every served local epoch existed atomically on the authority. Writes through the handle are read-your-write because success is not returned until the corresponding publication fence completes.

Cold operations execute against one authority session. If a coherent local HOT read is invalidated repeatedly, the entire batch is retried locally or returned from one authority snapshot; local rows are never combined with authority metadata.

One-argument working diff binds the branch's exact private certified cursor with an O(1) control lookup. The public lix_latest_checkpoint_commit_id() keeps its existing "latest real checkpoint" meaning.

Selective checkpoints remain ordinary semantic member commits during authority event capture; only metadata-only checkpoint boundaries use the authenticated complete-state alias. This keeps server-first history reconstructable without conflating the checkpoint's recovered head with its wire representation.

Browser authority observations use private registration and pull lanes so server-first history is never queued behind an unrelated finite worker operation and remains streaming and backpressured through custom fetch implementations. The pull lane buffers at most one bridge chunk and does not change a public API or HTTP endpoint. Superseding a stream while its browser fetch is still awaiting response headers aborts that fetch; dropping the Rust future cannot leak an SSE request and consume the per-origin connection budget.

Dynamic observation membership is generation-fenced and invalidated atomically under the membership lock. Stream opens, active streams, and retry waits are interruptible by a newer generation. The first multiplex SSE frame is published directly because its rows and mutation sequence were evaluated together in the authoritative session; there is no redundant execute request that can add head-of-line blocking or pair newer rows with an older sequence. Authority observation activation uses a session lifecycle guard plus a narrow branch-selector gate, not the write/publication gate. The selector gate spans both remote and local branch movement, preventing half-switched observations without delaying History behind unrelated publication.

Public interface impact

  • JavaScript remains openLix({ server: { mode: "sync" | "remote", ... } }).
  • Rust remains open_lix().with_server(ServerOptions::sync(...)).
  • The connected handle supports the same execution, transaction, observation, history, branch, and snapshot methods as a local handle; callers do not catch LIX_AUTHORITY_EXECUTION_REQUIRED or construct a protocol client themselves.
  • Public Rust types/signatures and JavaScript exports/types match main.
  • No SQL function/relation, storage surface, registered endpoint, query parameter, or public DTO was added.
  • OpenAPI changes only the intentional protocol constant from v2 to v6.

The one behavioral auth constraint is that a connected open_another_session().with_account(...) cannot override the account authenticated by the authority.

Performance and complexity

  • HOT read routing adds O(1) synchronization and zero network requests; query work remains O(rows examined/result size).
  • Working-diff cursor binding is O(1), down from a possible O(history depth) graph walk.
  • Delta HOT planning is O(P log P + sum(M_current log M_current) + sum((M_final + U) log(M_final + U))) time and O(P + max(M) + L_current + L_final + R) transient memory.
  • Delta network payload contains only newly final-live blob bytes: dead churn and already-certified inherited blobs transfer zero chunks.
  • Snapshot export is O(snapshot bytes) time and O(chunk + framing) memory.
  • Browser authority observations use O(active subscriptions + current chunks) bridge memory; registration is O(1), superseded opens are aborted, successful SSE bodies are never accumulated, and one authority execution per stream generation is removed.
  • Observation membership transitions add O(1) work plus the existing O(active subscriptions) membership snapshot on reconnect; cancellation and delivery waits retain no waiter after completion.

Final scorecard:

Case Bootstrap Allocated Working diff History requests
shallow history 179.9 ms 204.6 MB 1.09 ms 0
64-commit history 173.2 ms 204.7 MB 1.04 ms 0
3x wider rows 463.6 ms 557.8 MB 1.23 ms 0

The selected 1 MiB file read was 3.76-3.83 ms in the shallow/deep cases.

Validation

  • all simulations: 3,430 passed
  • sync-mode HTTP/filesystem E2E: 7 passed
  • Rust doctests: 10 passed
  • final-delta JavaScript remote/worker suite: 31 passed; Chromium suite: 19 passed, 2 skipped
  • server-protocol and wasm checks, strict Clippy, rustfmt, and diff check
  • adversarial inherited-blob, 64-read/16-switch atomicity, transaction/switch, account-recovery, snapshot-integrity, coherent-read, stream-backpressure, cancellation, and lost-wakeup regressions
  • deterministic blocked-finite-operation bypass, authoritative-frame/reconnect pairing, and History-shaped add/add/remove/add membership regressions
  • independent API, correctness, performance, browser-stream, checkpoint, observation-lifecycle, and production-trace reviews

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Aug 30, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-08-30T00:32:07.733807Z 169f023 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Aug 30, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
lix-website 06d974a Commit Preview URL

Branch Preview URL
Sep 01 2026, 06:24 AM

@blacksmith-sh

This comment has been minimized.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 169f023b1b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/lix/src/storage_spaces.rs Outdated
crate::sync::SYNC_REPOSITORY_EVENT_SPACE,
crate::sync::SYNC_REPLICA_STATE_SPACE,
crate::sync::SYNC_MATERIALIZED_STATE_ALIAS_SPACE,
crate::sync::SYNC_AUTHORITY_STATE_SPACE,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve mutability when restoring authority snapshots

When a snapshot comes from a served repository, authority admission has already written this marker, so including the space here copies it into the restored storage. A normal openLix.fromSnapshot() restore cannot select server mode (packages/js-sdk/src/open-lix.ts rejects that combination), so its adapter never receives authority-writer admission; every subsequent ordinary write adds a KeyAbsent precondition for this restored marker in storage_adapter/context.rs and fails with a transaction conflict. This makes the documented remote-export-to-fresh-local-restore workflow open successfully but remain permanently read-only; omit this process-ownership marker from portable snapshots or explicitly normalize it during restore.

Useful? React with 👍 / 👎.

@blacksmith-sh

This comment has been minimized.

@blacksmith-sh

This comment has been minimized.

@samuelstroschein
samuelstroschein force-pushed the codex/hot-state-authority-hard-cut branch from 4604485 to 06d974a Compare September 1, 2026 06:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants