Skip to content

feat(rivetkit): isolate includeState transaction reads with a committed snapshot - #5638

Open
abcxff wants to merge 1 commit into
mainfrom
stack/feat-rivetkit-isolate-includestate-transaction-reads-with-a-committed-snapshot-snxxvxko
Open

abcxff wants to merge 1 commit into
mainfrom
stack/feat-rivetkit-isolate-includestate-transaction-reads-with-a-committed-snapshot-snxxvxko

Conversation

@abcxff

@abcxff abcxff commented Sep 2, 2026 •

Copy link
Copy Markdown
Contributor

An includeState state transaction mutated actor state in place, so a
concurrent action reading state mid-transaction observed the owner's
uncommitted writes (a dirty read); the writes only reverted on rollback. That
gave atomic commit but not read isolation. The same held for hibernatable
connection state — and worse: a background save tick (serializeForTick, used
for periodic/sleep saves) could serialize and durably persist uncommitted
connection bytes, which rollback does not undo in storage.

Snapshot the committed actor and connection state when the transaction opens.
The transaction owner keeps mutating live state (so commit/rollback,
onStateChange, and retained-proxy semantics are unchanged), but every non-owner
context — actions, runtime save ticks, the inspector, sleep saves — reads the
snapshot instead:

  • actor state: ActorContextHandleAdapter#readState returns the snapshot for
    non-owner contexts.
  • connection state: NativeConnAdapter#readState returns the committed
    connection snapshot for non-owner contexts (gated by a new
    ownsActiveStateTransaction() predicate threaded like assertCanMutateState),
    and serializeForTick serializes the committed connection bytes for non-owner
    contexts so a background save can't durably flush uncommitted connection
    state.
    Snapshots are torn down on transaction exit.

Note: the driver-suite state-transaction tests require the native engine and
could not be run in this environment (they fail identically on unmodified
main — internal_error on stateTransactionCommit); validated by typecheck, the
mock-provider unit tests, biome, and review.

@abcxff

abcxff commented Sep 2, 2026 •

Copy link
Copy Markdown
Contributor Author

Stack for rivet-dev/actors

Get stack: forklift get 5638
Push local edits: forklift submit
Merge when ready: forklift merge 5638

change snxxvxko

@railway-app

railway-app Bot commented Sep 2, 2026 •

Copy link
Copy Markdown

🚅 Deployed to the actors-pr-5638 environment in rivet-frontend

Service Status Web Updated
frontend-cloud 😴 Sleeping (View Logs) Web Sep 22, 2026 at 7:39 pm UTC
frontend-inspector 😴 Sleeping (View Logs) Web Sep 15, 2026 at 4:10 pm UTC
kitchen-sink 😴 Sleeping (View Logs) Web Sep 15, 2026 at 4:10 pm UTC
website ❌ Build Failed (View Logs) Web Sep 15, 2026 at 3:58 pm UTC
ladle ✅ Success (View Logs) Web Sep 2, 2026 at 5:53 pm UTC
mcp-hub ✅ Success (View Logs) Web Sep 2, 2026 at 5:50 pm UTC

@claude

claude Bot commented Sep 2, 2026 •

Copy link
Copy Markdown
Contributor

Review (re-checked 5e504283, unchanged since the last pass)

New finding: #enterStateTransactions failure path can permanently poison state reads

In rivetkit-typescript/packages/rivetkit/src/registry/native.ts:3156-3204, the snapshot fields are set up in this order: activeStateTransactionOwner is set to the owner, then committedStateSnapshot is set from a structuredClone of the actor state baseline, and only after that does the code build connectionStateBaselines by walking actorConns() / connId() / connState() and assign it to committedConnStateSnapshot. The surrounding catch (error) block resets activeStateTransactionOwner back to undefined and releases the queue, but never clears actorState.committedStateSnapshot or committedConnStateSnapshot.

If any of the native calls while building connectionStateBaselines throw, committedStateSnapshot is left set while activeStateTransactionOwner is undefined. That combination breaks reads for the rest of the actors life until another transaction happens to overwrite it:

  • #readState() (native.ts:3540-3559) treats activeStateTransactionOwner !== this.#stateTransactionOwner as true whenever activeStateTransactionOwner is undefined, for every context, owner or not. So every subsequent c.state read anywhere in the actor returns the frozen pre-transaction snapshot instead of live state, even completely outside of any transaction.
  • serializeForTick() (native.ts:3346-3388) calls the same #readState() to build the persisted payload, so background saves would durably persist the stale frozen snapshot instead of whatever the actor legitimately wrote afterward. That is silent data loss until the condition clears.
  • It only self-heals if another includeState transaction later opens successfully, since that path sets a fresh activeStateTransactionOwner before recomputing committedStateSnapshot from live state. An actor that never retries an includeState transaction stays poisoned indefinitely.

Compare with #exitStateTransactions finally block (native.ts:3213-3223), which unconditionally clears both snapshot fields on every exit path. The catch in #enterStateTransaction should do the same so a mid-setup failure cannot leave a stale snapshot installed. This is the same class of bug the PR is fixing (a snapshot outliving its transaction), just on the failure path instead of the success path.

Carried over from the previous review pass (still valid, no code has changed):

  • Test coverage gap: the new test in actor-db.test.ts only exercises c.state. Per the PRs own description, the connection-state case is the more severe bug (a background save tick could serialize and durably persist uncommitted connection bytes, which rollback does not undo in storage), but there is no test that reads conn.state from a non-owner context during an open includeState transaction, or that asserts a serializeForTick/background save taken mid-transaction persists committed connection bytes rather than the owners uncommitted ones. The fixture already has createConnState: () => ({ atomicStateValue: null }), so this test would be a straightforward mirror of the new readAtomicStateValue state test.
  • Minor perf: NativeConnAdapter#readState() decodes committedBytes fresh on every read while a transaction is open (native.ts:1473-1476), producing a new object identity each call, unlike the actor-state path which decodes the snapshot once and reuses the reference. This defeats the get state() proxy memoization for the duration of any open transaction. Functionally harmless for read-only non-owner access, but worth decoding once per transaction if conn.state is read repeatedly while a transaction is held open.
  • CI coverage: the PR notes the driver-suite state-transaction tests need the native engine and could not be run locally (they reportedly fail identically on unmodified main). Worth confirming CIs native-engine driver run actually exercises actor-db.test.ts before merge, given how easy this class of concurrency bug is to get subtly wrong.

What holds up well

  • The ownership model is correctly threaded end to end: #stateTransactionOwner is a fresh Symbol per ActorContextHandleAdapter, and every dispatch path (makeActorCtx, withConnContext, #connMap, and all new NativeConnAdapter(...) call sites in buildNativeFactory) constructs a new instance and wires ownsActiveStateTransaction() through, so concurrent actions are correctly classified as non-owners.
  • The commit paths serializeForTick("save") call (native.ts:664) runs while activeStateTransactionOwner still identifies the owner and before #exitStateTransaction tears down the snapshots, so it correctly flushes the live, about-to-be-committed values rather than the pre-transaction baseline.
  • Snapshot teardown on the success/rollback exit path is fully synchronous, so there is no window where a non-owner read could observe a half-torn-down snapshot.

Solid fix for a real dirty-read/durability bug. The exception-safety gap on the setup path is worth closing before merge given the severity (silent, hard-to-detect state corruption) if it is ever hit.

@abcxff
abcxff force-pushed the stack/feat-rivetkit-isolate-includestate-transaction-reads-with-a-committed-snapshot-snxxvxko branch from 805dc7a to 8b21384 Compare September 3, 2026 16:37
@abcxff
abcxff force-pushed the stack/feat-rivetkit-isolate-includestate-transaction-reads-with-a-committed-snapshot-snxxvxko branch from 8b21384 to 575b775 Compare September 14, 2026 14:43

@the-company-company the-company-company Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 1 high-severity finding

Reviewed commit 575b775.

Comment on lines +3367 to 3370
const connSnapshot = isStateTransactionOwner
? undefined
: actorState.committedConnStateSnapshot;
const connHibernation = callNativeSync(() =>

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 High · Preserve connection updates after a snapshot save

A non-owner saveState({ immediate: true }) during an includeState transaction serializes the old bytes here, but the successful save consumes the core's pending hibernation update for the connection. The transaction owner later calls serializeForTick to commit, finds no dirty connection in actorDirtyHibernatableConns, and therefore omits its new connection state from the atomic commit. The SQL transaction succeeds while the connection reverts to the pre-transaction state after hibernation/reload.

Keep or restore the connection's pending update when serializing a non-owner snapshot (or otherwise ensure the owner commit emits every connection changed in the transaction). Add a regression that changes hibernatable connection state in a held transaction, performs a concurrent immediate save, commits, then reloads.

…ed snapshot

An includeState state transaction mutated actor state in place, so a
concurrent action reading state mid-transaction observed the owner's
uncommitted writes (a dirty read); the writes only reverted on rollback. That
gave atomic commit but not read isolation. The same held for hibernatable
connection state — and worse: a background save tick (serializeForTick, used
for periodic/sleep saves) could serialize and durably persist uncommitted
connection bytes, which rollback does not undo in storage.

Snapshot the committed actor and connection state when the transaction opens.
The transaction owner keeps mutating live state (so commit/rollback,
onStateChange, and retained-proxy semantics are unchanged), but every non-owner
context — actions, runtime save ticks, the inspector, sleep saves — reads the
snapshot instead:
  - actor state: ActorContextHandleAdapter#readState returns the snapshot for
    non-owner contexts.
  - connection state: NativeConnAdapter#readState returns the committed
    connection snapshot for non-owner contexts (gated by a new
    ownsActiveStateTransaction() predicate threaded like assertCanMutateState),
    and serializeForTick serializes the committed connection bytes for non-owner
    contexts so a background save can't durably flush uncommitted connection
    state.
Snapshots are torn down on transaction exit.

Note: the driver-suite state-transaction tests require the native engine and
could not be run in this environment (they fail identically on unmodified
main — internal_error on stateTransactionCommit); validated by typecheck, the
mock-provider unit tests, biome, and review.
@abcxff
abcxff force-pushed the stack/feat-rivetkit-isolate-includestate-transaction-reads-with-a-committed-snapshot-snxxvxko branch from 575b775 to 5e50428 Compare September 15, 2026 15:56

@the-company-company the-company-company Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 1 high-severity finding

Reviewed commit 5e50428.

Comment on lines +3367 to 3370
const connSnapshot = isStateTransactionOwner
? undefined
: actorState.committedConnStateSnapshot;
const connHibernation = callNativeSync(() =>

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 High · Preserve connection updates after a snapshot save

A non-owner saveState({ immediate: true }) during an includeState transaction serializes the old bytes here, but the successful save consumes the core's pending hibernation update for the connection. The transaction owner later calls serializeForTick to commit, finds no dirty connection in actorDirtyHibernatableConns, and therefore omits its new connection state from the atomic commit. The SQL transaction succeeds while the connection reverts to the pre-transaction state after hibernation/reload.

Keep or restore the connection's pending update when serializing a non-owner snapshot (or otherwise ensure the owner commit emits every connection changed in the transaction). Add a regression that changes hibernatable connection state in a held transaction, performs a concurrent immediate save, commits, then reloads.

This branch had an error being deployed

1 failed deployment
rivet-frontend / actors-pr-5638 — 5e504283 Deployed Sep 15, 2026 by railway-app[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant