Repository navigation
feat(durable-messaging): add durable inbox processing - #11284
Open
ReubenBond wants to merge 59 commits into
Open
ReubenBond wants to merge 59 commits into
ReubenBond wants to merge 59 commits into
Conversation
Contributor
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Unresolved lifecycle, selection-mutation, scheduling, recovery, retention, validation, and outbox-contract issues remain.
Get a fresh assessment by requesting another Copilot review.
Review effort: Lite
Findings: 2
Open (3)
What changed in this PR
Adds a non-packable Durable Messaging inbox runtime integrated with Journaling and Durable Jobs, including recoverable ownership, transactional handler processing, retries, deduplication, and dead-lettering.
Changes:
- Adds durable messaging contracts, routing, diagnostics, and configuration.
- Implements journal-backed acceptance, recovery, pump coordination, and handler processing.
- Adds Journaling integration and comprehensive contract and functional tests.
| File | Reviewed change |
|---|---|
test/Orleans.DurableMessaging.Tests/Support/SnapshotProbeTests.cs |
Snapshot probe tests |
test/Orleans.DurableMessaging.Tests/Support/SnapshotProbe.cs |
Snapshot waiting helper |
test/Orleans.DurableMessaging.Tests/Support/ReceiverTestServices.cs |
Test service composition |
test/Orleans.DurableMessaging.Tests/Support/JournaledTestOutbox.cs |
Journal-backed test outbox |
test/Orleans.DurableMessaging.Tests/Support/HandlerProbe.cs |
Handler synchronization helper |
test/Orleans.DurableMessaging.Tests/Support/DurableMessagingMetricProbe.cs |
Metrics test probe |
test/Orleans.DurableMessaging.Tests/Support/DurableMessagingClusterFixture.cs |
Messaging test cluster |
test/Orleans.DurableMessaging.Tests/Support/DurableMessagingBehaviorTestBase.cs |
Shared behavior test base |
test/Orleans.DurableMessaging.Tests/Support/ControlledJournalStorageProvider.cs |
Journal failure injection |
test/Orleans.DurableMessaging.Tests/Support/ControlledDurableJobManager.cs |
Controlled job scheduling |
test/Orleans.DurableMessaging.Tests/Orleans.DurableMessaging.Tests.csproj |
Test project configuration |
test/Orleans.DurableMessaging.Tests/Functional/MessagingRoutingAndDeadLetterTests.cs |
Routing and dead-letter tests |
test/Orleans.DurableMessaging.Tests/Functional/InboxHandlerTransactionTests.cs |
Handler transaction tests |
test/Orleans.DurableMessaging.Tests/Functional/InboxCapacityBehaviorTests.cs |
Capacity behavior tests |
test/Orleans.DurableMessaging.Tests/Functional/InboxCallbackOwnershipTests.cs |
Callback ownership tests |
test/Orleans.DurableMessaging.Tests/Functional/InboxAcceptanceBehaviorTests.cs |
Acceptance behavior tests |
test/Orleans.DurableMessaging.Tests/Functional/DurableMessagingClusterCollection.cs |
Test collection definition |
test/Orleans.DurableMessaging.Tests/Functional/DedupeExpiryBehaviorTests.cs |
Deduplication expiry tests |
test/Orleans.DurableMessaging.Tests/Contracts/HandlerRoutingContractTests.cs |
Handler routing contracts |
test/Orleans.DurableMessaging.Tests/Contracts/DurableMessagingPumpResultsTests.cs |
Pump result contracts |
test/Orleans.DurableMessaging.Tests/Contracts/DurableEnvelopeContractTests.cs |
Envelope contracts |
test/Orleans.DurableMessaging.Tests/Contracts/DeliveryAndOptionsContractTests.cs |
Delivery and options contracts |
test/Orleans.DurableMessaging.Tests/Contracts/ActivationValidationTests.cs |
Activation validation contracts |
src/Orleans.Journaling/IJournaledStateObserver.cs |
Journal lifecycle observer API |
src/Orleans.Journaling/IJournaledStateManager.cs |
Observer registration API |
src/Orleans.Journaling/IJournaledGrainParticipant.cs |
Participant contract |
src/Orleans.Journaling/DurableGrain.cs |
Participant initialization |
src/Orleans.DurableMessaging/RoutePrefixHandler.cs |
Prefix route handler |
src/Orleans.DurableMessaging/RouteKeyHandler.cs |
Exact route handler |
src/Orleans.DurableMessaging/README.md |
Runtime documentation |
src/Orleans.DurableMessaging/Orleans.DurableMessaging.csproj |
Messaging project configuration |
src/Orleans.DurableMessaging/InboxHandlerSelectionContext.cs |
Handler selection context |
src/Orleans.DurableMessaging/InboxHandlerContext.cs |
Handler context implementation |
src/Orleans.DurableMessaging/IInboxHandlerContext.cs |
Handler context API |
src/Orleans.DurableMessaging/IInboxHandler.cs |
Handler contract |
src/Orleans.DurableMessaging/IDurableOutbox.cs |
Outbox contract |
src/Orleans.DurableMessaging/IDurableMessagingDiagnostics.cs |
Diagnostics API |
src/Orleans.DurableMessaging/IDurableInboxExtension.cs |
Inbox extension API |
src/Orleans.DurableMessaging/IDurableInbox.cs |
Inbox API |
src/Orleans.DurableMessaging/DurableMessagingTime.cs |
Time arithmetic helpers |
src/Orleans.DurableMessaging/DurableMessagingStateNames.cs |
Durable state names |
src/Orleans.DurableMessaging/DurableMessagingStateManagerCapabilities.cs |
State manager capability checks |
src/Orleans.DurableMessaging/DurableMessagingPumpResults.cs |
Pump execution results |
src/Orleans.DurableMessaging/DurableMessagingPumpCoordinator.cs |
Pump coordination |
src/Orleans.DurableMessaging/DurableMessagingJobOwnership.cs |
Job ownership metadata |
src/Orleans.DurableMessaging/DurableMessagingInstruments.cs |
Messaging metrics |
src/Orleans.DurableMessaging/DurableMessagingGrainParticipant.cs |
Messaging participant |
src/Orleans.DurableMessaging/DurableMessagingActivationValidator.cs |
Activation validation |
src/Orleans.DurableMessaging/DurableMessageState.cs |
Retry and dead-letter state |
src/Orleans.DurableMessaging/DurableInbox.cs |
Inbox storage and routing |
src/Orleans.DurableMessaging/DurableEnvelopeData.cs |
Deferred payload access |
src/Orleans.DurableMessaging/DurableEnvelope.cs |
Durable envelope model |
src/Orleans.DurableMessaging/DurableDeadLetterRetention.cs |
Dead-letter retention |
src/Orleans.DurableMessaging/DeliveryStatus.cs |
Delivery status values |
src/Orleans.DurableMessaging/DeliveryResult.cs |
Delivery result contract |
src/Orleans.DurableMessaging/CorrelationHandler.cs |
Correlation routing |
src/Orleans.DurableMessaging/Configuration/DurableInboxOptions.cs |
Inbox configuration |
src/api/Orleans.Journaling/Orleans.Journaling.cs |
Generated Journaling API |
Orleans.slnx |
Solution project registration |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Contributor
Code coverage
Report-only conclusion: improved. The current-main baseline is commit Coverage combines every CI test matrix job, including providers, CodeGen, .NET 8/10, Linux, Windows, and macOS, using canonical physical source and branch identities. The comparison remains report-only while normal line and branch variance is calibrated. Coverage details |
Contributor
There was a problem hiding this comment.
Copilot review overview
🔵 Needs a closer look
Six unresolved moderate findings and one nit remain, covering state rollback, option validation, shutdown and recovery handling, test-outbox idempotency, and an unused injected field.
Review effort: Lite
Findings: 2
Contributor
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
One critical build issue and five moderate correctness, lifecycle, and test-isolation issues remain unresolved.
Get a fresh assessment by requesting another Copilot review.
Review effort: Lite
Findings: 3
Contributor
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Six unresolved findings remain, including build-blocking and critical correctness issues.
Get a fresh assessment by requesting another Copilot review.
Review effort: Lite
Findings: 5
Open (6)
ReubenBond
force-pushed
the
rb-potential-happiness
branch
from
September 16, 2026 23:05
4d18916 to
b05c432
Compare
Contributor
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Unresolved ownership validation, compilation, and routing/performance issues require changes before approval.
Get a fresh assessment by requesting another Copilot review.
Review effort: Lite
Findings: 6
Open (6)
Resolved since last review (1)
ReubenBond
force-pushed
the
rb-potential-happiness
branch
from
September 17, 2026 01:13
b05c432 to
c7b6bd9
Compare
Contributor
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Unresolved ownership-race, retry/dead-letter accounting, deduplication, and shutdown-fault handling issues block approval.
Get a fresh assessment by requesting another Copilot review.
Review effort: Lite
Findings: 1
Open (2)
Resolved since last review (7)
ReubenBond
force-pushed
the
rb-potential-happiness
branch
from
September 17, 2026 01:27
c7b6bd9 to
67a4a2e
Compare
Preserve exact retry assertions while awaiting the requested pump after a coalesced local drain. Refs dotnet#11318.
Contributor
There was a problem hiding this comment.
🔵 Needs a closer look
The broad durable messaging and journaling changes include an outstanding moderate validation issue requiring review.
0 open findings
🧠 Review effort: Lite
Give feedback about Copilot approvals in this survey to enter a drawing for a $150 gift card.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Durable inbox acceptance and handler effects need reliable recovery, bounded retries and deduplication with a journal acknowledgement boundary shared with outgoing messages.
Solution
Inbox acceptance confirms its durable self-job before synchronously staging the envelope and exact scheduler-returned ownership handle, and succeeds after ordinary journal acknowledgement. Grain-local state determines work for duplicate, early and orphan wakeups.
Handlers use
HandleAsync -> ValueTaskand synchronouscontext.Complete(). Asynchronous preparation, validation and cancellation checks use local values. From the first shared mutation through method return, handlers synchronously stage safe-to-commit business changes, outgoing envelopes and completion.Complete()immediately stages inbox removal, message-state removal and processed/deduplication records, so a queued writer captures the whole logical outcome even before the handler-return continuation runs. An active repeated completion coalesces; retired or foreign contexts and new sends/acquisitions after completion are rejected.The runtime awaits the handler outcome, owns ordinary persistence and actual acknowledgement, and retires real acquisition tasks and batches through their actual outcomes. An exception after explicit completion is logged, then reported after the staged outcome is persisted and resources retire. Storage failure retains its authoritative cause. Successful return requires completion. Before-completion cancellation and preparation errors use the trusted handler contract, allowing retry on the same healthy activation and physical owner.
Eight canonical standard journaled states retain their owner/codec bindings, streams and wire identities. Private ownership accounting acknowledges immutable operation facts, including successful persistence followed by an after-hook error. Owned persistence failures recover in a fresh owner; deletion follows stop/drain, actual journal deletion and disposal/deactivation. Capacity, retries, dead letters, retention, bounded metrics and constant-time counts retain their existing behavior.
Rationale and dependency
Handlers supply the business-mutation boundary through the synchronous final block. The runtime keeps functional attempt identity, completion and resource ownership while relying on that coding contract for ordinary grain state. Deterministic lifecycle controls establish fresh retry completion, release-before-await duplicate ordering, and exact timer admission after the preceding lease releases.
The synthetic scoped-state fixture supplies the real
GrainPropertiesResolverand explicit grain metadata required by merged #11409, preserving actual scoped codec identity, single owner enrollment and persistence/reset behavior.Exact contracts parent #11282:
b3b18dfe330489c6a26a0399bdf224d8abeb39b4. Published inbox head:d51245a5851cd8c8d572b0efdf5cddc4049f3482. Selected main:8bc9fd244427351ad24ccc039a9e7642a42c1cc4. Immutable inbox comparison. This layer uses actual Journaling and Durable Jobs with an isolated test output collaborator; #11285 supplies production delivery and #10693 supplies public hosting and packaging.Microsoft Reviewers: Open in CodeFlow