adr: own the durable-job backend as an event-sourced apalis Backend

[0xgleb]

Motivation

• Durable jobs were coupled to cqrs/es (ADR-0001) to get atomic enqueue and
  durability from the event store itself. The current implementation breaks
  that coupling: it leans on apalis-sqlite and hand-writes rows into apalis's
  private Jobs table to fake atomicity.
• Fallout (all surfaced in review of the stack): a sqlx 0.8 (worker) vs 0.9
  (enqueue) split against one table; an exact =rc.8 pin tracking apalis's
  churning private schema; an inconsistent/contorted atomic-enqueue story; and
  the empty-event-commands-silently-drop-jobs gap (HIGH on adr: jobs replace services #8/feat: replace EventSourced::Services with Jobs and wire atomic enqueue #15).

Solution

• ADR-0004: implement apalis_core::Backend over an event-sourced job model
  owned by event-sorcery, and drop apalis-sqlite. apalis stays the execution
  engine; we own storage + the job lifecycle, with job transitions as events in
  the same store -- the Lifecycle pattern one layer up.
• Atomic enqueue becomes native (the Enqueued event commits in the triggering
  aggregate's transaction); the empty-event gap dissolves; double-claim safety
  via optimistic-concurrency on the job stream; durable retry owned by the model.
• Proposal for review (Status: Proposed) -- no code yet. Supersedes parts of
  ADR-0001 and reshapes the jobs stack (feat: add job module #11/feat: add buffered JobQueue and apalis enqueue #14/feat: replace EventSourced::Services with Jobs and wire atomic enqueue #15). Open question in the ADR:
  rework the in-flight stack before merge vs land the backend as a follow-up.

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 775b9782-460f-4f24-a60d-c0835c2a5f27

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch adr/event-sourced-job-backend

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[0xgleb]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• adr: own the durable-job backend as an event-sourced apalis Backend #30 👈 (View in Graphite)
• adr: gradual consumer migration from services to jobs #19
• docs: migrate docs to the jobs model #20
• feat: migrate examples to the jobs model #16
• feat: replace EventSourced::Services with Jobs and wire atomic enqueue #15
• feat: add buffered JobQueue and apalis enqueue #14
• feat: add build_supervised_worker macro #13
• feat: job worker #12
• feat: add job module #11 : 1 other dependent PR (#31 )
• adr: jobs replace services #8
• master

---

How to use the Graphite Merge Queue

Add the label add-to-gt-merge-queue to this PR to add it to the merge queue.

You must have a Graphite account in order to use the merge queue. Sign up using this link.

An organization admin has required the Graphite Merge Queue in this repository.

Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.

This stack of pull requests is managed by Graphite. Learn more about stacking.