Repository navigation
P1: Make Loom REST API the Orchestrator application boundary #936
Description
Objective
Make Loom's REST API a first-class application boundary over the Loom Orchestrator. REST must submit and observe work through the Orchestrator rather than bypassing orchestration and calling execution, workers, or model providers directly.
Architecture
HTTP / MCP / CLI/TUI adapters -> Loom API boundary -> Orchestrator -> worker coordination -> workers -> arbiter -> verification -> persistence
Loom owns the REST API and its integration with the Orchestrator. model-router owns provider/model/account selection; agent-setup owns profiles, policy, budget, and effective configuration.
Requirements
- Define a stable Orchestrator contract inside Loom.
- Route REST task submission through that contract.
- Expose task/execution/session status through the same orchestration boundary.
- Expose results, failures, cancellation/deadline state, and provenance consistently.
- Prevent REST handlers from bypassing the Orchestrator to invoke workers,
ExecutionEngine, or model providers as workflow authorities. - Keep MCP and CLI/TUI adapters on the same application-facing orchestration boundary.
- Support both local workers and the distributed fleet worker implementation behind the same worker contract.
- Preserve the separation between Orchestrator coordination and model selection performed by
model-router.
Acceptance criteria
- A real REST request can execute a repository task through
Orchestrator -> worker -> arbiter -> verification -> persistence. - REST can retrieve the task/session lifecycle and final result through Loom.
- Worker failure/requeue and verification failure are observable through the API with explicit failure semantics.
- The same orchestration contract is usable by MCP and other Loom adapters.
- Tests prove REST cannot become a second orchestration path.
- Live dogfood exercises the REST path as part of the end-to-end release gate.
Related work
- Distributed fleet workers: Loom P1: Implement distributed fleet worker layer #935
- End-to-end dogfood/release gate: FlossWare/FlossWare Add PostgreSQL + pgvector backend as an optional Loom implementation #4
This is an implementation requirement, not merely documentation. The goal is to make the architectural boundary executable and testable.
Activity
sfloess commentedon Sep 16, 2026
ADR-002 (#997) now makes the external-agent boundary explicit: REST and MCP are adapters over the same Orchestrator/application contract, and external agents must not create a second execution path or call Workers/ExecutionEngine/providers directly. #936 is therefore the core implementation path for making that architecture executable.
sfloess commentedon Oct 16, 2026
[Astra] The language-neutral Orchestrator application boundary is now documented in ADR-0011 (merged). The Python realization is in loom-ai-python PR #31: REST submit/observe/continue delegates to Orchestrator, durable state transitions are centralized, cancellation remains distinct, and request-validation errors are separated from application failures. This issue remains open because deadline/cancellation API semantics, distributed-worker conformance, and live external-client dogfood are not yet qualified.
sfloess commentedon Oct 16, 2026
[Astra] The Python application-boundary slice is now implemented and squash-merged in loom-ai-python PR #32 (commit
7be0db344a65699344ac0fa08e3ab9bc36afcada).LoomServernow delegates submit/observe/continue lifecycle operations to a sharedOrchestratorservice; REST no longer accesses the Arbiter or execution store as an alternate workflow path. Cancellation persists ascancelled, and existing routes/JSON error representation are preserved.Validation on Python 3.11, 3.12, and 3.13: 49 tests passed per runtime, Ruff lint/format passed, and the security audit passed.
This closes the Python implementation slice only. The broader issue remains open for language-neutral Orchestrator semantics, distributed-worker support, and end-to-end external-client qualification.
sfloess commentedon Oct 16, 2026
[Astra] Follow-up implementation hardening is now squash-merged in loom-ai-python PR #33 (commit
e74cb2487b20cf30299545c5c87b9b4276267735). The Orchestrator now returns a structured submission containing the execution ID and result, persists safe failure state/evidence when execution raises, validates HTTP request representations separately from application execution, and exposes the durable execution ID on failed submissions. The REST boundary continues to delegate through Orchestrator; HTTP handlers do not invoke the Arbiter or state store as an alternate workflow path.Validation: 51 tests passed on Python 3.11, 3.12, and 3.13, Ruff lint/format passed, and the security audit passed.
The older duplicate PR #31 is closed as superseded. #936 remains open for the remaining language-neutral/API semantics and qualification requirements: deadline/cancellation API semantics, distributed-worker conformance, and a real external-client/Crush dogfood run.
sfloess commentedon Oct 16, 2026
[Astra] Progress update: loom-ai-python PR #34 added durable structured-result persistence, and PR #38 added cooperative cancellation and absolute deadlines through the Orchestrator, HTTP, and generic MCP boundaries. Both were squash-merged after the full Python 3.11/3.12/3.13 quality gate and security audit passed. I am leaving #936 open: the acceptance criteria still require a real repository task through the complete Orchestrator → Worker/Arbiter → verification → persistence path and the applicable deployment/distributed-worker qualification. Unit and boundary tests are not a substitute for that evidence.
sfloess commentedon Oct 16, 2026
[Astra] One remaining conformance gap surfaced during cross-repository review: the Python Orchestrator still maps
ExecutionState.statusdirectly fromWorkerResult.statusand has no distinct Evaluation record. That can conflate Worker completion with task success, contrary to ADR-0010/ADR-0011. I opened loom-ai-python issue #41 to track separate lifecycle, Result outcome, and Evaluation fields with durable/HTTP/MCP regression coverage. #936 remains open pending implementation and real-task qualification.sfloess commentedon Oct 16, 2026
[Astra] The Python application boundary has advanced:
loom-ai-pythonPR #44 is now squash-merged as1ebebf6. Durable HTTP observation now separates lifecycle, structured Result, and explicit Evaluation; REST and generic MCP can append result-bound evaluations without rewriting lifecycle/outcome, and version-1 snapshots migrate forward. CI passed on Python 3.11/3.12/3.13 and the security audit passed. This is implementation/test evidence for the application boundary, not full closure of #936: its live dogfood acceptance still depends on #949 completing a fresh Crush-driven repository change and verification through Loom.sfloess commentedon Oct 16, 2026
Implementation update:
loom-ai-pythonPR #51 is now squash-merged (d4da5e3). Durable submissions using an existing caller-suppliedexecution_idnow return HTTP 409 and cannot overwrite the prior execution or invoke Workers again; the regression tests cover both the Orchestrator and HTTP boundary. The contract-side rule and language-neutral fixture are also merged here in PR #1045 (1ed91f6). This closes one concrete idempotency gap in the application boundary, but it does not by itself satisfy the full #936 acceptance criteria for live distributed-worker and end-to-end dogfood.sfloess commentedon Oct 16, 2026
Additional implementation update:
loom-ai-pythonPR #53 (4b70785) is merged. REST now returns 409 when a client tries to continue a terminal execution, and only an explicitly interrupted lifecycle can resume; the generic MCP/process-boundary tests cover this transition. The language-neutral rule is merged inloom-aiPR #1047. The core Orchestrator boundary is better defined and tested, but #936 remains open because live repository-task execution through the full worker/verification/persistence path and the distributed fleet integration are not yet qualified.sfloess commentedon Oct 16, 2026
REST failure semantics update:
loom-ai-pythonPR #56 is merged (49a58d0). All HTTP errors now include a stable machine-readablecategorywhile preserving the existingerrorstring and status codes. Categories distinguish invalid input, oversized payload, missing execution, unavailable durable state, duplicate identity, non-resumable continuation, invalid evaluation, state errors, and unexpected server failures. Tests passed on Python 3.11/3.12/3.13. SonarCloud's only failure is the deferred new-code duplication threshold; per project priority, I left that for the later cleanup pass. #936 remains open for the unqualified live end-to-end and distributed-worker path.sfloess commentedon Oct 16, 2026
[Astra] Model-backed Worker prompt preservation is now fixed in loom-ai-python PR #57, squash-merged as
f35d764.Intent.acceptanceis now passed to the provider alongside requirements and constraints, with regression coverage for multiple criteria. Python 3.11/3.12/3.13 quality gates and the security audit passed. This prevents a real input-loss case, but does not close #936: full Orchestrator/verification dogfood and distributed-worker qualification remain outstanding.sfloess commentedon Oct 18, 2026
[Astra] The generic capability-discovery/handoff contract is now merged in FlossWare/loom#45, and the distributed Worker lease/idempotency contract is merged in #46. These establish the language-neutral semantics required by this issue, but do not satisfy its runtime acceptance criteria. The remaining work is to complete the distributed Worker reference implementation (#935), qualify REST -> Orchestrator -> Worker/Arbiter -> verification -> persistence end to end, and run live dogfood. Keeping #936 open.
sfloess commentedon Oct 18, 2026
[Astra] Follow-up merged in
loom-ai-python#79(72d5242):loom-servernow accepts--state-dirto enable file-backed execution snapshots, with CLI tests for durable vs ephemeral mode and README startup guidance. Python 3.11/3.12/3.13 quality gates and security audit passed. This improves repeatable process-boundary setup but does not close #936: the CLI's built-in Worker is intentionally a no-op smoke Worker; the real repository-task path must still run through configured Worker → Arbiter → verification → persistence, and the release gate must exercise it.sfloess commentedon Oct 18, 2026
[Astra] I opened draft loom-ai-python#83 for the next application-boundary slice. It adds a
FleetDispatchWorkerbehind the existing Arbiter/Orchestrator path, propagates durable execution IDs into worker context, wires the CLI's REST server to the existing authenticated fleet worker service, and removes direct task submission from the worker-facing HTTP protocol. The draft includes a REST-to-remote-agent integration test and timeout/cancellation coverage. CI is pending; this is not yet qualified or merged, and the multi-host/security dogfood gates remain open.sfloess commentedon Oct 18, 2026
[Astra] loom-ai-python#83 is now merged (
277a3bb0bf3ece39b00d6718fcac49283b507e4c). It adds a REST-to-fleet integration test:POST /intents→ Orchestrator → Arbiter → FleetDispatchWorker → authenticated remote Worker agent → persisted result, with durable execution/task correlation. The worker-facing fleet HTTP server no longer exposes task submission, keeping creation behind Orchestrator. Python 3.11/3.12/3.13, SonarCloud, security audit, and GitGuardian all passed. Issue #936 remains open until live multi-host and external-client dogfood qualify the whole path; CI integration coverage is necessary but not the deployment acceptance gate.sfloess commentedon Oct 19, 2026
[Astra] loom-ai-python#84 is now merged (
12b62b057c96e197680810def8defbcbf1d55d6b). Its process-level integration test drivesPOST /intentsthrough the Orchestrator and FleetDispatchWorker to a separately running Worker process, then observes the persisted execution and fleet task result. This closes a meaningful integration-test gap, but does not yet satisfy the full issue: repository-changing execution, failure/requeue and verification-failure observability, shared REST/MCP/CLI application boundary, and live dogfood still need proof.sfloess commentedon Oct 19, 2026
[Astra] Follow-up merged in loom-ai-python#86, commit
875e2a0ffe9aa5acba5d163583f1f95055c6659f. The REST → Orchestrator → FleetDispatchWorker integration test now runs both successful and failed remote Worker outcomes and verifies the durable execution plus fleet task preserve the terminal failure and failure category. Python 3.11/3.12/3.13, SonarCloud, security audit, and GitGuardian all passed. Full repository-changing verification dogfood and generic MCP qualification remain open.