security: confine session workers and add standalone proxy

[ebursztein]

Issue #206 found that the VM owner and service retained ambient authority over guest-facing network and per-session ledger resources. This change moves those resources behind generation-bound descriptor grants: one confined proxy worker and one confined ledger worker per session, supervised by the trusted service. It also adds capsem proxy, a VM-free OpenAI-compatible endpoint that reuses the same policy, credential, telemetry, and audit engine without accepting client-selected upstream authority.

The worker boundary is enforced before readiness. Linux uses Landlock plus seccomp and macOS uses Seatbelt; workers clear inherited environments and descriptors, bind every capability to the trusted session and generation, and attest that unrelated files, ambient network and control-socket paths, direct dialing, process execution, and parent signals are denied. Linux also denies socket creation. Seatbelt permits authority-free private unnamed socketpairs while denying external bind, listen, and connect operations. Broker grants are typed, single-use, and revoked with their generation. KVM sockets are prepared before the VM owner enters confinement, and the trusted service owns the global MCP discovery cache so a VM owner retains only its session, read-only assets, and granted IPC.

The ledger worker is the sole SQLite and native-zstd owner for its session. VM, proxy, service-read, stopped-session, retention, WARC, and body operations use bounded typed channels. Producer commitments are ordered and anchored outside the ledger directory, with explicit limits for unanchored tail loss and compromised producers. The zstd archive extension preserves deflate reads, corruption detection, durability ordering, and history across flushes.

capsem proxy supports official Python and TypeScript OpenAI clients for Chat Completions and Responses, including streaming, tools, usage, cancellation, and upstream error propagation. It exposes no CONNECT or forward-proxy path, cannot change its configured provider authority, starts no VM, and shuts down its worker and audit session together. The listener is unauthenticated as specified by #206; SDK gateway authentication remains separate.

The rebuilt docs include the implemented process and capability model, VM and standalone traffic paths, OS differences, lifecycle and revocation behavior, integrity limits, and a working SDK base_url example.

Validation for current 3153cada3311f6c3a71aa638cb8e3298a3e0664d:

• just fast-test passed 46/46 in 3m44s (20261010-161320-1dbfc5-test-fast), including 1,436 Citadel tests, dependency audits, workspace clippy, SDK/frontend tests, and docs/release-site builds
• the direct complete Kingslanding black-box suite passed 58/58 in 15m00s, including restored publication listeners, port collisions, TCP reset bursts, owner death, policy block/ask/plugin audit freshness, private-network isolation, OCI admission, MCP, workload seccomp, and lifecycle behavior
• focused current-tree checks passed: upstream grant protocol 14/14, service broker 19/19, ledger client 6/6, VM-owner sandbox 3/3, exposure Citadel 8/8, Linux worker confinement 1/1, and the 35-test publication owner after correcting one error-chain assertion
• predecessor CI run 38065273803 exposed one macOS import warning, two lifecycle-test races under coverage load, and a ledger shutdown race where child reap could beat delivery of the queued terminal event; the three scoped fixes pass exact coverage reproductions, all focused suites, 16 concurrent core repetitions, 100 ledger-retirement repetitions, and three-crate all-target clippy
• the rebuilt documentation site produced all 49 pages, including the permanent host-isolation architecture and the descriptor/reset/cache-freshness details exposed by the Linux review
• prior hosted run 37989232862 passed static/release contracts, ARM64 and x86_64 Linux, docs/sites, the full 5,786-test macOS unit suite, confined worker subprocess tests, integration, frontend, Python distribution/install, lint, TypeScript package acceptance, and MCP package acceptance; its install-only failure was the unpublished stable 0.7 runtime assets
• hosted run 38067066064 is the fresh ARM64 Linux, x86_64 Linux, macOS, static, package, and site matrix for this exact head
• the authoritative complete retry is pending because the gate requires explicit approval for just test ... force after its previous complete attempt failed; physical-Mac proof also remains pending before merge

The branch also updates vulnerable locked dependencies where patched releases exist and retains the repository's time-boxed build-only exception for braces, for which no patched release exists.

Closes #206

[ebursztein]

CI diagnosis for run 37971649154, ARM64 job 113959771469:

ledger_worker::tests::crashed_slot_restarts_only_after_reap_with_a_fresh_generation failed because the replacement fake ledger exited cleanly before accepting its first descriptor. This run changed only the proxy subprocess test harness; the ledger production and test code is identical to the preceding candidate, whose ARM64 job passed. The focused ledger restart test also passes 100/100 locally after this failure. I therefore classify this single sighting as a suspected scheduling/process-harness flake under full ARM64 coverage load. I will permit one failed-job rerun after the current macOS and x86_64 jobs finish; a second occurrence will be treated as a defect and fixed forward.

[ebursztein]

CI diagnosis for run 37971649154, macOS job 113959771017:

The confined-worker step passed (capsem-ledger 2/2 and capsem-proxy 5/5), confirming the macOS proxy broker-lifetime fix. The later Python SDK step failed deterministically because test_package_install.py requires the configured wheel and sdist, while the macOS job did not build either on a fresh runner. Head 2922a8f adds the missing package-build step before Python SDK tests and a workflow contract that fails when that prerequisite or ordering is removed. The exact build command produced both distributions locally, their isolated wheel/sdist acceptance passed 2/2, and the affected workflow/contracts passed 84/84 before the 0.7 refresh. Exact CI rerun: 37974377516.

[ebursztein]

CI diagnosis for run 37974377516, macOS job 113968904613:

The full 5,786-test Rust suite, ledger 2/2, proxy 5/5, integration, frontend, Python distribution/install, lint, and Python SDK checks all passed. The later TypeScript SDK package acceptance failed because the CI job entered its clean offline npm consumer without first running the package prewarm owned by the authoritative local gate. A fresh macOS runner's pnpm store does not imply that npm's separate cache contains zod, so both offline package tests correctly failed with ENOTCACHED.

Head 9f38147 adds pnpm run prewarm:package before the TypeScript SDK tests and strengthens the CI ownership contract to require that ordering. The contract failed before the workflow change and now passes 7/7; an explicitly empty npm cache was prewarmed online and the clean offline package acceptance then passed 2/2. The exact replacement run will revalidate the complete matrix. The install job remains independently blocked on the unpublished stable 0.7 runtime assets.

[ebursztein]

CI diagnosis for run 37977436941, macOS job 113979231430:

Static, both Linux architectures, docs/sites, the full 5,786-test macOS Rust suite, ledger 2/2, proxy 5/5, integration, frontend, Python distribution/install, lint, Python SDK, and the corrected TypeScript SDK package acceptance all passed. The subsequent MCP packed-package acceptance failed because CI likewise skipped prewarm:packed; its clean consumer correctly refused to fetch @modelcontextprotocol/sdk after entering offline mode. The authoritative local gate already runs this prewarm outside the sandbox.

Head df77b45 adds pnpm run prewarm:packed before MCP tests and strengthens the existing MCP CI ownership contract to require the exact install/prewarm/test order. The contract failed before the workflow change and now passes 12/12; an explicitly empty npm cache was prewarmed online and the packed SDK/MCP offline consumer then passed 1/1. The exact replacement run will revalidate the complete matrix. The install job remains independently blocked on the unpublished stable 0.7 runtime assets.

[ebursztein]

CI run 37986536371 reached the build-system release contracts and failed one debt-ratchet assertion: the integrated tree contains 21 Python # type: ignore suppressions while config/gate.toml still expected 22.

The latest 0.7 mock-server lease fix removed one suppression without lowering the exact ratchet. Candidate 49e42017b94bb798bbe82ac320aa6f69c5a7bf4a lowers the configured count to 21. The focused contract test_python_suppressions_and_exclusions_cannot_grow_unnoticed passes 1/1 locally. I am rerunning the exact fast and hosted gates on that corrected head.