feat(inspect-ai): add OCI container execution mode and Compose parser

[tholop]

Summary

Lets Inspect AI evaluations that declare a Docker image or a single-service compose.yaml run on Capsem 0.7's native OCI container runtime (Hypervisor.create(image=...) and ExecTarget.WORKLOAD), without an in-guest Docker daemon (tracking issue #311, handoff issue #342). This PR only touches the Inspect AI provider (integrations/inspect-ai) and its hermetic gate fixture (tests/ironbank/test_sdk_live.py); it adds no container functionality to capsem-service, the guest runtime, or the SDKs. Stacked on #338.

Changes

• Inspect -> Capsem 0.7 container translation (integrations/inspect-ai/inspect_capsem/):
  • Adds execution_mode="container" and image: str | None = None to CapsemSandboxConfig (requiring an explicit image or compose_file when execution_mode="container", with no ambient default image) and deletes the legacy 0.6 in-guest dockerd backend.
  • Reconciles onto 0.7's bounded single-service Compose parser (ComposeInputs, ComposeLimits / _BoundedSafeLoader, InterpolationBudget, compose_service.py) to map image, command/entrypoint, environment, working_dir, user, bind and declared empty named volumes, healthcheck, and ${VAR} interpolation (SAMPLE_METADATA_*) onto Hypervisor.create(image=...) plus ExecTarget.WORKLOAD execs.
  • Fails closed at parse time on Compose features 0.7 cannot honour (network_mode: none, multi-service topologies, ports mappings, undeclared/external named volumes, anonymous single-target volumes, and build: / Dockerfile before feat(inspect-ai): add host-side Dockerfile and Compose build image support #340; init: true and expose emit an explicit warning and are stripped) — see Known Limitations below.
• Workload ergonomics & identity enforcement:
  • Stages /workspace, preserves file modes on single-file and directory bind mounts, creates empty target directories for declared top-level named volumes, and stages tool bundles (onedir) with 0700 permissions.
  • Supports non-root user execution (su -m / setpriv when the container runs as root; when the image USER is already non-root under 0.7's no-new-privileges, exec(user=<matching image user>) succeeds as a no-op with normalized USER/LOGNAME/HOME, while exec(user="root") or switching to a different user fails loudly with PermissionError).
• Host-boundary guards:
  • Enforces default-deny host environment interpolation (SAMPLE_METADATA_* allowlist with .env spoofing rejection), symlink-safe bind-mount containment (os.path.realpath within the project directory, refusing any bind whose host source or container target basename is docker.sock), and operator-only SdkCapsemController(registry_ca_pem=...) constructor injection (rejected in task configs).
  • Operators grant extra host env vars or bind roots per run via CAPSEM_INSPECT_ALLOWED_HOST_ENV and CAPSEM_INSPECT_ALLOWED_HOST_PATHS; task configs (CapsemSandboxConfig.allowed_host_env and allowed_host_paths) can only narrow — never widen — the operator's settings.

Installed-Package Qualification & Live Benchmarks (#342)

• Static, unit & installed-package gate (integrations/inspect-ai/tests, linux/x86_64, Python 3.12.14):
  • ruff check, ruff format --check, and ty check --error-on-warning --python-platform all: PASS (0 findings).
  • pytest integrations/inspect-ai/tests -q: 92 passed (97% line coverage), covering tests/containers/*, offline wheel + sdist installation (test_package_install.py), non-root container exec(user=...) identity checks, and fast-fail ValueError refusal on Compose build: / Dockerfile.
• Hermetic installed-package live VM + OCI workload acceptance (tests/ironbank/test_sdk_live.py::test_inspect_ai_live_vm_sandbox_acceptance, 134.57s):
  • Installs capsem-0.7.0 wheel + inspect-capsem-sandbox-0.1.0 wheel and sdist offline (--no-deps, --no-install-local) into isolated prefixes outside the repo and runs image_package_acceptance.py and live_acceptance.py under python -I. Both archives emitted SDK_IMAGE_PACKAGE_ACCEPTANCE_OK, INSPECT_CAPSEM_CONTAINER_ACCEPTANCE_OK, and INSPECT_CAPSEM_VM_ACCEPTANCE_OK.
  • Serves a digest-pinned busybox OCI image built from the guest initrd (tests/oci_workload_fixture.py) over a loopback TLS registry with a throwaway CA (SdkCapsemController(registry_ca_pem=ca_pem), zero public network access), runs sample_init(image=...), ExecTarget.WORKLOAD execs, text/binary file roundtrips, and real Inspect eval_async (accuracy=1.000), verifies history(layer=EXEC) + session.db exec_events (target="vm" and target="workload"), and confirms foreign persistent VM survival and 0 leaked VMs.

Suite / Task	Mode	Samples	Status / Result	Wall Time	Leftover VMs	Notes
tests/ironbank/test_sdk_live.py (test_inspect_ai_live_vm_sandbox_acceptance)	Installed wheel + sdist (0.7 VM + hermetic OCI workload)	4 eval_async + 2 self_check (43/43, large-binary skipped)	1/1 passed (wheel + sdist)	134.57s	0	SDK_IMAGE_PACKAGE_ACCEPTANCE_OK + INSPECT_CAPSEM_CONTAINER_ACCEPTANCE_OK + INSPECT_CAPSEM_VM_ACCEPTANCE_OK
integrations/inspect-ai/tests/live_acceptance.py	0.7 VM + OCI container	3	3/3 passed (44/44 VM + 44/44 container self_check)	121.16s	0	Verifies /workspace write+read, non-root developer (1000:1000) chown/exec/env, onedir 0700, and numeric 1000:1000 Compose user
inspect_evals/humaneval	VM (default)	3	success (0 errors)	12s	0	Clean VM boot, exec, and teardown on stock 0.7 assets
inspect_evals/gsm8k	VM (default)	3	success (0 errors)	12s	0	Clean VM boot, exec, and teardown on stock 0.7 assets
inspect_evals/agentdojo (with_injections=false)	VM (default)	2	success (0 errors)	11s	0	Clean VM boot, exec, and teardown on stock 0.7 assets
compose_bind_smoke	Container (python:3.11-slim)	1	success (accuracy=1.000, 0 errors)	17s	0	Verifies bind mount + ${SAMPLE_METADATA_*} in OCI workload container
inspect_harbor/terminal_bench_2 (build-pov-ray)	Container (Compose image:)	1	success (0 errors)	43s	0	Pulls/unpacks Harbor OCI image, runs bash tool loop + harbor_scorer
bigcodebench / swe_bench (default) / cybench (2-service)	Compose fast-fail	3	Fast-fail ValueError	6s–7s	0	Rejects network_mode: none and multi-service Compose before VM creation
inspect_evals/swe_bench_verified_mini (-T allow_internet=true)	Container (~655 MB compressed / ~1.5 GB uncompressed)	3	3/3 completed (2/3 = 0.667 score) only with a locally patched service (launch.py:484 timeout=30 -> 300, raised CREATE_READY_TIMEOUT); fails on stock 0.7 (HTTP 500 / 504 during image unpack)	902s	0	See Limitation 1 below

Known Limitations & Follow-Ups (Non-Blocking)

Single-service container evals whose images fit the stock 0.7 pull/unpack budget and use default networking work end to end (terminal_bench_2, compose_bind_smoke, and the hermetic OCI workload acceptance above run on unpatched 0.7). The items below reflect what 0.7 exposes today:

1. Very large OCI images can exceed the stock 0.7 create budget — e.g. swe_bench_verified_mini (~655 MB compressed / ~1.5 GB uncompressed). Warm host cache: ~36s in launch.py:assemble() plus > 30s in umoci unpack trips subprocess.run(..., timeout=30) at guest/artifacts/container/launch.py:484 (HTTP 500 after 68s). Cold cache: pull (104–108s) + unpack exceeds CREATE_READY_TIMEOUT = 110s (crates/capsem-service/src/container_setup.rs:28, HTTP 504 create_timeout; inspect-capsem deletes the half-created VM via CreateTimeoutError.vm_id). Typical eval images (< 500 MB) are unaffected; POST /images/pull (0.7: Polish Python SDK for the image runtime #303) can pre-warm the host cache ahead of create, and Minimal Capsem runtime with OCI application images (replaces profiles) #289's planned .erofs / unpack-once path removes most of the unpack cost.
2. oci-cache-lock 30s stripe-lock contention under concurrent creates of the same image — Puller::blob (crates/capsem-assets/src/oci/pull.rs:570) acquires the exclusive per-digest blob stripe lock cache.lease(&descriptor.digest) (locks/<hex[..2]>.lock, LockMode::Exclusive) before calling BlobCache::copy_hit (crates/capsem-assets/src/oci/cache.rs:219-246) and holds it across the entire verified_copy SHA-256 read/write pass even after copy_hit opens the read-only blob descriptor and drops lookup_lease; meanwhile lock_with (cache.rs:488) polls oci-cache-lock for 30s instead of PULL_TIMEOUT (300s). On stock 0.7, in a 4-way concurrent evaluation (max_sandboxes=4) on a 400 MiB image, 3/4 samples fail after 30s with OCI cache lock timed out on cold cache and 2/4–3/4 fail on warm cache (0 leaked VMs). A small standalone service fix PR (fix/oci-cache-lock-lease) releases the stripe lock in copy_hit_with_lease as soon as open_file opens the cached blob descriptor and aligns lock_with with PULL_TIMEOUT (300s), taking both cold and warm 4-way creates to 4/4 passing (0 errors).
3. No per-VM air-gap / egress-deny toggle in ProvisionRequest — benchmarks whose Compose file sets network_mode: none (swe_bench_verified_mini default allow_internet=False and bigcodebench) are rejected at parse time rather than silently given network access, because 0.7 has no per-VM egress control (swe_bench_verified_mini runs with -T allow_internet=true).
4. Multi-service Compose (> 1 service / sidecars) is unsupported by design — inspect-capsem fails closed when a Compose file defines more than one service (e.g. cybench default + victim), matching Minimal Capsem runtime with OCI application images (replaces profiles) #289's single-workload scope for 0.7.
5. Dockerfile / Compose build: is layered on top in feat(inspect-ai): add host-side Dockerfile and Compose build image support #340 (Integrate and qualify Inspect AI on current 0.7 (Pierre handoff) #342 step 4) — in this PR (feat(inspect-ai): add OCI container execution mode and Compose parser #339), compose_fields.py, _compose.py, and config.py fail fast with ValueError when a task specifies build: / Dockerfile instead of a pre-built image:. feat(inspect-ai): add host-side Dockerfile and Compose build image support #340 stacks on this PR to add the default-off HostBuildGrant (CAPSEM_INSPECT_HOST_BUILD=1) build path.

[codecov-commenter]

❌ 5 Tests Failed:

Tests completed	Failed	Passed	Skipped
6052	5	6047	0

View the top 3 failed test(s) by shortest run time

capsem-foundation::unix::change_watch::tests::directory_path_watch_tracks_ancestor_replacement_and_retains_original_root

Stack Traces | 0.073s run time

thread 'unix::change_watch::tests::directory_path_watch_tracks_ancestor_replacement_and_retains_original_root' (221796) panicked at .../unix/change_watch/tests.rs:20:5:
assertion failed: !watch.changed().unwrap()
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

capsem-assets::oci::cache::inventory::tests::reused_materialization_controls_invalidate_inventory_on_acquire_and_after_release

Stack Traces | 0.203s run time

thread 'oci::cache::inventory::tests::reused_materialization_controls_invalidate_inventory_on_acquire_and_after_release' (144000) panicked at .../cache/inventory/tests.rs:130:39:
called `Result::unwrap()` on an `Err` value: cache changed during inventory observation
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

capsem-assets::oci::worker::tests::inventory_worker_publishes_only_quiet_bounded_observations_and_tracks_nested_changes

Stack Traces | 2.39s run time

thread 'oci::worker::tests::inventory_worker_publishes_only_quiet_bounded_observations_and_tracks_nested_changes' (148648) panicked at .../oci/worker/tests.rs:45:6:
called `Result::unwrap()` on an `Err` value: TimedOut { label: "linked-inventory", attempts: 8, timeout: 2s }
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

capsem-assets::oci::worker::tests::incompatible_receipts_are_observed_once_until_their_metadata_changes

Stack Traces | 2.7s run time

thread 'oci::worker::tests::incompatible_receipts_are_observed_once_until_their_metadata_changes' (148577) panicked at .../oci/worker/tests.rs:221:10:
called `Result::unwrap()` on an `Err` value: TimedOut { label: "foreign-cache-observed", attempts: 8, timeout: 2s }
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

capsem-core::fs_monitor::tests::a_workspace_swapped_for_a_host_link_is_never_walked_or_read

Stack Traces | 360s run time

No failure message available

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}