Skip to content

tools/test-run.sh takes the fleet's run-time correctness slot itself #1004

Closed
lukstafi/ocannl-staging
#803

Description

@lukstafi

From lukstafi/ludics#160 (PR lukstafi/ludics#166). The fleet's run-time correctness cap is now fleet-worker.sh execution slot -- <command>, an N-holder flock a worker wraps around each test batch on its box. Today every wave brief has to remember the wrapper and every worker has to honour it. If tools/test-run.sh run takes the slot itself when FLEET_SLOT_STATE (or the fleet roster) is configured on the box, no brief has to say it and no worker can forget it; a box without the fleet configuration runs as before. One function: locate fleet-worker.sh through the deployed skill link, exec through execution slot when present, otherwise run directly. The ludics-lite side is done; this is the runner's half.

Activity

  1. added this to the v1.1.1 milestone on Sep 17, 2026
  2. lukstafi commented on Sep 23, 2026

    @lukstafi
    CollaboratorAuthor

    New motivation from #1033 (staging#769). The native per-slot widths (hip BOX_JOBS_SDMA_CAP / BOX_JOBS_NATIVE_GPU_SLOTS = 8 / 2, cuda 8) hard-code the fleet's slot count 2 (lukstafi/ludics#316). If test-run.sh took the fleet slot itself, the per-slot width could follow the fleet's count instead of restating it in a second repository.

  3. lukstafi commented on Sep 26, 2026

    @lukstafi
    CollaboratorAuthor

    In review as lukstafi#803, with lukstafi/ludics#435 as its companion. On a fleet box tools/test-run.sh run/start now takes the slot itself. It declares --cpu only when ocannl_read_config resolves a CPU backend from every test configuration, so an unset OCANNL_BACKEND is fine; otherwise it declares --gpu. A worker's own execution slot wrapper runs the batch inside that one slot through ludics-lite's new FLEET_SLOT_HELD marker. The runner only takes a slot when the deployed fleet-worker answers execution slot --probe, so the two PRs can land in either order.

  4. lukstafi commented on Sep 26, 2026

    @lukstafi
    CollaboratorAuthor

    Landed in lukstafi#803, with lukstafi/ludics#435 as its companion.

    • The runner takes the slot: on a fleet box, tools/test-run.sh run/start take the fleet's run-time correctness slot themselves, through tools/fleet-slot-run.sh. A worker's own execution slot wrapper is harmless: the ludics-lite side exports FLEET_SLOT_HELD, and a nested slot runs inside the enclosing one.
    • The kind (--cpu/--gpu) is resolved, and fails closed:
      • --gpu if the argv can pick its own backend (dune exec, or a command-line --ocannl_backend naming a non-CPU backend).
      • --gpu if the targets can reach a stanza that names a GPU backend. Test_utils.Slot_kind works this out from the markers env_var_deps enforces, reading only a closed set of argv shapes; any other shape counts as GPU.
      • Otherwise, --cpu only when ocannl_read_config resolves a CPU backend from both test/config and arrayjit/test. An empty backend counts as GPU, because Context.auto tries GPUs first.
    • Refusals and opt-outs: a refused or unreachable slot is its own verdict, SLOT REFUSED with exit 75, and the slot's wait takes at most half of --cap. repeat takes no slot, and OCANNL_TOOL_FLEET_WORKER=none turns it off.
    • Activation: it applies once a box's deployed issue-wave skill answers execution slot --probe. Until then the runner runs without a slot, as before.

    Worked examples: on rog, runtest test/operations is a GPU batch because it reaches test_cuda_pool_offset. A single CPU test's alias, @test/operations/scans, runtest test/einsum and @slow take a CPU slot.

  5. added a commit that references this issue on Oct 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions