feat(claude): tell agents /tmp is node-local and controllers are jobs - #40
Merged
Merged
Conversation
Two failures kept recurring in agent sessions. An agent writes a script or an input to its scratchpad directory, which lives under /tmp, then runs it through srun; the job lands on another node, where the file does not exist, and the run fails with "No such file or directory". And a snakemake controller started in the session, or in an srun held open from it, dies with the session and takes the rest of the pipeline with it. The briefing now says both. /tmp and everything under it — $TMPDIR and the scratchpad included — is that node's alone, so the rule is to ask who has to read a file before staging it on /tmp: whatever crosses the session/job line goes on the shared filesystem, and the briefing names the cluster's scratch for it (/scratch/alpine/$USER where that exists, else ~/scratch). A job's own intermediates, made and consumed inside one allocation, stay on node-local disk, which the storage skill already recommends and now scopes to that case. A workflow controller is submitted with sbatch as a job of its own, bounded by nothing but its own -t. The hpc-compute, hpc-storage and slurm-batch skills carry the same rules in their own terms, with an sbatch script for a snakemake controller in slurm-batch, and the scripting docs and changelog follow. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XMdc1VPnWGf95LCg1dJvJR
The synthetic-walltime test stamped the state file with one now_epoch() and observed it with another. When a second ticked over between the two, the watch reported 1799 remaining instead of 1800 and the exact-value assertion failed — as it did on CI for #40. The test now takes one clock reading and uses it for every write and observation, so the remaining seconds it asserts on are the ones it wrote. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XMdc1VPnWGf95LCg1dJvJR
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Two failures kept recurring in agent sessions:
/tmp, then runs it throughsrun. The job lands on another node, where the file does not exist, and the run fails withNo such file or directory.srunheld open from it, dies with the session and takes the rest of the pipeline with it.What
sinteractive claude context) gains two paragraphs./tmpand everything under it ($TMPDIR, the scratchpad) is that node's alone; before staging on/tmp, ask who has to read it. Whatever crosses the session/job line goes on the shared filesystem, and the briefing names the cluster's scratch:/scratch/alpine/$USERwhere it exists, else~/scratch, as<scratch>/<topic>/. A workflow controller (snakemake, nextflow) is submitted withsbatchas a job of its own.hpc-computegets both rules as subsections under "Run work in its own allocation".hpc-storagescopes the existing "write scratch to node-local/tmp" advice to a single allocation, and says where cross-node files go on each cluster.slurm-batchgets ansbatchscript for a snakemake controller, with notes on the slurm executor,--qos=long/cpu-longfor long pipelines, and stopping it.docs/scripting.mdand the changelog follow.Checks
cargo fmt --check,cargo clippy --workspace --all-targets -D warningsandcargo test --workspacepass locally, run in a Slurm allocation.🤖 Generated with Claude Code
https://claude.ai/code/session_01XMdc1VPnWGf95LCg1dJvJR