[mia-autodev]

Closes #73: For allen-label-free-prediction-3d-hipsc: dataset allen-label-free-prediction-3d-hipsc has no snippet that reads it.

Implemented by snippets: Claude Code running in a sandboxed job on the cluster. The agent's own
summary is the first comment.

• Merge: approve this PR, or comment /approve <head sha> (needed when running under a personal token; a
  trusted worker posts it when the output of this head is approved).
  snippets merges once an approval matches the current head and checks are not failing.
• Request changes: submit a review with Request changes, or comment /revise <what to change>.
  Up to 3 revision rounds.
• Stop: close this PR.

[mia-autodev]

snippets: round 1 pushed 04ea0dabe946 to snippets/issue-73 · $3.57

Added the snippet czi-targz-member (commit 04ea0da, not pushed); it reads the record's sample archive, and smoke and a 4-crop run both pass on the real data.

What it does

• Reader: one Zeiss CZI z-stack that is a member of a .tar.gz, returned as czyx, or as zyx when channel: <k> picks one channel.
• Reaching the member: the gzip stream is inflated from the archive's start through io.open range reads, up to the member's end. The 11 GB archive is never fetched whole.
• CZI parsing: standard library plus numpy, so no new allowlist dependency. It handles uncompressed grey subblocks, one plane per (C, Z).
• Refused at open: compressed subblocks, mosaics, and multiple scenes or timepoints, each with a message naming the cause.
• Params: member (tar member name; default is the first .czi) and channel (0-based index).
• Access: always file. In the tested file the directory and metadata come before the pixel data, so open costs 1 MiB and the first read loads the planes.

Example binding

examples/allen-label-free-prediction-3d-hipsc.yaml binds the first member of dic_lamin_b1.tar.gz with channel: 6. That is the transmitted-light channel, which the file names 63X_DIC. No labels are bound, since the record has restoration pairs only.

Testing

Smoke, run as CLAUDE.md prescribes with MIA_HARNESS_ALLOWED_HOSTS=s3-us-west-2.amazonaws.com:

{"ok": true, "shapes": {"raw": [[70, 624, 924]], "labels": []}, "warnings": [], "errors": [], "bytes_fetched": 301989888}

• Crops: harness crop --n 4 on the same binding made 4 of 4, with no warnings or errors and 301,989,888 bytes fetched.
• Real data: the member opens as czyx 7×70×624×924 uint16 with voxel size x/y 171.96 nm and z 290 nm, matching the record's notes. A rendered plane shows DIC cells in channel 6 and lamin B1 nuclear envelopes in channel 3.
• Synthetic CZI: an uncommitted script checked exact pixel round-trips for both directory-first and directory-last layouts, channel selection, pyramid subblocks being ignored, and each refusal message.
• CI checker: ci/check_pr.py --base 3621d82 exits 0.
• Test suites: 235 harness tests pass when run against this checkout's harness source, and 21 ci tests pass. Run against the pinned harness on PATH, 6 harness tests fail because that install is older than this checkout's tests (for example, IO has no fetch_all).

Things to know

• No committed tests: the repo rule allows only the four snippet files, so the synthetic test lives outside the repo.
• Trimmed record: the record.yaml used for smoke kept the fields smoke reads (technical.arrays) and dropped the layout tree and notes text.
• Byte cost grows with depth: each later member costs every compressed byte before it, so only the first several members of an archive fit under the 5 GB cap. Each CZI archive holds dozens of these stacks.
• Not yet exercised on real data: the 63x files (2 GB members) and the archive whose metadata sits at the end of the file (st6gal1). The directory-last path is covered only by the synthetic test.

[mia-autodev]

Review judge: approve

The PR adds a new czi-targz-member snippet. It reads one uncompressed grey Zeiss CZI z-stack from a .tar.gz member by streaming the gzip through io range reads, parses the CZI with the stdlib and numpy, and can pick one channel. It also includes the example binding the issue asked for: the first member of dic_lamin_b1.tar.gz, using the transmitted-light channel. The change stays inside one new snippet directory, declares only numpy, and follows the spec's I/O and import rules; its known limits (byte cap, memory, layout restrictions) are documented and none blocks merging.

item	severity	note
diff	ok	Adds only new files, all under one new snippets/czi-targz-member/ directory. No existing snippet, harness/, ci/, SPEC.md or .github/ path is touched (SPEC §5).
snippets/czi-targz-member/snippet.yaml	ok	The name matches the directory, requires is [numpy] only, formats is [czi], and tested_on names the one example binding that is present. The member and channel params are both declared.
snippets/czi-targz-member/snippet.py	ok	Imports only the stdlib (struct, tarfile, typing, xml.etree), numpy and mia_harness.api. open_array is typed as the spec requires. All network I/O goes through io.open, with an io.fetch fallback on NeedsFetch. No socket, subprocess or banned module.
snippets/czi-targz-member/snippet.py	ok	The CZI segment offsets look right: MetadataPosition at 60, directory entries at 128, the DV entry layout, the 256-byte metadata header, and the subblock data offset max(256, 16+entry)+metadata. It refuses compressed, mosaic and multi-scene files rather than guessing.
snippets/czi-targz-member/snippet.py	suspect	open_array reads headers only if the directory and metadata come before the pixel data. Otherwise it inflates and keeps the whole member at open, which strains the spec's 'must not read voxel data yet'. The README documents this as unavoidable for a stream with no index.
snippets/czi-targz-member/snippet.py	suspect	Any member is reachable only by inflating the archive from its first byte, so members deep in an archive can exceed the byte cap. This limit is inherent to tar.gz, the issue names it, and the README documents it.
snippets/czi-targz-member/examples/allen-label-free-prediction-3d-hipsc.yaml	ok	Binds the record's technical.sample archive (dic_lamin_b1.tar.gz) and its first member by name, with channel 6. That is the transmitted-light raw the record's alignment_notes describe (last channel in the DIC files). There are no labels, which fits restoration-pairs, and each non-default choice has a reason.
ci.json	suspect	CI was not awaited (shadow), so no check or mia-data/crops result is available. The smoke byte count given in the README cannot be verified from the evidence.

confidence 0.8 · model claude-opus-5-5 · prompt f5f9c39cc98e · policy v1 · $0.28

Shadow mode: this verdict is recorded and compared with the human label; it decides nothing.

[mia-autodev]

Crops at 04ea0dabe946347a4f4f9eeaf63a5b2cfc4831ed: passed, 16 of 16 verified (at least 8 needed); the crop job fetched 301989888 bytes.

16 crops

raw	overlay	caption
		i27492880-e1794718480222-b0-000 · z=65 c=None · (358, 531)
		i27492880-e1794718480222-b0-001 · z=62 c=None · (257, 605)
		i27492880-e1794718480222-b0-002 · z=45 c=None · (238, 595)
		i27492880-e1794718480222-b0-003 · z=6 c=None · (233, 299)
		i27492880-e1794718480222-b0-004 · z=21 c=None · (301, 651)
		i27492880-e1794718480222-b0-005 · z=28 c=None · (362, 24)
		i27492880-e1794718480222-b0-006 · z=8 c=None · (107, 251)
		i27492880-e1794718480222-b0-007 · z=15 c=None · (290, 30)
		i27492880-e1794718480222-b0-008 · z=15 c=None · (123, 349)
		i27492880-e1794718480222-b0-009 · z=18 c=None · (260, 153)
		i27492880-e1794718480222-b0-010 · z=44 c=None · (255, 309)
		i27492880-e1794718480222-b0-011 · z=14 c=None · (309, 420)
		i27492880-e1794718480222-b0-012 · z=22 c=None · (108, 381)
		i27492880-e1794718480222-b0-013 · z=55 c=None · (223, 137)
		i27492880-e1794718480222-b0-014 · z=20 c=None · (81, 616)
		i27492880-e1794718480222-b0-015 · z=22 c=None · (200, 13)

Binding (sha256 d688a09aec32815f0f8ee64a712d8b407ce875f0df6b9f5bc04b6a7ef42322d3):

record: allen-label-free-prediction-3d-hipsc
record_commit: 33a53209f11e2173560c3230304c194297b342ed
snippets_commit: 04ea0dabe946347a4f4f9eeaf63a5b2cfc4831ed
raw:
- snippet: czi-targz-member
  urls:
  - https://s3-us-west-2.amazonaws.com/downloads.allencell.org/publication-data/label-free-prediction/dic_lamin_b1.tar.gz
  params:
    member: dic_lamin_b1/DICLAMINB_100X_20170815_13_210_1-Scene-03-P3-B02.czi
    channel: 6
labels: []
reasons:
- 'czi-targz-member: the record''s data are tar.gz archives of Zeiss CZI z-stacks
  (data.formats czi, uncompressed Gray16 subblocks); a tar.gz has no index, so the
  member is read by inflating the archive''s gzip stream from its start'
- 'dic_lamin_b1.tar.gz: the record''s technical.sample, the smallest paired archive
  (10,995,482,838 bytes); the archive is not fetched whole, which would pass the job''s
  byte cap'
- 'member dic_lamin_b1/DICLAMINB_100X_20170815_13_210_1-Scene-03-P3-B02.czi: the archive''s
  first member (566,121,664 bytes, czyx 7x70x624x924 uint16 as in technical.arrays),
  so only the first 288 MiB of the archive are read (301,989,888 bytes in the smoke
  run); it is named rather than left to the default so the binding says which file
  it is. Each later member costs every compressed byte before it, so one member is
  bound, not the archive'
- 'channel 6: the record''s raw array is the transmitted-light channel, the last of
  the 7 in the DIC files (alignment_notes); the file names its channels CMDRP_2, CMDRP,
  EGFP_2, EGFP, H3342_2, H3342, 63X_DIC, and 63X_DIC (TL Brightfield) is index 6.
  The raw is zyx 70x624x924; without the param the sampler would draw crops from the
  six fluorescence channels, which are the record''s targets, not its input'
- 'No labels bound: annotations.types is restoration-pairs (fluorescence target channels
  of the same CZI), not segmentation masks'
- 'The snippet reads the voxel size from the file (Metadata/Scaling): x/y 171.96 nm,
  z 290 nm, the record''s note for dic_lamin_b1; the record''s voxel_size_nm is null
  because it differs per subset'
- snippets_commit is the commit before this snippet's own (a binding can't name the
  commit that adds it); a binding written by the worker names a commit that contains
  the snippet