Skip to content

New snippet for ome-zarr/http: allen-emt-timelapse-hips-colonies - #81

Open
mia-autodev[bot] wants to merge 1 commit into
mainfrom
snippets/issue-75
Open

mia-autodev[bot] wants to merge 1 commit into
mainfrom
snippets/issue-75

Conversation

@mia-autodev

@mia-autodev mia-autodev Bot commented Nov 15, 2026

Copy link
Copy Markdown
Contributor

Closes #75: For allen-emt-timelapse-hips-colonies: dataset allen-emt-timelapse-hips-colonies has no snippet that reads it.

Implemented by snippets: Claude Code running in a sandboxed job on the cluster. The agent's own
summary is the first comment.

  • Merge: approve this PR, or comment /approve <head sha> (needed when running under a personal token; a
    trusted worker posts it when the output of this head is approved).
    snippets merges once an approval matches the current head and checks are not failing.
  • Request changes: submit a review with Request changes, or comment /revise <what to change>.
    Up to 3 revision rounds.
  • Stop: close this PR.

…mepoint (t), tczyx presented as czyx

Reads a level array of an OME-Zarr image (Zarr v2, chunk keys from .zarray, no
listing) and fixes its time axis at params.t, so the rank-5 tczyx arrays of
allen-emt-timelapse-hips-colonies open as czyx. Axis order and voxel size come
from the image group's multiscales; a one-entry channel axis is not presented,
so a one-channel mask pairs with every channel of its raw.

Closes #75

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@mia-autodev

mia-autodev Bot commented Nov 15, 2026

Copy link
Copy Markdown
Contributor Author

snippets: round 1 pushed 350884b530ff to snippets/issue-75 · $3.72

The new snippet ome-zarr2-http-timepoint is committed (350884b, not pushed) and reads the record's fixed image and its all-cells mask: smoke is ok and harness crop --n 4 makes 4 crops with no errors.

What changed

One new directory, snippets/ome-zarr2-http-timepoint/ (snippet.py, snippet.yaml, README.md, examples/allen-emt-timelapse-hips-colonies.yaml); nothing else is touched.

  • Reader: opens one level array of an OME-Zarr v2 image from its .zarray URL and reads chunk objects by key, with no listing. Open requests only the array's .zarray and the group's .zattrs.
  • Time axis: not presented. params.t picks the index; it is required when T > 1 (open raises, naming the range) and optional when T = 1.
  • Axes and voxel size: taken from the group's multiscales (axis names, level scale × unit → nm). This image gives z 2880, y 270.83, x 270.83 nm.
  • One-entry channel axis is dropped, so the mask opens as zyx. I added this after the first crop run: the harness reads a label's c at the raw crop's channel index, so a czyx one-channel mask raised IndexError for every crop from raw channels 1–3. Smoke does not catch this.
  • Binding: the issue's draft with the snippet renamed, axes: tczyx removed, and no t (both stores have T = 1); the reasons are rewritten to match.

Smoke summary

{"ok": true, "shapes": {"raw": [[4, 30, 1248, 1848]], "labels": [[30, 1248, 1848]]}, "warnings": [], "errors": [], "bytes_fetched": 4623507}

Testing

  • Example binding: harness smoke ok; harness crop --n 4 made 4 crops from raw channels 0, 1 and 3, with no errors. The one overlay I viewed (bright-field) has mask outlines along the cell boundaries.
  • harness verify: fails all 4 crops with "crop.json has no licence". The issue's Record block has no licence field; verify reported nothing else.
  • Timelapse, outside the binding: 3500006256_11 (raw T = 121, 3 channels) and its all-cells mask pass smoke and crop --n 4 with t: 60 on both. Without t, open raises "has 121 timepoints … set params.t (0..120 …)".
  • Synthetic arrays: 23 throwaway checks against arrays written by zarr-python over the local range server pass. They cover every t against source data, the four compressors and uncompressed, both separators, a missing chunk, big-endian data, 0.3 axes, missing metadata and units. They are not committed, since a PR may only add the snippet directory.
  • Repo suites: harness tests 235 passed, CI-checker tests 21 passed, and ci/check_pr.py --base origin/main exits 0.
  • Environment: pixi is not on PATH, so I ran the suites with the pinned harness environment's Python. That environment's installed mia_harness is older than this checkout, and 6 harness tests fail against it. All 235 pass with PYTHONPATH=harness/src, and crops under this checkout's harness also succeed.

Not covered

  • H2B nuclear segmentations are not bound: the label has 120 timepoints against the raw's 121, and the stores don't say which raw timepoint is missing. The README says not to bind them with this snippet until that is known.
  • One timepoint per bound array: to crop several timepoints of a timelapse, the binding lists the store once per t.

@mia-autodev

mia-autodev Bot commented Nov 15, 2026

Copy link
Copy Markdown
Contributor Author

Review judge: approve

The PR adds a new snippet, ome-zarr2-http-timepoint, which reads one level of a Zarr v2 OME-Zarr image over HTTP(S) at one timepoint. It presents the tczyx array as czyx (or zyx when there is one channel) and takes axes and voxel size from the group's multiscales metadata. It adds the requested example binding for the Allen EMT dataset's fixed sample, stays within one new snippet directory and appears to follow the spec's import, I/O and metadata rules.

item severity note
diff ok Only adds files under the new snippets/ome-zarr2-http-timepoint/ directory (snippet.py, snippet.yaml, README.md, examples/.yaml), which follows SPEC §1 and §5. No existing snippet, harness, ci, or SPEC path is touched.
snippets/ome-zarr2-http-timepoint/snippet.py ok Does what the issue asks: reads a rank-5 tczyx Zarr v2 array with blosc chunks, takes chunk keys from .zarray without listing the store, picks one timepoint with params.t, presents the result as czyx, and takes voxel size from the OME-NGFF multiscales scale in the group's .zattrs.
snippets/ome-zarr2-http-timepoint/snippet.py ok Imports only mia_harness.api, numpy, imagecodecs and the stdlib modules json, zlib and typing. All I/O goes through io.open/io.fetch with a NeedsFetch fallback, open_array has the required type hints, and no voxel data is read at open.
snippets/ome-zarr2-http-timepoint/snippet.py suspect A 404 is detected through e.response.status_code on whatever io raises. Whether the harness's exceptions carry that attribute is not shown in the evidence, so missing chunks or a missing .zattrs might raise instead of falling back.
snippets/ome-zarr2-http-timepoint/snippet.py ok A channel axis with one entry is dropped, so the mask opens as zyx (30,1248,1848) against raw czyx (4,30,1248,1848). This fits SPEC §4 (labels must match raw on every axis except c), and the README gives the reason.
snippets/ome-zarr2-http-timepoint/snippet.yaml ok name matches the directory, the only declared param is t, requires [numpy, imagecodecs] are the same packages SPEC's example uses, and tested_on matches the example binding.
snippets/ome-zarr2-http-timepoint/examples/allen-emt-timelapse-hips-colonies.yaml ok The binding uses the record's sample URLs with /0/.zarray appended, keeps the record and snippets commits from the issue, gives a reason for each decision, and leaves out t because both arrays have one timepoint.
snippets/ome-zarr2-http-timepoint/README.md ok Documents the H2B 120-vs-121 timepoint mismatch the issue raises and says not to bind those labels with this snippet, which matches the issue's note that this pairing needs more than a snippet.
ci.json suspect CI was not awaited (shadow), so neither the check job nor the real-data mia-data/crops status confirms the change. The smoke and crop runs described in the README are the agent's own claims.

confidence 0.85 · model claude-opus-5-5 · prompt f5f9c39cc98e · policy v1 · $0.28

Shadow mode: this verdict is recorded and compared with the human label; it decides nothing.

@mia-autodev

mia-autodev Bot commented Nov 15, 2026

Copy link
Copy Markdown
Contributor Author

Crops at 350884b530ff68b6a7a2abba5e87553901b5d560: passed, 16 of 16 verified (at least 8 needed); the crop job fetched 48051975 bytes.

16 crops
raw overlay caption
raw overlay i27493373-e1794718973781-b0-000 · z=28 c=0 · (74, 1137)
raw overlay i27493373-e1794718973781-b0-001 · z=12 c=2 · (425, 737)
raw overlay i27493373-e1794718973781-b0-002 · z=24 c=3 · (517, 1263)
raw overlay i27493373-e1794718973781-b0-003 · z=20 c=2 · (529, 639)
raw overlay i27493373-e1794718973781-b0-004 · z=15 c=3 · (53, 943)
raw overlay i27493373-e1794718973781-b0-005 · z=15 c=0 · (841, 824)
raw overlay i27493373-e1794718973781-b0-006 · z=19 c=3 · (318, 660)
raw overlay i27493373-e1794718973781-b0-007 · z=27 c=2 · (529, 74)
raw overlay i27493373-e1794718973781-b0-008 · z=15 c=2 · (706, 1286)
raw overlay i27493373-e1794718973781-b0-009 · z=18 c=0 · (413, 1259)
raw overlay i27493373-e1794718973781-b0-010 · z=1 c=1 · (886, 1377)
raw overlay i27493373-e1794718973781-b0-011 · z=8 c=0 · (900, 1047)
raw overlay i27493373-e1794718973781-b0-012 · z=14 c=2 · (828, 1327)
raw overlay i27493373-e1794718973781-b0-013 · z=9 c=3 · (991, 274)
raw overlay i27493373-e1794718973781-b0-014 · z=18 c=3 · (449, 519)
raw overlay i27493373-e1794718973781-b0-015 · z=10 c=3 · (611, 187)

Binding (sha256 3b931fcf079bf00102131b42c7a4b930968d38d9200341b551fb1b078cb4c093):

record: allen-emt-timelapse-hips-colonies
record_commit: 33a53209f11e2173560c3230304c194297b342ed
snippets_commit: 350884b530ff68b6a7a2abba5e87553901b5d560
raw:
- snippet: ome-zarr2-http-timepoint
  urls:
  - https://allencell.s3.amazonaws.com/aics/emt_timelapse_dataset/data/3500007015_10_E3_N-cadherin_E-cadherin_raw_converted.ome.zarr/0/.zarray
  params: {}
labels:
- snippet: ome-zarr2-http-timepoint
  urls:
  - https://allencell.s3.amazonaws.com/aics/emt_timelapse_dataset/data/3500007015_10_EcadNcad_all_cells_mask.ome.zarr/0/.zarray
  params: {}
  pairs_with: 0
  label_key: unknown-cell-semantic
  segmentation_type: binary
reasons:
- 'ome-zarr2-http-timepoint: the record''s stores are OME-Zarr images (Zarr v2, OME-NGFF
  0.4, blosc-lz4, dimension_separator ''/'') in an S3 bucket read over https, and
  their level arrays are rank 5 in tczyx order; an Array has no t axis, so the reader
  presents one timepoint as czyx. zarr2-http, the other zarr reader, has no way to
  drop the time axis and does not read the multiscales scale'
- 'urls are the record''s technical.sample.urls (fixed 4-channel image 3500007015_10
  and its all-cells mask, paired by Data ID in the manifest although the file stems
  differ) with ''/0/.zarray'' appended: the snippet takes one level array, not the
  image group; level 0 is the full-resolution array (1, 4, 30, 1248, 1848), and the
  .zarray is an object that exists, so `harness fetch` and the prefetch of `harness
  stage`, which fetch every binding url whole, get a file with validators instead
  of the directory''s 404'
- 'no t param on either array: both stores are fixed samples with a time axis of length
  1 (raw (1, 4, 30, 1248, 1848), mask (1, 1, 30, 1248, 1848)), so the only timepoint,
  0, is taken on the raw and on its label alike. A live timelapse of this record (T
  up to 1001) needs an explicit t, the same on the raw and its all-cells mask'
- 'axes: the group''s multiscales axes are t, c, z, y, x, matching the record''s tczyx.
  The raw is presented without t as czyx 4 x 30 x 1248 x 1848 (channels bright-field,
  mEGFP and two immunostains per the store''s omero metadata); the mask''s channel
  axis has one entry, which the snippet does not present, so the label is zyx 30 x
  1248 x 1848 and is read at the raw''s z, y, x whichever raw channel a crop is drawn
  from'
- voxel size z 2880, y 270.83, x 270.83 nm comes from the multiscales scale of dataset
  '0' (2.88, 0.2708, 0.2708 micrometer); the record's imaging.voxel_size_nm is null
  because it differs per image
- 'label_key: annotations.source ''mixed'' -> unknown; all_cells_mask marks cells
  -> cell; encoding binary -> semantic; segmentation_type binary (uint8 values {0,
  255})'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

New snippet for ome-zarr/http: allen-emt-timelapse-hips-colonies

0 participants