Repository navigation
New snippet for tiff/http: empiar-10553-gemmata-fib-sem-membranes - #51
Open
mia-autodev[bot] wants to merge 2 commits into
Open
mia-autodev[bot] wants to merge 2 commits into
mia-autodev[bot] wants to merge 2 commits into
Conversation
…ized to a zyx binary mask For empiar-10553-gemmata-fib-sem-membranes (issue #43): parses the IMODV1.2 binary model over io.open, selects one object per model, and fills (even-odd) or outlines its contours on the model's image grid. Several models of one image are OR-ed. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Contributor
Author
snippets: round 1 pushed
|
| label_key | objects (C1, C2) | draw |
|---|---|---|
manual_gt-cell_envelope-semantic |
1, 1 | outline |
manual_gt-cytoplasmic_membrane-semantic |
2, 4 | outline |
manual_gt-nucleoid-semantic |
3, 3 | fill |
manual_gt-polyphosphate_granule-semantic |
4, 2 | fill |
Testing
- Smoke (
MIA_HARNESS_ALLOWED_HOSTS=ftp.ebi.ac.uk):ok: true, raw and all four labels[470, 1602, 2268],errors: [],bytes_fetched: 12692411. The only warnings are four oflabel N: dtype uint8 is not the record's ['unknown'], expected because the record has no dtype for polygon labels. - Orientation on real data: I overlaid the C1 outlines on raw plane 110 fetched through the harness. They sit on the cell envelope, cytoplasmic membrane and nucleoid, which confirms the y-flip (IMOD's y runs bottom-up) and the 0-based z.
- Synthetic models (ad hoc, not committed, since a PR may only add the snippet directory): fill with a nested hole, y-flip, z rounding, open-contour outline, multi-model union, sub-region and stepped reads, and each refusal path all pass.
- CI checker and harness suite:
ci/check_pr.pyagainst the base commit exits 0; the harness tests give 222 passed.
For the human reviewer
- Outline vs fill for the two membrane classes (issue's question b): I chose
outline, because the contours trace the membranes and filling them would give the whole cell and the cytoplasm compartments. All contours are closed, so switching tofillis a one-word change per label. - C2 model included (question c): its URL is not in
technical.sample.urls; I derived it fromdata.download_urlplus the layout tree. I merged both cells into one mask per class because two labels on one raw cannot share alabel_key. - Kind (question a): the snippet emits one binary mask per class and does not number cells as instances, hence
-semantickeys withsegmentation_type: binary. - Partial coverage: only two cells are modelled, on z 65..158 (C1) and 151..256 (C2). Zero elsewhere means "not annotated", not background.
- Possibly interpolated contours: 84 of the 94 C1 cell-envelope contours carry contour flag
0x20. I believe this marks interpolator-generated contours but did not confirm it; if so, "manual" there means about 10 drawn sections.
Contributor
Author
|
Crops at 16 crops
Binding (sha256 record: empiar-10553-gemmata-fib-sem-membranes
record_commit: bd965a656de8f35269c064a77912455b3d4058e7
snippets_commit: f971a6e181bcb14b9de09e57943c5afff6fdd204
raw:
- snippet: tiff-zip-member
urls:
- https://ftp.ebi.ac.uk/empiar/world_availability/10553/data/Seeger_5_reg.tif
params: {}
labels:
- snippet: imod-model-mask
urls:
- https://ftp.ebi.ac.uk/empiar/world_availability/10553/data/20190819_Gobs_model_C1.mod
- https://ftp.ebi.ac.uk/empiar/world_availability/10553/data/20190830_Gobs_model_C2.mod
params:
objects: 1,1
draw: outline
pairs_with: 0
label_key: manual_gt-cell_envelope-semantic
segmentation_type: binary
- snippet: imod-model-mask
urls:
- https://ftp.ebi.ac.uk/empiar/world_availability/10553/data/20190819_Gobs_model_C1.mod
- https://ftp.ebi.ac.uk/empiar/world_availability/10553/data/20190830_Gobs_model_C2.mod
params:
objects: 2,4
draw: outline
pairs_with: 0
label_key: manual_gt-cytoplasmic_membrane-semantic
segmentation_type: binary
- snippet: imod-model-mask
urls:
- https://ftp.ebi.ac.uk/empiar/world_availability/10553/data/20190819_Gobs_model_C1.mod
- https://ftp.ebi.ac.uk/empiar/world_availability/10553/data/20190830_Gobs_model_C2.mod
params:
objects: 3,3
pairs_with: 0
label_key: manual_gt-nucleoid-semantic
segmentation_type: binary
- snippet: imod-model-mask
urls:
- https://ftp.ebi.ac.uk/empiar/world_availability/10553/data/20190819_Gobs_model_C1.mod
- https://ftp.ebi.ac.uk/empiar/world_availability/10553/data/20190830_Gobs_model_C2.mod
params:
objects: 4,2
pairs_with: 0
label_key: manual_gt-polyphosphate_granule-semantic
segmentation_type: binary
reasons:
- 'raw: record technical.arrays[0] is one ImageJ TIFF stack (data/Seeger_5_reg.tif,
zyx, 470x1602x2268, uint8, uncompressed) at a plain https URL from technical.sample.urls,
so tiff-zip-member with default params (no axes override)'
- 'labels with imod-model-mask: the record''s label arrays are IMOD binary contour
models (format other, encoding polygons, ''Contours need rasterizing to masks'');
both model headers have dims 2268x1602x470, the raw''s grid, so each object rasterizes
to a zyx 470x1602x2268 mask that pairs with raw 0'
- 'both models in every label: C1 and C2 each outline one cell of the same stack with
the same four classes, and two labels of one raw cannot share a label_key, so each
class is one mask that is 1 where either model has it. The C2 URL is not in technical.sample.urls;
it is data.download_url plus the layout tree''s data/20190830_Gobs_model_C2.mod,
which path_pattern data/2019*_Gobs_model_C?.mod covers'
- 'objects per class, one number per url (C1, C2), from the OBJT names in the files:
cell envelope 1,1 (''Cell Envelope'', ''CE (cell1)''); cytoplasmic membrane 2,4
(''Cytoplasmic Membrane'', ''Cytoplasmic membrane (2)''); nucleoid / DNA 3,3 (''Nucleoid'',
''DNA''); large poly-phosphate granule 4,2 (''Large Poly-P granule'', ''Large poly-P
granule''). The two models order their objects differently, hence objects and not
object'
- 'draw: outline for cell envelope and cytoplasmic membrane: the contours trace the
membranes, so the outline is the class the record names; filled they would be the
whole cell and the cytoplasm (with invaginations as holes). All contours of both
models are closed, so draw: fill also works: a human should confirm which is wanted
(the record does not say)'
- 'draw: fill (default) for nucleoid / DNA and the poly-phosphate granule: the contours
are drawn around bodies, so the class is the enclosed region'
- 'label_keys manual_gt-<class>-semantic with segmentation_type binary: annotations
are manual (provenance manual_gt), each mask is {0,1} for one class, and binary
folds to semantic in the key. Encoding ''polygons'' has no kind in the label_key
rule; the rasterizer emits one binary mask per class and does not number the two
cells as instances'
- 'coverage is partial: only two cells of the stack are modelled, and only on the
sections that carry contours (C1 z 65..158, C2 z 151..256); everywhere else the
masks are 0, which is ''not annotated'', not ''background'''
- 'voxel size: none from either array (TIFF uncalibrated; the snippet does not report
the models'' pixsize 5 nm and z scales 4.49 / 4.05, which disagree and which the
record calls unreliable)' |
Contributor
Author
|
/revise the crops at Test crops at
|
Contributor
Author
|
Round 2 started: agent job |
The mia-data crops rejected the binding because its record_commit (bd965a6) was not the record's current commit (b4c57a6). Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Contributor
Author
snippets: round 2 pushed
|
Contributor
Author
|
Crops at 16 crops
Binding (sha256 record: empiar-10553-gemmata-fib-sem-membranes
record_commit: b4c57a61ef1f5cde7a19c4bcd5320f909c43f406
snippets_commit: fb9ec63e1e4b8f665dfc8061cb0a10a280f909b0
raw:
- snippet: tiff-zip-member
urls:
- https://ftp.ebi.ac.uk/empiar/world_availability/10553/data/Seeger_5_reg.tif
params: {}
labels:
- snippet: imod-model-mask
urls:
- https://ftp.ebi.ac.uk/empiar/world_availability/10553/data/20190819_Gobs_model_C1.mod
- https://ftp.ebi.ac.uk/empiar/world_availability/10553/data/20190830_Gobs_model_C2.mod
params:
objects: 1,1
draw: outline
pairs_with: 0
label_key: manual_gt-cell_envelope-semantic
segmentation_type: binary
- snippet: imod-model-mask
urls:
- https://ftp.ebi.ac.uk/empiar/world_availability/10553/data/20190819_Gobs_model_C1.mod
- https://ftp.ebi.ac.uk/empiar/world_availability/10553/data/20190830_Gobs_model_C2.mod
params:
objects: 2,4
draw: outline
pairs_with: 0
label_key: manual_gt-cytoplasmic_membrane-semantic
segmentation_type: binary
- snippet: imod-model-mask
urls:
- https://ftp.ebi.ac.uk/empiar/world_availability/10553/data/20190819_Gobs_model_C1.mod
- https://ftp.ebi.ac.uk/empiar/world_availability/10553/data/20190830_Gobs_model_C2.mod
params:
objects: 3,3
pairs_with: 0
label_key: manual_gt-nucleoid-semantic
segmentation_type: binary
- snippet: imod-model-mask
urls:
- https://ftp.ebi.ac.uk/empiar/world_availability/10553/data/20190819_Gobs_model_C1.mod
- https://ftp.ebi.ac.uk/empiar/world_availability/10553/data/20190830_Gobs_model_C2.mod
params:
objects: 4,2
pairs_with: 0
label_key: manual_gt-polyphosphate_granule-semantic
segmentation_type: binary
reasons:
- 'raw: record technical.arrays[0] is one ImageJ TIFF stack (data/Seeger_5_reg.tif,
zyx, 470x1602x2268, uint8, uncompressed) at a plain https URL from technical.sample.urls,
so tiff-zip-member with default params (no axes override)'
- 'labels with imod-model-mask: the record''s label arrays are IMOD binary contour
models (format other, encoding polygons, ''Contours need rasterizing to masks'');
both model headers have dims 2268x1602x470, the raw''s grid, so each object rasterizes
to a zyx 470x1602x2268 mask that pairs with raw 0'
- 'both models in every label: C1 and C2 each outline one cell of the same stack with
the same four classes, and two labels of one raw cannot share a label_key, so each
class is one mask that is 1 where either model has it. The C2 URL is not in technical.sample.urls;
it is data.download_url plus the layout tree''s data/20190830_Gobs_model_C2.mod,
which path_pattern data/2019*_Gobs_model_C?.mod covers'
- 'objects per class, one number per url (C1, C2), from the OBJT names in the files:
cell envelope 1,1 (''Cell Envelope'', ''CE (cell1)''); cytoplasmic membrane 2,4
(''Cytoplasmic Membrane'', ''Cytoplasmic membrane (2)''); nucleoid / DNA 3,3 (''Nucleoid'',
''DNA''); large poly-phosphate granule 4,2 (''Large Poly-P granule'', ''Large poly-P
granule''). The two models order their objects differently, hence objects and not
object'
- 'draw: outline for cell envelope and cytoplasmic membrane: the contours trace the
membranes, so the outline is the class the record names; filled they would be the
whole cell and the cytoplasm (with invaginations as holes). All contours of both
models are closed, so draw: fill also works: a human should confirm which is wanted
(the record does not say)'
- 'draw: fill (default) for nucleoid / DNA and the poly-phosphate granule: the contours
are drawn around bodies, so the class is the enclosed region'
- 'label_keys manual_gt-<class>-semantic with segmentation_type binary: annotations
are manual (provenance manual_gt), each mask is {0,1} for one class, and binary
folds to semantic in the key. Encoding ''polygons'' has no kind in the label_key
rule; the rasterizer emits one binary mask per class and does not number the two
cells as instances'
- 'coverage is partial: only two cells of the stack are modelled, and only on the
sections that carry contours (C1 z 65..158, C2 z 151..256); everywhere else the
masks are 0, which is ''not annotated'', not ''background'''
- 'voxel size: none from either array (TIFF uncalibrated; the snippet does not report
the models'' pixsize 5 nm and z scales 4.49 / 4.05, which disagree and which the
record calls unreliable)' |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #43: For JaneliaSciComp/mia-workflow-data#222: dataset
empiar-10553-gemmata-fib-sem-membraneshas no snippet that reads it.Implemented by snippets: Claude Code running in a sandboxed job on the cluster. The agent's own
summary is the first comment.
/approve <head sha>(needed when running under a personal token; atrusted worker posts it when the output of this head is approved).
snippets merges once an approval matches the current head and checks are not failing.
/revise <what to change>.Up to 3 revision rounds.