Don't read selection indices past the end of variably shaped cells

[sjperkins]

Summary

Reading a variably shaped column with a secondary-dimension selection fails if any selected cell is shorter than the selection:

ArrowIndexError: Selection index 2 exceeds dimension 1 of shape [2, 2] in column VAR

dask-ms hits this whenever it chunks a secondary dimension of a ragged column (e.g. chan chunked as (3, 1) over cells with 4 and 2 channels). To avoid it, dask-ms reads whole cells into a buffer of the maximal shape and slices the block out in numpy, so each row is read once per chunk.

With this PR, when getcol is given a result buffer, selection indices past the end of a variably shaped cell are handled like -1 indices: they are not read, and those positions of the result are left untouched. This matches #247, which pads result buffers that are larger than a cell. dask-ms can then pass each block's selection straight through.

Implementation

• Validate and pad cell shapes on reads and writes, fix GIL deadlock #247's per-row cell extents become cell bounds: for each row and dimension, an upper limit on the disk indices read from that cell. The same rule now covers both cases: indices outside the cell in a selected dimension, and result dimensions larger than the cell in an unselected dimension.
• If all selected rows share a bound, RebuildSelection rewrites the selection once, replacing the out-of-range indices with -1. Blocks that fit inside every cell keep the original selection.
• If bounds differ by row, DataPartition::Make masks each row's disk indices against that row's bound. This reuses the per-row path from Validate and pad cell shapes on reads and writes, fix GIL deadlock #247.

Limits (these cases still raise)

• Fixed-shape columns: an index past the end is a caller error.
• Reads without a result: arcae allocates the buffer uninitialised, so skipped positions would contain garbage.
• Writes: putcol past the end of an existing cell is a genuine shape mismatch.

Behaviour change: on a variably shaped column with a result buffer, an index that is past the end of every cell now pads silently instead of raising. A variably shaped column has no overall bound to check against, and a block lying entirely past a group of short rows is a legitimate dask-ms read.

Tests

• C++ (result_shape_test): common-bound rewriting with unsorted indices, a selection entirely past the cells, per-row bounds, fixed-shape columns still raising, and MakeRead without a result still raising.
• Python: test_getcol_selection_past_cell compares reads against a NumPy reference computed row by row. It covers row orders, unsorted and explicit -1 channel selections, undefined rows, and correlation padding. Also added: test_putcol_selection_past_cell, and an updated test_getcol_result_explicit_padding.
• ctest 10/10 passed; pytest 238 passed, 4 skipped, 1 xfailed.

🤖 Generated with Claude Code

[sjperkins]

Treating this as an unlikely edge case for now. Closing.