Skip to content

fix(ooc): make the 2D out-of-core feature paths match the in-RAM path - #395

Merged
sameeul merged 1 commit into
PolusAI:mainfrom
darkclad:main-io-2d-ooc-fix
Aug 5, 2026
Merged

fix(ooc): make the 2D out-of-core feature paths match the in-RAM path#395
sameeul merged 1 commit into
PolusAI:mainfrom
darkclad:main-io-2d-ooc-fix

Conversation

@darkclad

@darkclad darkclad commented Jul 24, 2026

Copy link
Copy Markdown

Re-cut on main per review. This branch previously sat on top of the IO series (#389, since
closed and re-split into #397#400), which made it unreviewable on its own. It is now a single
commit on main
with no dependency on any IO PR, and the fixes carry pytest CI coverage.

The problem

Nyxus computes every feature along two paths:

path trigger pixel source entry point
trivial (in-RAM) ROI footprint < ram_limit LR::raw_pixels / LR::aux_image_matrix calculate()
non-trivial (out-of-core) ROI footprint ≥ ram_limit LR::raw_pixels_NT (disk-backed) osized_calculate()

They were separate hand-written implementations of the same math, and the osized_* copies had
drifted: older binning helpers, older formulas, older degenerate-ROI conventions, and none of the bug
fixes their in-RAM twins received. 9 of 14 2D feature families disagreed; GLRLM and NGTDM crashed;
and intensity features could not produce a row at all.

The approach

Structural, not arithmetic: stop maintaining a second implementation. Each 2D osized_calculate()
now materializes the ROI from its disk-backed cloud via two new LR helpers
(rebuild_raw_pixels_from_cloud, rebuild_aux_image_matrix_from_cloud) and calls its own
calculate(). Agreement between the paths becomes a property of the code shape rather than something
to re-establish after every future edit. ~2100 lines of drifted duplicate streaming code are deleted,
including the whole 2d_geomoments_basic_nt.cpp translation unit, GLCMFeature's
Extract_Texture_Features2_NT / calculateCoocMatAtAngle_NT, GaborFeature's GaborEnergy_NT2 /
conv_dud_NT / GetStats_NT, and ZernikeFeature::mb_zernike2D_nontriv.

Doesn't materializing the ROI defeat out-of-core? For 2D, no: every one of these families already
materialized the full ROI, either as a disk-backed WriteImageMatrix_nontriv of the same ROI or as an
in-RAM std::vector copy of the cloud. The streaming was never a smaller footprint — it was the same
footprint with every pixel access turned into a file read (erosion: ~285 s vs 0.23 s on a 100×100 ROI,
~1250×, for an identical result). Genuine 3D whole-volume streaming is a separate mechanism and is
untouched here.

What was wrong, by family

Six of the eight dense-image families share one root cause: the trivial path moved to
TextureFeature::bin_intensities() and the streaming copies kept to_grayscale(). Two crash on it;
four return silently wrong numbers, which is worse.

family symptom out-of-core
GLCM co-occurrence index exceeded the trivially-sized matrix → threw "invalid vector subscript"
GLRLM ran off the end of the run-length matrix → out-of-bounds crash
NGTDM binning drift plus N/S/P indexed by raw grey level against unique-count-sized vectors → out-of-bounds crash
GLSZM, GLDM zone / dependence scan matched no grey levels → every feature 0
GLDZM zones grouped by different grey levels → wrong values
GABOR originalScore baseline never assigned (left 0), so each frequency divided by the tiny-number floor → astronomically large values instead of a ratio in [0,1]
ZERNIKE2D agreed on ordinary ROIs but lacked calculate()'s constant-ROI guard
2D geometric moments intensity and shape moments shared one osized_calculate that dropped the intenfunction, so shape moments summed intensities (SPAT_MOMENT_00 returned the intensity sum, not the ROI area)
CHORDS subsampled — stepped over columns instead of scanning each, so max/min/median chord lengths ran short
ellipse fitting eccentricity = sqrt(1 − major²/minor²) → negative radicand → NaN; elongation = sqrt(minor/major) instead of minor/major
erosion ran the erosion chain over disk-backed matrices, and lacked the in-RAM driver's constant-ROI skip so it reported the sanity cap where in-RAM reports 0

2D NGLDM needed no change — it already used one shared gather_unique_intensities() template for both
paths, so it never drifted. It is asserted equal by the new tests regardless.

Also fixed

Out-of-core dispatch — without this, no oversized 2D ROI produces an intensity row.
PixelIntensityFeatures and IntensityHistogramFeatures need slide properties for
COVERED_IMAGE_INTENSITY_RANGE, so their real work lives in a Dataset-taking osized_calculate and
their Dataset-less overload is a guard that throws — but the generic
FeatureMethod::osized_scan_whole_image called exactly that guard. Every out-of-core run requesting
intensity features failed with "illegal use of PixelIntensityFeatures::osized_calculate()". The
dispatch now carries the Dataset; the default implementation ignores it, so every other feature
is behaviourally unchanged, and 3D is untouched.

Nine intensity features returned 0 out-of-core. osized_calculate accumulated its dispersion and
median-AD loops over r.raw_pixels, which an oversized ROI never populates — zeroing
MEAN_ABSOLUTE_DEVIATION, VARIANCE, VARIANCE_BIASED, STANDARD_DEVIATION,
STANDARD_DEVIATION_BIASED, COV, STANDARD_ERROR and MEDIAN_ABSOLUTE_DEVIATION. Alongside:
EXCESS_KURTOSIS used KURTOSIS − 3 where in-RAM uses Moments4::excess_kurtosis() (equal on
ordinary data, −3 vs 0 on a constant ROI), and hyperskewness/hyperflatness now use the in-RAM
sum-of-powers definition.

ROBUST_MEAN_ABSOLUTE_DEVIATION was 0 out-of-core — the same duplicate-implementation disease one
level down. TrivialHistogram::initialize() had a hand-copied OutOfRamPixelCloud overload beside the
template; the copy thresholded the robust mean on bin counts (bins100_[10], bins100_[90])
instead of percentile values (p10_, p90_) and never normalized mean1090val_ — a stale copy of
a bug already fixed in the template (in-RAM 9774.587 vs 0.0). Both sources now share one
template <class Src>.

Degenerate-ROI and cross-ROI state hygiene. The out-of-core loop reuses one persistent
feature-method instance across ROIs, where the in-RAM path constructs a fresh one per ROI, so any
member left un-reset on an early-return path reports the previous ROI's value. Zernike's blank-ROI
guard used coeffs.resize(...) — a no-op at unchanged length — and now uses assign(); erosion resets
its count before its skip; LR::aux_min/aux_max were uninitialized scalars read by every
family's aux_min == aux_max blank-ROI guard and are now default-initialized; and the out-of-core scan
recomputes the extrema from the pixels it actually streamed instead of trusting a scalar carried over
from phase 1.

Fail loudly instead of emitting a wrong row. processIntSegImagePairInMemory took its
unprocessed_rois list by value, so the in-memory montage path — which has no out-of-core support
— never reported an oversized ROI and returned a frame of zeros. Now a reference.

One deliberate in-RAM behaviour change

Everything above changes only the out-of-core path. This one does not, so it is called out separately:
PixelIntensityFeatures::calculate() intercepted a constant-intensity ROI and set every intensity
feature except MEAN/MEDIAN/MIN/MAX/RANGE to the soft-NAN sentinel
— including
INTEGRATED_INTENSITY, ENERGY, MODE, ROOT_MEAN_SQUARED, the percentiles and the dispersion
measures, all of which are perfectly well defined on a constant ROI (the dispersion ones are simply 0).
The out-of-core path computed them; in-RAM discarded them. The interception is removed — the general
code below it handles a constant ROI correctly.

Consequence: featurizing a constant-intensity ROI in-RAM now returns real numbers in columns that
previously held the sentinel. Resolving the disagreement the other way would have meant preserving a
loss of information, so this is the direction chosen. Flagging it as a product decision, not a side
effect.

Tests

Two new files, split by test kind per tests/vetting/SPEC.md §2 (which requires the four kinds to
live in separate files) — 10 tests total, running in CI:

  • tests/python/test_ooc_invariant.py (9 tests) — the required relation trivial == out-of-core.
    SPEC §2 defines invariant as "does output obey a required property/bound/relation?", which is exactly
    this.
  • tests/python/test_ooc_mechanics.py (1 test) — gating only: the montage path must fail loudly.

Each invariant test featurizes the same input twice (ram_limit large → in-RAM, 0 or 1 →
out-of-core) and requires every numeric column to agree within 1e-6·max(|a|,|b|,1) + 1e-9 — tighter
than SPEC §7's rel=1e-3 floor for a same-definition comparison, since both sides are the same build
and the same formula and the only admissible difference is float summation order:

test scope
test_ooc_2d_matches_in_ram *ALL_INTENSITY*, 500×500, ram_limit=1 (real byte-threshold oversizing)
test_ooc_2d_texture_matches_in_ram all 7 texture families, incl. the two that crashed
test_ooc_2d_moments_matches_in_ram *GEOMOMS*
test_ooc_2d_gabor_matches_in_ram GABOR
test_ooc_2d_morphology_matches_in_ram *ALL_MORPHOLOGY*
test_ooc_2d_zernike_matches_in_ram ZERNIKE2D
test_ooc_2d_blank_matches_in_ram all 14 groups on a degenerate constant-intensity ROI
test_ooc_2d_all_groups_match_in_ram all 14 groups on an ordinary ROI — the umbrella invariant
test_ooc_2d_two_rois_no_state_leak a gradient ROI and a constant ROI in one image — the instance-reuse mechanism above
test_ooc_montage_oversized_fails_loudly (mechanics) montage path must raise, not return zeros

ram_limit is a process-global, so each test sets it on both sides. Nyxus also refuses a limit
above currently-available RAM and keeps the previous value without raising, so the helper reads the
value back and asserts it was accepted — otherwise a refused setting silently sends the "in-RAM" side
out-of-core and the failure surfaces somewhere unrelated. test_tiff_loader.py's
test_uint32_tiff_from_disk_matches_in_memory gets the same treatment, so a preceding out-of-core test
can no longer leave the global at 0 and make even a 3×4 ROI "oversized".

Neither kind claims vetting: they assert that two Nyxus code paths agree, not that either matches an
external oracle, so no oracle_coverage.csv row is touched (check_coverage.py has no status that
could express an invariant or mechanics assertion — it allows only vetted/regression/untested,
and rejects a non-vetted row carrying an oracle). No oracle token appears in any file or function
name. The correct reading is "2D out-of-core equals in-RAM, which is separately vetted" — not
"2D out-of-core is correct".

Verification

gate result
gtest (whole binary, MSVC Release) 722/722
pytest (whole directory) 82 pass, 1 skip; 7 pre-existing Apache Arrow functionality is not available failures on the tiff-only build
ASan + UBSan gtest (gcc 13.3) 718/718 (4 TEST_HU_LOADER_DICOM_* are DCMTK-gated)
ASan + UBSan pytest 82 pass, 1 skip, same 7 Arrow
AddressSanitizer / LeakSanitizer / CHECK-failed / UBSan reports 0 / 0 / 0 / 0
tests/vetting/check_coverage.py --check exit 0; registry untouched, drift unchanged

Scope

@vjaganat90

Copy link
Copy Markdown
Member

This looks fine but can't be merged until PR 389. I suggest, the 2D fixes are real correctness/crash blockers for OOC, they should get CI coverage via pytest, and they are portable to main without #389. As a short rebased/port PR.

As is this PR can't be reviewed or merged.

@darkclad
darkclad force-pushed the main-io-2d-ooc-fix branch 2 times, most recently from 4978b1c to 14073bd Compare August 5, 2026 15:42
@darkclad darkclad changed the title Fix 2D out-of-core feature paths to match the in-RAM path fix(ooc): make the 2D out-of-core feature paths match the in-RAM path Aug 5, 2026
The osized_calculate implementations of the 2D texture (GLCM, GLRLM, NGTDM,
GLSZM, GLDM, GLDZM), moments, gabor, chords, ellipse-fitting, erosion and
zernike features had drifted from their calculate() twins: they binned
intensities with to_grayscale() instead of TextureFeature::bin_intensities(),
GLRLM and NGTDM indexed their matrices out of bounds and crashed, GLCM threw
"invalid vector subscript", GLSZM and GLDM returned 0, gabor never assigned its
originalScore baseline so every frequency was divided by the tiny-number floor,
geomoments funneled intensity and shape moments through one shared path that
dropped the intenfunction (so shape moments summed raw intensities), chords
stepped over columns instead of scanning every one, and ellipse fitting used
drifted eccentricity/elongation formulas.

Each osized_calculate now materializes the ROI from the disk-backed pixel cloud
through the new LR::rebuild_raw_pixels_from_cloud and
LR::rebuild_aux_image_matrix_from_cloud helpers and reuses calculate(), so the
trivial and non-trivial paths agree by construction rather than by duplicated
logic. The out-of-core-only code left with no caller after this is removed:
GLCMFeature's Extract_Texture_Features2_NT and calculateCoocMatAtAngle_NT,
GaborFeature's GaborEnergy_NT2 / conv_dud_NT / GetStats_NT,
ZernikeFeature::mb_zernike2D_nontriv, and the whole
2d_geomoments_basic_nt.cpp translation unit.

Also fixes the defects that stopped the out-of-core path from producing any
usable intensity row at all, and the ones found while probing degenerate
(constant-intensity) and multi-ROI inputs:

- PixelIntensityFeatures and IntensityHistogramFeatures reached the out-of-core
  dispatch through FeatureMethod::osized_scan_whole_image, which called their
  Dataset-less osized_calculate -- a guard that throws -- so every oversized 2D
  ROI failed with "illegal use of ...::osized_calculate()". The dispatch now
  carries the Dataset and those two features override it; every other feature is
  unaffected.
- The intensity out-of-core path accumulated MEAN_ABSOLUTE_DEVIATION, VARIANCE,
  STANDARD_DEVIATION, COV, STANDARD_ERROR and MEDIAN_ABSOLUTE_DEVIATION over
  r.raw_pixels, which an oversized ROI never populates, so they came back 0.
- TrivialHistogram had a hand-copied OutOfRamPixelCloud overload of initialize()
  whose robust-mean thresholds were bin counts rather than percentile values and
  which never normalized the p10..p90 mean, so ROBUST_MEAN_ABSOLUTE_DEVIATION was
  0 out-of-core. Both sources now share one template, so the two histograms stay
  identical by construction.
- The in-RAM intensity path replaced INTEGRATED_INTENSITY, ENERGY, MODE,
  ROOT_MEAN_SQUARED and the percentiles with the soft-NAN sentinel on a constant
  ROI even though all of them are well defined there, and its out-of-core excess
  kurtosis used KURTOSIS-3 rather than Moments4 (equal on ordinary data, not on a
  constant ROI); hyperskewness/hyperflatness now use the in-RAM sum-of-powers
  definition on both paths.
- The erosion out-of-core path ran its entire erosion chain over disk-backed
  matrices, copying and rescanning one per iteration at roughly 1250x the in-RAM
  time, and lacked the in-RAM driver's constant-ROI skip so it reported the
  sanity cap where the in-RAM path reports 0.
- Zernike's blank-ROI guard resized its coefficient vector instead of assigning
  it; since the out-of-core loop reuses one feature instance across ROIs, a
  constant ROI reported the previous ROI's moments.
- LR::aux_min / aux_max were uninitialized scalars, and the out-of-core scan did
  not recompute them from the pixels it streamed, so a degenerate-ROI guard could
  key off a value carried over from phase 1.
- processIntSegImagePairInMemory took its unprocessed_rois list by value, so the
  in-memory montage path -- which has no out-of-core support -- silently emitted
  an all-zero row for an oversized ROI instead of reporting it.

Adds pytest coverage, split by test kind per tests/vetting/SPEC.md section 2:

- tests/python/test_ooc_invariant.py (9 tests) asserts the required relation
  trivial == out-of-core across every 2D feature group, on an ordinary ROI, on a
  degenerate constant-intensity ROI, and on an image holding one of each. These
  claim no vetting: they compare two Nyxus code paths, not Nyxus against an
  external tool.
- tests/python/test_ooc_mechanics.py (1 test) covers gating only -- the montage
  path has no out-of-core support and must fail loudly rather than emit a
  silently wrong row.

Both run in CI. ram_limit is a process-global and Nyxus refuses a limit above
available RAM while keeping the previous value, so the helper reads the setting
back and asserts it was accepted; otherwise a refused limit silently sends the
in-RAM side out-of-core and the failure surfaces on an unrelated assertion.
@darkclad
darkclad force-pushed the main-io-2d-ooc-fix branch from 14073bd to 2bdc651 Compare August 5, 2026 15:57
@sameeul
sameeul merged commit 9a9ad02 into PolusAI:main Aug 5, 2026
46 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants