Repository navigation
fix: round fixed-point YIQ coefficients to remove systematic under-counting - #185
Conversation
calculatePixelColorDeltaSimd quantises the twelve YIQ constants with @intFromFloat(c * 4096), which truncates toward zero, so every constant comes out low in magnitude. The errors do not cancel. The Y row of the YIQ matrix sums to exactly 1, but its truncated coefficients sum to 4094/4096, making every luminance value 0.049% low; the metric is quadratic in the YIQ differences so that doubles, and the truncated Y weight adds a further 0.034%. The result is a systematic ~0.13% underestimate of every colour delta. Because the delta is compared against a fixed threshold, that bias turns into one-directional false negatives: a difference whose true delta sits just above the cutoff can be dropped, but one is never invented. On test/png/odiff-logo-dark.png against its rotated counterpart, white changing to #E4E5E4 has a true delta of 352.607 against a threshold of 352.15, yet the truncated path computes 352.139 and misses the pixel. Rounding the constants is comptime-only and costs nothing at runtime. Over 8M random pixel pairs the worst deviation from the f64 calculatePixelColorDelta drops from 38.09 to 8.37 and becomes two-sided; near the t=0.1 threshold, where miscounts actually happen, the worst gap drops from 0.72 to 0.19. The fixed-point path now agrees exactly with the f64 reference on every image pair under test/. The layout-difference expectation moves from 2270068 to 2270071. Cropping both logo images to their overlapping region and counting with the f64 path gives 148495 against the truncated path's 148492 -- the old expectation encoded the bias, and 2270071 is the correct count. The AVX-512 and RVV kernels are unaffected: vxdiff.asm and rvv.c carry their own floating-point copies of these constants.
The existing checks compare calculatePixelColorDeltaSimd against the f64
calculatePixelColorDelta over ten hand-picked pixel pairs with bounds
loose enough (max < 0.5%, avg < 0.2%) that a systematically biased
quantisation passes them unnoticed -- truncating the YIQ constants toward
zero sat at 0.19% max and went undetected.
Add two sweeps over 200k deterministic pixel pairs, covering arbitrary,
opaque, near-identical and semi-transparent pixels. The first bounds how
far the fixed-point kernel may drift from the reference in absolute and
relative terms. The second asserts both directions of rounding error
occur, which is the property that actually matters: because the delta is
compared against a fixed threshold, a one-signed error becomes one-signed
miscounts, and odiff silently drops differences sitting just above the
cutoff. Bounding magnitude alone does not catch that.
Bounds are calibrated with roughly 2x headroom against the constants as
of the preceding commit, measured over 2M pairs:
max abs err max rel err overestimates
rounded (current) 9.72 0.00056 1786920/2000000
truncated (previous) 38.38 0.00197 0/2000000
Reverting to truncation fails both sweeps, reporting "max abs error
37.9687" and "0.0% overestimate, 100.0% underestimate" respectively,
while every pre-existing colour delta test still passes -- which is why
the bias survived in the first place.
The sweep is seeded and the arithmetic is integer and f64, so the result
is reproducible across platforms. It exercises calculatePixelColorDeltaSimd
directly rather than the RVV dispatch helper, since the property under
test belongs to the fixed-point quantisation.
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (3)
📝 WalkthroughWalkthroughThe SIMD color-delta implementation now rounds YIQ values before fixed-point conversion. New deterministic randomized tests measure accuracy and quantization bias. A rotated-layout regression expectation is updated. ChangesColor Delta Quantization
Estimated code review effort: 3 (Moderate) | ~20 minutes Merge Risk: ⚪ Minimal · up to This localized change corrects fixed-point color-delta rounding and updates coverage for the intentional near-threshold diff-count change; no actionable merge-blocking risk remains beyond normal checks and review. Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
While comparing against
pixelmatch, Claude noticed that fixed-paint delta calculation is biased and could be fixed by rounding. It might affect results of comparisons that are close to the cutoff.Summary
calculatePixelColorDeltaSimdquantises its twelve YIQ constants with@intFromFloat(c * 4096), which truncates toward zero. Every constant therefore comesout low in magnitude, and the errors accumulate rather than cancel — the result is a
systematic ~0.13% underestimate of every colour delta, which turns into silently
missed differences because the delta is compared against a fixed threshold.
Two commits: the one-line-per-constant fix, and a sweep test that would have caught it.
Root cause
The Y row of the YIQ matrix sums to exactly 1 — luminance is normalised so white maps
to Y = 255. Quantising each coefficient independently by truncation destroys that:
Every luminance value comes out 0.0488% low. The metric is quadratic in the YIQ
differences (
delta = W_Y·ΔY² + …) so that doubles, and the truncatedW_Yadds afurther 0.034%. Predicted worst-case error against the maximum delta of 35215 is ~46;
measured over 20M random pairs, 40.12.
Decomposition over 8M pairs confirms the mechanism — rounding the matrix alone leaves
W_Y's −3.425e−4 as the only systematic term, predicting 12.06 against 12.16 measured:Impact
Because the bias is one-signed, odiff can miss a difference just above the cutoff
but never invents one — false negatives, the wrong direction for a visual-regression
tool. From
test/png/odiff-logo-dark.pngagainst its rotated counterpart:At the default threshold, white changing to a light grey is not reported.
The fix
@intFromFloat(c * SHIFTED_1)→@intFromFloat(@round(c * SHIFTED_1))for all twelveconstants. Comptime-only, no runtime cost.
Worst deviation from the f64
calculatePixelColorDeltadrops 38.09 → 8.37 andbecomes two-sided. Near the
-t 0.1threshold, where miscounts actually happen, theworst gap drops 0.72 → 0.19. The fixed-point path now agrees exactly with the f64
reference — and with upstream
pixelmatch, whose delta is algebraically identical onceconfigured with
checkerboard: false— on every image pair undertest/.Behaviour change
Counts move by a handful of pixels on images with near-threshold content:
images/donkey.pngvsdonkey-2.pngtest_core.zig's expectation moves from 2270068 to 2270071. That old value encoded thebug: cropping both logo images to their overlapping region and counting with the f64
path gives 148495, against 148492 for the truncated path — exactly the 3-pixel gap.
(Pixels outside the overlap are counted by
alpha != 0alone, so no delta math isinvolved and they cannot account for the change.)
The other six fixture pairs under
test/are unchanged at-t 0.1.Tests
The existing checks compare the two paths over ten hand-picked pixel pairs with bounds
loose enough (max < 0.5%, avg < 0.2%) that the biased quantisation passed them
unnoticed at 0.19%. Added two sweeps over 200k deterministic pairs covering arbitrary,
opaque, near-identical and semi-transparent pixels:
The second is the load-bearing one. A one-signed error is what becomes one-signed
miscounts, and no magnitude bound detects it.
Bounds carry ~2× headroom, calibrated over 2M pairs:
Reverting the fix fails both, reporting
max abs error 37.9687and0.0% overestimate, 100.0% underestimate, while every pre-existing colour-delta test still passes — whichis why the bias survived. Seeded PRNG plus integer/f64 arithmetic, so it is reproducible
across platforms.
Verified
zig build test-all -Doptimize=ReleaseFast -Dcpu=native— 386/386 steps, 44/45 testspass (1 skipped). Baseline on unmodified
mainrun first for comparison; identical.npm run test -w @odiff/tests— 19/19 pass, now reporting 101841 where it previouslyreported 101834.
toHaveScreenshotOdiffdefaults tomaxDiffPixels ?? 0and identical pixels havedelta 0, so only a test currently passing with a nonzero diff under an allowance
could shift. Worth confirming in CI.
Notes, out of scope
--enable-asmwill still disagree onmain. The AVX-512 kernel uses f32constants and is unaffected by this change, but it carries a separate transparent-pixel
bug: on
main,--enable-asmreports 101837 on the donkey pair both before and afterthis PR. That is fixed independently; do not read it as a regression here.
src/vxdiff.asmandsrc/rvv.chold their own floating-point transcriptions of thesetwelve constants. They need no fix.
Summary by CodeRabbit
Bug Fixes
Tests