Skip to content

fix: round fixed-point YIQ coefficients to remove systematic under-counting - #185

Merged
dmtrKovalenko merged 2 commits into
dmtrKovalenko:mainfrom
serpent7776:fix-color-delta-coefficient-rounding
Aug 24, 2026
Merged

dmtrKovalenko merged 2 commits into
dmtrKovalenko:mainfrom
serpent7776:fix-color-delta-coefficient-rounding

Conversation

@serpent7776

@serpent7776 serpent7776 commented Aug 15, 2026 •

Copy link
Copy Markdown
Contributor

While comparing against pixelmatch, Claude noticed that fixed-paint delta calculation is biased and could be fixed by rounding. It might affect results of comparisons that are close to the cutoff.

Summary

calculatePixelColorDeltaSimd quantises its twelve YIQ constants with
@intFromFloat(c * 4096), which truncates toward zero. Every constant therefore comes
out low in magnitude, and the errors accumulate rather than cancel — the result is a
systematic ~0.13% underestimate of every colour delta, which turns into silently
missed differences
because the delta is compared against a fixed threshold.

Two commits: the one-line-per-constant fix, and a sweep test that would have caught it.

⚠️ This changes diff counts. odiff now reports slightly more differing pixels
in near-threshold cases. See Behaviour change.

Root cause

The Y row of the YIQ matrix sums to exactly 1 — luminance is normalised so white maps
to Y = 255. Quantising each coefficient independently by truncation destroys that:

row exact × 4096 truncated err rounded err
Y 4096.0000 4094 −2.000 4096 −0.000
I 0.0000 0 +0.000 0 +0.000
Q 0.0000 −1 ... −1 −1.000

Every luminance value comes out 0.0488% low. The metric is quadratic in the YIQ
differences (delta = W_Y·ΔY² + …) so that doubles, and the truncated W_Y adds a
further 0.034%. Predicted worst-case error against the maximum delta of 35215 is ~46;
measured over 20M random pairs, 40.12.

Decomposition over 8M pairs confirms the mechanism — rounding the matrix alone leaves
W_Y's −3.425e−4 as the only systematic term, predicting 12.06 against 12.16 measured:

variant worst error direction
all truncated (current) 38.09 entirely under
matrix rounded only 12.16 mostly under
weights rounded only 24.09 mostly under
all rounded 8.37 two-sided

Impact

Because the bias is one-signed, odiff can miss a difference just above the cutoff
but never invents one — false negatives, the wrong direction for a visual-regression
tool. From test/png/odiff-logo-dark.png against its rotated counterpart:

a = 0xFFFFFFFF (white)   b = 0xFFE4E5E4 (#E4E5E4)
true delta  = 352.6068  >  352.15  → different ✓
fixed delta = 352.1394  <  352.15  → missed   ✗

At the default threshold, white changing to a light grey is not reported.

The fix

@intFromFloat(c * SHIFTED_1) → @intFromFloat(@round(c * SHIFTED_1)) for all twelve
constants. Comptime-only, no runtime cost.

Worst deviation from the f64 calculatePixelColorDelta drops 38.09 → 8.37 and
becomes two-sided. Near the -t 0.1 threshold, where miscounts actually happen, the
worst gap drops 0.72 → 0.19. The fixed-point path now agrees exactly with the f64
reference — and with upstream pixelmatch, whose delta is algebraically identical once
configured with checkerboard: false — on every image pair under test/.

Behaviour change

Counts move by a handful of pixels on images with near-threshold content:

pair before after f64 / pixelmatch
images/donkey.png vs donkey-2.png 101834 101841 101841
logo vs rotated (test expectation) 2270068 2270071 2270071

test_core.zig's expectation moves from 2270068 to 2270071. That old value encoded the
bug: cropping both logo images to their overlapping region and counting with the f64
path gives 148495, against 148492 for the truncated path — exactly the 3-pixel gap.
(Pixels outside the overlap are counted by alpha != 0 alone, so no delta math is
involved and they cannot account for the change.)

The other six fixture pairs under test/ are unchanged at -t 0.1.

Tests

The existing checks compare the two paths over ten hand-picked pixel pairs with bounds
loose enough (max < 0.5%, avg < 0.2%) that the biased quantisation passed them
unnoticed at 0.19%. Added two sweeps over 200k deterministic pairs covering arbitrary,
opaque, near-identical and semi-transparent pixels:

  • drift — bounds absolute and relative error against the f64 reference
  • bias — asserts both directions of rounding error occur

The second is the load-bearing one. A one-signed error is what becomes one-signed
miscounts, and no magnitude bound detects it.

Bounds carry ~2× headroom, calibrated over 2M pairs:

max abs err max rel err overestimates
rounded (this PR) 9.72 0.00056 1786920/2000000
truncated (before) 38.38 0.00197 0/2000000

Reverting the fix fails both, reporting max abs error 37.9687 and 0.0% overestimate, 100.0% underestimate, while every pre-existing colour-delta test still passes — which
is why the bias survived. Seeded PRNG plus integer/f64 arithmetic, so it is reproducible
across platforms.

Verified

  • zig build test-all -Doptimize=ReleaseFast -Dcpu=native — 386/386 steps, 44/45 tests
    pass (1 skipped). Baseline on unmodified main run first for comparison; identical.
  • npm run test -w @odiff/tests — 19/19 pass, now reporting 101841 where it previously
    reported 101834.
  • Not run: the Playwright suite (no browsers available locally). Risk is narrow —
    toHaveScreenshotOdiff defaults to maxDiffPixels ?? 0 and identical pixels have
    delta 0, so only a test currently passing with a nonzero diff under an allowance
    could shift. Worth confirming in CI.

Notes, out of scope

  • --enable-asm will still disagree on main. The AVX-512 kernel uses f32
    constants and is unaffected by this change, but it carries a separate transparent-pixel
    bug: on main, --enable-asm reports 101837 on the donkey pair both before and after
    this PR. That is fixed independently; do not read it as a regression here.
  • src/vxdiff.asm and src/rvv.c hold their own floating-point transcriptions of these
    twelve constants. They need no fix.

Summary by CodeRabbit

  • Bug Fixes

    • Improved SIMD color-difference calculations by reducing rounding-related inaccuracies.
    • Updated rotated-layout comparison results to reflect corrected calculations.
  • Tests

    • Added randomized validation to verify color-difference accuracy and detect systematic rounding bias.

calculatePixelColorDeltaSimd quantises the twelve YIQ constants with
@intFromFloat(c * 4096), which truncates toward zero, so every constant
comes out low in magnitude. The errors do not cancel. The Y row of the
YIQ matrix sums to exactly 1, but its truncated coefficients sum to
4094/4096, making every luminance value 0.049% low; the metric is
quadratic in the YIQ differences so that doubles, and the truncated Y
weight adds a further 0.034%. The result is a systematic ~0.13%
underestimate of every colour delta.

Because the delta is compared against a fixed threshold, that bias turns
into one-directional false negatives: a difference whose true delta sits
just above the cutoff can be dropped, but one is never invented. On
test/png/odiff-logo-dark.png against its rotated counterpart, white
changing to #E4E5E4 has a true delta of 352.607 against a threshold of
352.15, yet the truncated path computes 352.139 and misses the pixel.

Rounding the constants is comptime-only and costs nothing at runtime.
Over 8M random pixel pairs the worst deviation from the f64
calculatePixelColorDelta drops from 38.09 to 8.37 and becomes two-sided;
near the t=0.1 threshold, where miscounts actually happen, the worst gap
drops from 0.72 to 0.19. The fixed-point path now agrees exactly with the
f64 reference on every image pair under test/.

The layout-difference expectation moves from 2270068 to 2270071. Cropping
both logo images to their overlapping region and counting with the f64
path gives 148495 against the truncated path's 148492 -- the old
expectation encoded the bias, and 2270071 is the correct count.

The AVX-512 and RVV kernels are unaffected: vxdiff.asm and rvv.c carry
their own floating-point copies of these constants.
The existing checks compare calculatePixelColorDeltaSimd against the f64
calculatePixelColorDelta over ten hand-picked pixel pairs with bounds
loose enough (max < 0.5%, avg < 0.2%) that a systematically biased
quantisation passes them unnoticed -- truncating the YIQ constants toward
zero sat at 0.19% max and went undetected.

Add two sweeps over 200k deterministic pixel pairs, covering arbitrary,
opaque, near-identical and semi-transparent pixels. The first bounds how
far the fixed-point kernel may drift from the reference in absolute and
relative terms. The second asserts both directions of rounding error
occur, which is the property that actually matters: because the delta is
compared against a fixed threshold, a one-signed error becomes one-signed
miscounts, and odiff silently drops differences sitting just above the
cutoff. Bounding magnitude alone does not catch that.

Bounds are calibrated with roughly 2x headroom against the constants as
of the preceding commit, measured over 2M pairs:

                       max abs err   max rel err        overestimates
  rounded (current)           9.72       0.00056      1786920/2000000
  truncated (previous)       38.38       0.00197            0/2000000

Reverting to truncation fails both sweeps, reporting "max abs error
37.9687" and "0.0% overestimate, 100.0% underestimate" respectively,
while every pre-existing colour delta test still passes -- which is why
the bias survived in the first place.

The sweep is seeded and the arithmetic is integer and f64, so the result
is reproducible across platforms. It exercises calculatePixelColorDeltaSimd
directly rather than the RVV dispatch helper, since the property under
test belongs to the fixed-point quantisation.
@coderabbitai

coderabbitai Bot commented Aug 15, 2026 •

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 514d44a9-6d03-475f-b0dd-c93e944e0ca1

📥 Commits

Reviewing files that changed from the base of the PR and between 6166515 and 32e62d8.

📒 Files selected for processing (3)
  • src/color_delta.zig
  • src/test_color_delta.zig
  • src/test_core.zig

📝 Walkthrough

Walkthrough

The SIMD color-delta implementation now rounds YIQ values before fixed-point conversion. New deterministic randomized tests measure accuracy and quantization bias. A rotated-layout regression expectation is updated.

Changes

Color Delta Quantization

Layer / File(s) Summary
SIMD fixed-point rounding
src/color_delta.zig
YIQ coefficients and channel weights use @round before conversion to i64.
Accuracy and regression validation
src/test_color_delta.zig, src/test_core.zig
Deterministic tests check absolute error, relative error, and overestimate/underestimate shares across 200,000 samples. The rotated-layout comparison expects 2,270,071 differing pixels in both directions.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk: ⚪ Minimal · up to 32e62

This localized change corrects fixed-point color-delta rounding and updates coverage for the intentional near-threshold diff-count change; no actionable merge-blocking risk remains beyond normal checks and review.

Suggested reviewers: dmtrkovalenko

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main fix: rounding fixed-point YIQ coefficients to remove systematic under-counting.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@dmtrKovalenko
dmtrKovalenko merged commit 644c234 into dmtrKovalenko:main Aug 24, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants