noun: fix validation, refcounts, and results in lagoon jets - #1057
Open
sigilante wants to merge 1 commit into
Open
noun: fix validation, refcounts, and results in lagoon jets#1057sigilante wants to merge 1 commit into
sigilante wants to merge 1 commit into
Conversation
sigilante
force-pushed
the
sigilante/lagoon-check-fixes
branch
from
July 7, 2026 21:06
8356793 to
7ab226f
Compare
sigilante
added a commit
to sigilante/urbit
that referenced
this pull request
Jul 7, 2026
Three latent bugs in untested arms, found while auditing the lagoon jets against this library (urbit/vere#1057): * +diag built its result with (turn (flop (gulf 0 (dec n))) ...) and so returned the diagonal REVERSED. * +el-wise-op (whose only user is +abs) mapped over (flop (ravel a)), reversing the element order. The comment says the flop is to "compensate for LSB", but there is nothing to compensate: +rep places the first list element at the low end and +ravel yields row-major order, so the round trip already preserves element order (as +bin-op, which has no flop, demonstrates). * +range's bloq-4 (%rh) branch is missing the =. shape.meta ~[(lent bad)] that its 5/6/7 siblings have, so it returned a ray whose shape is whatever the caller passed in .meta rather than the element count. Note on jet skew: the +abs jet in vere has always returned natural (unreversed) order, so jetted and unjetted ships already disagree about +abs today -- this change converges the Hoon with the deployed jet. The +diag jet crashed unconditionally whenever it fired, and the +range jet recomputed the count its own way, so there is no deployed behavior to preserve for those. urbit/vere#1057 fixes the jets to match this change exactly; the two should land together. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Prompted by ~dozreg-toplud's refcount linter, which flagged leaks in _check, plus a hand audit of the surrounding code. The fixes fall into four groups. _check helper: * take [meta data] as two borrowed nouns instead of consing a cell at every call site (the cell leaked, and it silently assumed ownership of the borrowed sample legs); drop the leaked u3qa_dec result * stop punning raw c3_d/c3_w values as nouns: the shape product was truncated through a u3_atom and compared against a noun, so any ray with 2^31 or more elements failed _check spuriously (and a crafted overflowing shape could pass it) * compute the product with 64-bit overflow detection; an overflowed product cannot match the block count of a real atom, mirroring the Hoon exactly * validate the bloq before calling u3r_met, which is UB at or above block size 37 (release builds compile the guard out); at such sizes any nonzero atom is one block * bail %exit only where +check itself crashes (cell dims, improper shape list, zero data underflowing +dec) wrapper/Hoon divergences (jet computed where the Nock crashes, or vice versa, or produced a different noun): * transpose/diag passed the whole core to _check, reading the gate's battery as ray metadata -- those jets unconditionally bailed %exit whenever they fired * the elementwise (+add/+sub/+mul/+div/+mod), comparison (+gth/+gte/+lth/+lte), scalar (+add-scalar &c), +abs, and +trace wrappers never called _check at all, though the Hoon asserts it * +ravel has no +check and never reads the kind, so its wrapper now punts (not bails) on an inconsistent ray * +dot only asserts equal shapes outside the %i754 path; the full meta equality and consistency checks moved inside the %i754 case * +mmul never compared the two rays' bloq/kind, and its gemm switch silently returned a zero matrix for an out-of-range bloq; it now punts on both * reduction results (+cumsum/+min/+max/+dot) hard-coded shapes ~[1] or ~[1 1] or ~[n 1]; +scalar-to-ray gives an all-ones shape of the input's rank * _set_rounding returns c3n on an unrecognized mode and callers punt, instead of bailing %fail kernel bugs (wrong results, previously unobservable because the wrapper bailed or untested): * trace computed (dot d d) -- the sum of SQUARES of the diagonal -- instead of (cumsum (diag a)); identity-matrix tests masked it * transpose read and wrote with rows/columns swapped, correct only for square inputs, and the wrapper did not swap the result shape * argmin/argmax returned (len - i - 1) instead of the ravel index i * diag and abs return elements in natural order, in lockstep with urbit/urbit#7388, which removes the erroneous flops in +diag and +el-wise-op (the old +abs jet already returned natural order, so jetted and unjetted ships disagreed about +abs; the diag jet never returned at all) * trace honors the door rounding mode; linspace fences an indirect count atom before using it as a raw integer * mod (ray and scalar) rounded the quotient with a hardcoded round-toward-zero; +toi rounds in the door mode (7 mod 2 is -1 under %n, not 1). It also returned values where the Hoon crashes: a non-finite quotient (zero or NaN divisor) is (need ~). And mod-scalar/div-scalar multiplied by a rounded 1/n instead of dividing -- wrong even for exact quotients (21 mod 7 gave 7, 21/7 gave 2.9999998). All four now divide directly, round in softfloat_roundingMode, and bail %exit on a non-finite quotient * gth/gte used SoftBLAS's f_gt/f_ge macros, which are (!le)/(!lt) and thus TRUE whenever either operand is NaN; IEEE gt/ge are false. Same disease via f_min/f_max in min/max/argmax. All now use lt/le with swapped operands, which also reproduces the Hoon reel-fold exactly (NaN in the head is sticky, interior NaN is skipped, first-of-ties wins) * range counted elements with one-shot ceil((b-a)/d), but the Hoon iterates x+d in the door mode, testing each sum against b -- for d=.0.1 the shapes disagreed outright (10 vs 11). The kernel now replicates the iteration (values are the accumulated sums) and the wrapper takes the count from the result's block count; it punts on non-finite bounds/step, a zero step, or a stalled accumulator, where the Nock loops forever leaks: * the _check cell at every call site; u3qa_dec inside _check; _get_dims arrays in the diag/dot wrappers and trace kernel (plus raw c3_d dimensions used directly as nouns there); argmin/argmax scan buffers; linspace/range result-shape overkeeps Verified on a fresh fake ship against the base lagoon (with the urbit/urbit#7388 fixes applied), differentially against a de-jetted copy of the same library: 36/36 probes agree, including transpose of non-square, trace, diag, reduction shapes at rank 1, argmin/argmax with NaN at and after the head, mod under %n/%u, mod-scalar and div-scalar on exact quotients, gth/gte/lth against NaN, range with d=.0.1 and a negative step, and add/cumsum/mod-scalar/range under non-default rounding modes; zero-divisor mod crashes on both doors. A 40x40 %i754 mmul runs ~3s jetted vs ~25s interpreted, confirming the jets fire. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
sigilante
force-pushed
the
sigilante/lagoon-check-fixes
branch
from
July 7, 2026 23:32
7ab226f to
ff06e8a
Compare
sigilante
added a commit
to sigilante/urbit
that referenced
this pull request
Jul 8, 2026
Twenty-two oracles for behaviors that have broken before, in this library or in its vere jets: non-square transpose, diag order and trace-sums-not-squares, abs element order, reduction result rank, argmin/argmax indices and NaN stickiness, NaN comparisons, non-square mmul, %mod door-mode rounding and zero-divisor crash, exact scalar mod/div quotients, range accumulation count/values (ship-verified under the %z bare-la default) and its bloq-4 result shape, and linspace n=1. They pass identically interpreted and jetted; a jet that disagrees with any of them is wrong. Verified green on a fake ship running the urbit/vere#1057 jets over this branch's library. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
sigilante
added a commit
to urbit/numerics
that referenced
this pull request
Jul 8, 2026
Ports the full urbit/vere#1057 fix set (validation, refcounts, NaN comparisons, kernel results -- see that PR) into the %int2-extended 32-bit copy and the stateless-SoftBLAS 64-bit copy, preserving the int2/twoc code and the _la_rnd plumbing. One deliberate divergence from the reference: %mod here follows THIS desk's C-fmod semantics (quotient truncated %z regardless of door mode; canonical quiet NaN, not a crash, on a zero divisor or non-finite quotient), per the desk's own "Vere follow-on" note. The int2 dot case also gains the meta-sing and consistency checks +bin-op asserts. tests/lib/lagoon-jet-parity encodes 22 regression oracles for everything that broke (transpose/diag/trace/abs order and values, reduction ranks, argmin/argmax NaN behavior, NaN comparisons, exact scalar mod/div, desk-fmod %mod, range accumulation and bloq-4 shape, linspace n=1) -- green interpreted against this desk; running it on a jetted numerics binary is the port's acceptance test. A stock- semantics twin lives upstream in urbit/urbit#7388 tests/lib/lagoon. Also corrects the +la doccords: bare la rounds %z (the bunt of $rounding-mode), not %n. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This was referenced Jul 8, 2026
Merged
sigilante
added a commit
to urbit/numerics
that referenced
this pull request
Jul 12, 2026
lagoon: jet parity with urbit/vere#1057 — Hoon fixes, C ports, regression suite
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Prompted by ~dozreg-toplud's refcount linter, which flagged leaked nouns in the lagoon jets'
_check, plus a hand audit of the surrounding code against the Hoon library the jets serve. The audit turned up considerably more than the linter findings: broken validation, jet/Nock semantic divergences, wrong-result kernel bugs, and a family of leaks. Everything lands inpkg/noun/jets/i/lagoon.c._checkhelperThe old
_checkwas wrong in four independent ways:u3qa_dec's result was never freed (the linter's finding), andu3r_met's rawc3_wreturn was type-punned throughu3_atom.c3_d, truncated into au3_atom, and compared word-for-word against a noun — so any well-typed ray with ≥ 2³¹ elements spuriously failed_check(jet crashes where the Nock succeeds), and a crafted shape whose product wraps could pass it.u3r_metunvalidated;u3r_metrequiresa_y < 37and only enforces it withc3_dessert, which release builds compile out — undefined behavior on a hostile bloq.The rewrite takes
meta/dataas two borrowed nouns (no cell is consed, nothing is allocated), computes the product with overflow detection (an overflowed product cannot equal the block count of any real atom, which mirrors the Hoon), handles bloqs ≥ 37 the way+metdoes, and bails%exitonly where+checkitself crashes (cell dims, improper shape list, zero data underflowing+dec).Every call site previously passed
_check(u3nc(x_meta, x_data)).u3nctransfers ownership, but the metas/datas are borrowed from the sample, and the cell was never freed — one leaked cell per jet invocation, with the ownership violation masked by the leak (naively freeing the cell would have corrupted the sample's refcounts).Wrapper ↔ Hoon divergences
_check(_check(cor)), which then read the gate's battery as ray metadata — these two jets unconditionally bailed%exitwhenever they fired. This had gone unnoticed because there are no transpose tests in the desks that exercise the jets.+add/+sub/+mul/+div/+mod), comparison (+gth/+gte/+lth/+lte), scalar (+add-scalar&c),+abs, and+tracewrappers never called_checkat all, though the Hoon asserts it — the jet computed a result where the Nock deterministically crashes.+ravelhas no+checkand never reads the kind, so its wrapper now punts (not bails) on an inconsistent ray.+dotonly asserts equal shapes outside the%i754path; the full meta-equality and consistency assertions moved inside the%i754case so other kinds punt to the Nock instead of bailing.+mmulnever compared the two rays' bloq/kind (the Hoon computes elementwise on each ray's own meta; the kernel assumes one width), and its gemm switch had no default — an out-of-range bloq silently returned a zero matrix. Both cases now punt.+cumsum/+min/+max/+dot) hard-coded shapes~[1],~[1 1], or~[n 1];+scalar-to-rayactually returns an all-ones shape of the input's rank. The dot wrapper's~[n 1]produced a shape/data-inconsistent ray outright._set_roundingnow reports an unrecognized rounding mode so callers punt, instead ofu3m_bail(c3__fail).Kernel bugs (wrong results)
All previously unobservable because the wrapper bailed before reaching them, or untested:
(dot d d)— the sum of squares of the diagonal — instead of(cumsum (diag a)). Identity-matrix tests mask this (1² = 1).len - i - 1instead of the ravel indexi.+diagand+el-wise-op(the "compensate for LSB" comment there is mistaken —+repputs the first list element lowest, so the flop reverses the elements). Skew note: the old abs jet already returned natural order, so jetted and unjetted ships disagree about+abstoday — the Hoon fix converges with deployed jet behavior; the old diag jet never returned at all. The two PRs should land together.softfloat_roundingModewas left over); linspace fences an indirect count atom before using it as a raw integer in allocation sizes.+toirounds in the door mode —7 mod 2is-1under%n, not1. It also returned values where the Hoon crashes: a non-finite quotient (zero or NaN divisor) is(need ~). And mod-scalar/div-scalar multiplied by a rounded1/ninstead of dividing — wrong even for exact quotients (21 mod 7gave7;21/7gave2.9999998). All four kernels now divide directly, round viasoftfloat_roundingMode, and bail%exiton a non-finite quotient.f_gt/f_gemacros, which are(!le)/(!lt)— true whenever either operand is NaN, where IEEE gt/ge are false. The same disease reaches min/max/argmax throughf_min/f_max. All now uselt/lewith swapped operands, which also reproduces the Hoon reel-fold exactly (NaN at the head is sticky, interior NaN is skipped, first-of-ties wins).ceil((b-a)/d), but the Hoon iteratesx+din the door mode testing each sum againstb— ford = .0.1the two shapes disagreed outright (10 vs 11 elements). The kernel now replicates the iteration (the values are the accumulated sums), the wrapper takes the count from the result's block count, and it punts on non-finite bounds/step, a zero step, or a stalled accumulator (x+d == xshort ofb), where the Nock loops forever.Also fixed leaks of
_get_dimsarrays in the diag/dot wrappers and trace kernel (with rawc3_ddimensions used directly as nouns — invalid for dims ≥ 2³¹), the argmin/argmax scan buffers, and over-kept result shape cells in linspace/range.Verification
Fresh fake ship (
-F bus,urbit-408kpill) on this branch, against the baselib/lagoon.hoonwith the urbit/urbit#7388 flop fixes applied (its%add-rays&c hints match the tree registration), differentially compared with a de-jetted copy of the same library (parent~% %nonhint renamed so nothing registers):%n/%u, mod-scalar and div-scalar on exact quotients, gth/gte/lth against NaN, abs on negative entries, ravel, linspace, range withd = .0.1and with a negative step, and add/cumsum/mod-scalar/range under non-default rounding modes via+lake.modcrashes on both doors (jet bails%exitwhere the Nock's(need ~)crashes).%i754mmul+trace runs in ≤3 s jetted vs ~25 s interpreted, with bit-identical results.Notes for reviewers:
u3r_metreturnsc3_w, so+checkon atoms of ≥ 2³² blocks is still constrained upstream of this file; and the SoftBLAS gemm readsBwithldcinstead ofldb(harmless here since the jet passesldb == ldc, but worth fixing in SoftBLAS). SoftBLAS'sf_gt/f_ge/f_min/f_maxmacros are NaN-incorrect for the same!lereason and deserve an upstream fix there too; this PR routes around them locally.🤖 Generated with Claude Code