Point-window cache (T13) + resolvability gate (#225)#227
Draft
masenf wants to merge 2 commits into
Draft
Conversation
…dows Fixes #225: zooming out from a drilled window brought fixed-rate "sampled points" back over the density surface — individual marks at a zoom where real points are sub-pixel, misrepresenting the dataset. Now: - Interactive density_view replies ship no point sample at all. The mean-color density surface (LOD doc §2) stands alone above the drill budget; real points still arrive the moment a window fits it. The retained first-payload sample (the standalone re-bin worker's CPU source) draws only below a resolvable-count gate: estimated in-view count <= LOD_DIRECT_POINT_BUDGET (lodOverlayResolvable, T9). Datasets under the budget keep the hybrid look everywhere. Point caching (LOD doc T13) so pan/zoom stops re-shipping the same points: - Kernel: a points-tier reply ships the largest ALIGNED window around the view whose exact count fits the budget (lod.aligned_window: bounds snap outward to a power-of-two grid over the trace's extent, per dimension; DRILL_PAD_TARGETS ladder, pyramid-count-gated, exact-verified, DRILL_PAD_SPAN_CAP guards the §16 re-encode ladder). Consecutive pans resolve to the SAME window; lod_blend stays keyed on the view's own count. Nonlinear axes keep the exact view window. - Client: a reply for a new window retires the previous exact drill into a bounded per-trace cache (lodRetireDrill/LOD_POINT_CACHE_WINDOWS) instead of overwriting its buffers — so does a drill dying outside its window — and any later view covered by a cached window promotes it back alpha-continuously with zero round-trips (lodPromoteCachedDrill). Outgrown windows free geometry-only (T11 rule, §27). - Kernel keeps a bounded drill-subset history (Trace.drill_history, DRILL_HISTORY_KEEP) so picks against a promoted retired window still translate exactly; expired seqs drop the pick, data changes clear it. - Client suppresses identical density_view requests (same window, screen, unchanged data) and accepts the suppressed duplicate's in-flight reply per-trace instead of killing it with the bumped seq — ending the re-request loop at gesture end. Spec: lod-architecture T9 rewritten + new T13, wire-protocol density_view/points/pick contracts, dossier constants table, CHANGELOG, README representation labels, docs ladder table. Tests: new test_drill_window_padding (alignment, budget, span cap, nonlinear skip, history clear), test_density_sample_resolvability_gate and test_drill_point_window_cache (headless-Chromium client probes), elision test extended for duplicate suppression; drill tests updated to the padded-window contract.
masenf
marked this pull request as draft
July 23, 2026 12:50
Greptile SummaryThis PR improves dense scatter rendering with reusable point windows and resolution-aware overlays. The main changes are:
Confidence Score: 4/5The in-place tier update path can leave density rendering stale and needs a fix before merging. Answered request memos do not expire. A tier update can change trace data without replacing the GPU record or clearing its memo. A matching view can therefore suppress the density request needed for the new data. js/src/54_kernel.ts
 T-Rex LogsWhat T-Rex did
Important Files Changed
Reviews (2): Last reviewed commit: "Bound density wire traffic: source-clamp..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Comment on lines
+160
to
+169
| // duplicate-suppressed requests leave their traces waiting on the ORIGINAL | ||
| // request's seq (T13 — identical window and screen). Accept it only when | ||
| // every trace it updates is still waiting on exactly this seq AND that seq | ||
| // names the trace's last actually-SENT request — a pending marker can | ||
| // outlive a debounce-cancelled send, and a reply matching such a phantom | ||
| // request is stale, not suppressed-current (T5). | ||
| _densityReplyCurrent(msg) { | ||
| const ids = (msg.traces || []).map((u) => Number(u.id)); | ||
| if (!ids.length && msg.trace !== undefined) ids.push(Number(msg.trace)); | ||
| if (!ids.length) return false; |
There was a problem hiding this comment.
When a batched request for traces A and B is followed by a view that A can serve from its point cache but B cannot, only B keeps waiting for the original sequence. _densityReplyCurrent then returns false because A no longer has that pending sequence, so the handler discards the whole response, including B's current update. B remains stale until the request times out and is sent again; reply currentness must be applied per trace rather than requiring every trace in the batch to remain pending.
Comment on lines
+152
to
+157
| _densityRequestDup(g, view, plotW, plotH, now) { | ||
| const last = g._lastDensityReq; | ||
| if (!last || last.key !== this._densityRequestKey(g, view, plotW, plotH)) return null; | ||
| if (last.answered) return last; | ||
| return now - last.sentAt < 1200 ? last : null; | ||
| }, |
There was a problem hiding this comment.
The duplicate check treats an answered key as permanently current for the lifetime of the GPU trace record, but trace resource rebuilds do not clear _lastDensityReq. If new data is applied while the record is retained, returning to the same window and screen size matches the old answered key and suppresses the density request, leaving the view rendered from stale data.
Artifacts
Repro: focused executable lifecycle harness
- Evidence file captured while the check ran.
Repro: runtime output showing retained answered memo, suppressed second request, and stale response
- The full command output behind this check.
…yered draw Field HAR from the 100M live drilldown (PR #227 review): every pan/zoom step at 200-450% zoom re-requested a ~2.7 MB full-screen density grid — including back-to-back windows differing by sub-pixel amounts — and the frame dropped to the blurriest cached texture whenever a pan left the freshest window. Three fixes, all recorded under LOD doc T13: - Kernel: pyramid-served grids are clamped to the source cells the finest level resolves under the window (interaction._pyramid_source_shape, ceil(base*frac)+1 per axis) — composing to full screen resolution just upsamples blocky cells the client's texture filtering reproduces from the source-resolution grid at a fraction of the bytes. The attainable per-axis cell size rides the reply as density.min_cell; exact and spatial grids (true full-detail bins) omit it. The HAR's 2.7 MB replies drop to ~0.5-1.4 MB in its zoom range and ~200 KB deeper. - Client request elision (lodDensityCacheServes): a cached texture that contains the view and is as detailed as anything the kernel could return — one texel per screen pixel, or already at min_cell (zooming into a source-limited texture cannot sharpen it) — answers with no round-trip. Guarded so the kernel stays in charge where it can do better: estimated in-view count must exceed budget x LOD_DENSITY_ELIDE_EST_FACTOR, and the cached window may be at most LOD_DENSITY_ELIDE_MAX_AREA_RATIO x the view area (uniform-density estimates overshoot in sparse corners). Entries without recorded counts never elide; lodRememberDensity now supersedes an unpinned same-window twin so elision always reads current facts. - Sub-texel request dedup: the identical-request memo compares windows numerically within half an output texel per edge (gesture-end and settle emit sub-pixel twins; a grid shifted below half a texel is the same picture) instead of exact string equality. - Finer-detail layering (lodDensityDetailForView): when no fine cached window contains the view, the smallest cached texture overlapping >= 5% of it draws on top of the broad backdrop after every crossfade layer, so the already-fetched region stays sharp while the request for the rest is in flight — a resolution seam beats uniform blur. Spec: lod-architecture T13 extended, wire-protocol density_view/density contracts (min_cell, clamped grids, elision), dossier constants table, CHANGELOG. Tests: test_density_wire_economy (source-shape math, min_cell presence/ absence, 20M-point clamp integration), test_density_request_economy (headless-Chromium probe: source-resolution elision + guards, sub-texel dedup, fine-over-broad layering).
| if (!last || !this._densityRequestSame(last, this._densityRequestWindow(g, view), plotW, plotH)) { | ||
| return null; | ||
| } | ||
| if (last.answered) return last; |
There was a problem hiding this comment.
After a request for window W is marked answered, an in-place tier_update can replace the trace data without clearing _lastDensityReq. Returning to W then takes this unbounded answered branch and suppresses the new density request, so the chart can keep rendering a texture built from the old data.
Artifacts
Repro: focused headless-browser pytest harness exercising the real client runtime
- Evidence file captured while the check ran.
- The full command output behind this check.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements two interconnected LOD improvements: a per-trace LRU cache of retired exact point windows (T13) that eliminates re-requests on pan/zoom sequences, and a resolvability gate (#225) that prevents sample overlays from drawing above the direct point budget where they misrepresent the dataset.
Key Changes
Point-Window Cache (T13)
LOD_POINT_CACHE_WINDOWS = 3constant to bound retired window memorylodRetireDrill(),lodPromoteCachedDrill(), and supporting functions to manage the per-trace LRU cache of exact point windowslodWindowServesView()as a shared containment check for both live drill and cached windowsResolvability Gate (#225)
lodOverlayResolvable()function that gates sample overlay drawing on estimated in-view countoverlay.sample.visible * (viewArea / windowArea) <= LOD_DIRECT_POINT_BUDGETdensity_viewreplies no longer ship point samples at all (only the first-payload sample is retained for the standalone re-bin worker)lodSampleForView()to apply the gate before considering overlay candidatesPadded Aligned Windows (T13 support)
aligned_window()and_padded_drill_window()to compute the widest aligned window that fits the budgetDRILL_PAD_SPAN_CAP) to keep deep zooms inside the §16 re-encode ladderDrill Subset History
drill_historydict toTraceto remember recent shipped subsets (bounded byDRILL_HISTORY_KEEP)drill_history()returns the shipped subset for a givendrill_seq, or None when expired (drops the pick)clear_drill_history()forgets subsets after data changesIdentical-Request Suppression (T13)
_densityRequestDup()to detect when the same window at the same screen size is already in flight or answeredClient-Side Changes
lodClearDrillState()consolidates drill lifecycle reset (used by drop, retire, promote)lodFreeDrillBuffers()frees all GPU buffers for a drill objectlodSameWindow()compares windows with floating-point toleranceNotable Implementation Details
(span * 1e-9 + 1e-300)to handle floating-point slopvisiblecount on the sample metadata; overlays without recorded counts (hand-built/legacy specs) keep drawinghttps://claude.ai/code/session_01XQT3v7NdL4dmC6S3g8HjJt