Cache bin-color resolution per trace, not per request

[masenf]

Summary

Resolves a critical performance regression where density_view quantized the entire color column into kernel LUT indices on every reply — an O(N) NumPy pass with multi-GB temporaries that cost the 100M-point FastAPI drilldown demo 1.3–7 s per request, ~10–100× the actual tier work.

The color resolution is now cached on the trace (interaction.trace_bin_colors) and materialized only by branches that feed bin_2d_mean_color. Pyramid replies compose prebuilt color planes and point drills ship sliced channels, so they never trigger the expensive full-column pass. The cache is invalidated on append and lazily re-resolved by the append's own refresh emit.

Key Changes

• interaction.py: Added trace_bin_colors() function that resolves and caches the full-column bin-color source once per trace. Added bin_color_cache_bytes() for memory reporting. Modified _ensure_pyramid() and density_view() to use the cached resolution only when needed (exact re-bins and first-paint emit), not for pyramid or drill replies.

• _trace.py: Added _bin_colors field to cache the resolved color source (None = never resolved, 0 = not applicable, otherwise the resolve_bin_colors kwargs dict). Documented as a rebuildable §27 derived cache.

• _payload.py: Updated _density_trace_spec() to use interaction.trace_bin_colors() instead of resolving inline, sharing the O(N) quantize with pyramid build and later grid replies.

• _figure.py: Updated memory_report() to include bin_color_bytes (itemized cache size) and include it in resident_array_bytes total.

• interaction.py append path: Added cache invalidation (t._bin_colors = None) in the append handler so the next grid reply re-resolves over the post-append column.

• Tests: Added comprehensive test suite in test_density_mean_color.py covering:

  • test_density_view_exact_band_resolves_colors_once: Verifies exact re-bins resolve once and reuse cached result
  • test_density_view_points_band_never_resolves_colors: Confirms drills never trigger resolution
  • test_pyramid_band_reuses_build_time_resolution: Pyramid replies compose prebuilt planes
  • test_append_re_resolves_bin_colors_exactly_once: Append invalidates and re-resolves exactly once
  • test_memory_report_itemizes_bin_color_cache_bytes: Memory accounting

• Benchmarks: Added colored_drilldown_figure fixture and test_adaptive_drilldown_cycle_mean_color regression tripwire that measures the drilldown cycle on a channel-bearing trace (the FastAPI demo shape). Ensures per-request cost stays free of full-column channel work.

• Documentation: Updated LOD architecture spec (§2) with cost note and caching contract. Updated benchmark methodology and results to document the regression guard and measured improvements (0.02–0.45 s per reply on the demo host, 223 ms → 99 ms for the 2.1M benchmark cycle).

Implementation Details

The resolution is a pure function of immutable channel values and their global domain, making it safe to cache for the trace lifetime. The cache is stored as either:

• None: never resolved
• 0: resolved but not applicable (no channel or constant values)
• dict: the resolve_bin_colors kwargs (idx/rgba planes + LUT)

Only the cheap channels.bins_mean_color() fact is decided up front in density_view; the expensive full-column source is materialized solely by branches that feed bin_2d_mean_color (exact re-bins and first-paint emit). This prevents charging pyramid and drill replies for work they never consume.

https://claude.ai/code/session_01NP6yfYGvvkUHu26x8Q2GWb

[greptile-apps]

Greptile Summary

This PR caches bin-color resolution on each trace and avoids redoing full-column color work during density replies. The main changes are:

• Adds trace_bin_colors() and _bin_colors to cache mean-color kernel inputs per trace.
• Uses cached color resolution only for paths that build mean-color grids.
• Invalidates the color cache on append and reports its memory use in memory_report().
• Bounds quantization and mean-color accumulator temporary memory.
• Adds benchmark and test coverage for colored drilldown, append refresh, no-rescan behavior, chunked quantization, and memory accounting.
• Updates LOD and benchmark docs to describe the caching contract.

Confidence Score: 5/5

Safe to merge with low risk.

The changed paths are focused on derived color-cache materialization, memory accounting, and bounded temporary allocations. Tests cover the main behavior changes: exact grids, drills, pyramid reuse, append invalidation, no-rescan retention, chunk equivalence, and memory reports. No verified functional or security issues were found.

No files require special attention.

T-Rex Logs

What T-Rex did

• The density regression proof was executed and its log was generated, capturing the command, working directory, pytest output, warnings, a summary, and EXIT_CODE: 0.
• The benchmark tripwire proof was executed and its log was generated, capturing the command, working directory, benchmark timing table, pytest summary, and EXIT_CODE: 0.

_{Ran code and verified through T-Rex}

Important Files Changed

Filename	Overview
python/xy/interaction.py	Caches full-column bin-color resolution per trace, gates materialization to mean-color grid consumers, and invalidates on append.
python/xy/channels.py	Adds chunk-bounded quantization helpers and wires them through bin-color resolution.
python/xy/_payload.py	Reuses trace-level bin-color resolution when building density trace specs.
python/xy/_figure.py	Includes cached bin-color buffers in memory reporting and resident byte totals.
src/kernels.rs	Adds memory-budget-based worker shedding for mean-color accumulation and coverage for thread selection.
tests/test_density_mean_color.py	Adds tests for once-per-trace resolution, append invalidation, no-rescan retention, chunk equivalence, and memory accounting.
benchmarks/test_codspeed_kernels.py	Adds a colored drilldown benchmark path covering cached color resolution.
spec/design/lod-architecture.md	Documents per-trace bin-color caching, no-rescan retention behavior, and chunked transient bounds.

_{Reviews (7): Last reviewed commit: "Name the aggregate floor on the drilldow..." | Re-trigger Greptile}

[codspeed-hq]

Merging this PR will not alter performance

✅ 102 untouched benchmarks
🆕 1 new benchmark
⏩ 1 skipped benchmark¹

Performance Changes

	Benchmark	BASE	HEAD	Efficiency
🆕	test_adaptive_drilldown_cycle_mean_color	N/A	314.8 ms	N/A

---

_{Comparing claude/fastapi-drilldown-perf-jb3mh6 (0234748) with main (960e132)}

Footnotes

1. 1 benchmark was skipped, so the baseline result was used instead. If it was deleted from the codebase, click here and archive it to remove it from the performance reports. ↩