fix(stats): surface summary-stat cache write side via telemetry (#951)#952
Merged
Merged
Conversation
The xorq snapshot cache did its I/O with no operator-visible signal. The firstpull.summary_stats span (#943) carried only cache_status/hits/misses/ secs, and _log_cache_stats emitted the write side (snapshots/bytes/ write_errors) at log.info to a server log file no deployment reads — so a cache that stopped writing, or a run with write_errors > 0 (#910), was invisible without counting .parquet files by hand. - xorq_buckaroo: add cache_snapshots/cache_bytes/cache_write_errors to the firstpull.summary_stats span attrs (already available from cache_run_stats()), so a write_errors > 0 run reaches a telemetry consumer. - xorq_stat_pipeline: attach the per-run cache outcome to the stat.xorq.total telemetry span so every process_table run — not just the firstpull — routes its cache outcome through a bound sink, independent of log level. The log.info summary line stays for local perf/debug sessions. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Contributor
📦 TestPyPI package publishedpip install --index-strategy unsafe-best-match --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ buckaroo==0.15.3.dev28542295615or with uv: uv pip install --index-strategy unsafe-best-match --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ buckaroo==0.15.3.dev28542295615MCP server for Claude Codeclaude mcp add buckaroo-table -- uvx --from "buckaroo[mcp]==0.15.3.dev28542295615" --index-strategy unsafe-best-match --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ buckaroo-table📖 Docs preview🎨 Storybook preview |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #951 — the xorq summary-stat snapshot cache did its I/O with no operator-visible signal.
What
The cache read/write itself is correct; this makes the write side observable rather than forensic.
xorq_buckaroo— addcache_snapshots/cache_bytes/cache_write_errorsto thefirstpull.summary_statsspan attrs (already available fromcache_run_stats()), alongside the existingcache_status/cache_hits/cache_misses/cache_secs. Awrite_errors > 0run now reaches a telemetry consumer.xorq_stat_pipeline— capture thestat.xorq.totalspan handle inprocess_tableand have_log_cache_stats(span)attach the per-run cache outcome to it. Everyprocess_tablerun — not just the firstpull — now routes its cache outcome through a bound telemetry sink, independent of log level. Thelog.infosummary line stays for local perf/debug sessions.Why
Verified against 0.15.3: a cold
/load_exprwrites 45 snapshot parquets on the miss and a warm one reads them back (0 rewrites) — correct — but across all 90 cache operations the server's stderr stayed empty and the telemetry carried no write-side signal._log_cache_statsemits atlog.infoto a server log file no deployment reads, so_write_snapshot's failure counter (#910) was invisible in practice: a cache that silently stops writing (a write regression, a permissions problem, an unstable key) could only be caught by counting.parquetfiles by hand. Telemetry is the channel a deployment actually consumes, so the write outcome now rides the span (#943).Tests
tests/unit/test_xorq_stats_v2.py::TestSnapshotCacheRun:test_cache_outcome_rides_total_span_to_telemetry— a cold run with a bound sink emitsstat.xorq.totalcarryingcache_status="miss",cache_snapshots > 0,cache_bytes > 0,cache_write_errors == 0.tests/unit/server/test_load_expr.py:test_load_expr_telemetry_emits_session_correlated_spans— extended to assert the three write-side fields on thefirstpull.summary_statsspan.Both confirmed failing against pre-fix source (KeyError on the new attrs) and passing with the fix. Full relevant suite green (102 passed across the stats, load_expr, perf-log and telemetry-sink files); ruff + paddy-format clean (pre-commit + pre-push).
🤖 Generated with Claude Code