Skip to content

fix(stats): surface summary-stat cache write side via telemetry (#951)#952

Merged
paddymul merged 1 commit into
mainfrom
fix/951-cache-stats-telemetry-visibility
Jul 1, 2026
Merged

fix(stats): surface summary-stat cache write side via telemetry (#951)#952
paddymul merged 1 commit into
mainfrom
fix/951-cache-stats-telemetry-visibility

Conversation

@paddymul

@paddymul paddymul commented Jul 1, 2026

Copy link
Copy Markdown
Collaborator

Fixes #951 — the xorq summary-stat snapshot cache did its I/O with no operator-visible signal.

What

The cache read/write itself is correct; this makes the write side observable rather than forensic.

  • xorq_buckaroo — add cache_snapshots / cache_bytes / cache_write_errors to the firstpull.summary_stats span attrs (already available from cache_run_stats()), alongside the existing cache_status / cache_hits / cache_misses / cache_secs. A write_errors > 0 run now reaches a telemetry consumer.
  • xorq_stat_pipeline — capture the stat.xorq.total span handle in process_table and have _log_cache_stats(span) attach the per-run cache outcome to it. Every process_table run — not just the firstpull — now routes its cache outcome through a bound telemetry sink, independent of log level. The log.info summary line stays for local perf/debug sessions.

Why

Verified against 0.15.3: a cold /load_expr writes 45 snapshot parquets on the miss and a warm one reads them back (0 rewrites) — correct — but across all 90 cache operations the server's stderr stayed empty and the telemetry carried no write-side signal. _log_cache_stats emits at log.info to a server log file no deployment reads, so _write_snapshot's failure counter (#910) was invisible in practice: a cache that silently stops writing (a write regression, a permissions problem, an unstable key) could only be caught by counting .parquet files by hand. Telemetry is the channel a deployment actually consumes, so the write outcome now rides the span (#943).

Tests

tests/unit/test_xorq_stats_v2.py::TestSnapshotCacheRun:

  • test_cache_outcome_rides_total_span_to_telemetry — a cold run with a bound sink emits stat.xorq.total carrying cache_status="miss", cache_snapshots > 0, cache_bytes > 0, cache_write_errors == 0.

tests/unit/server/test_load_expr.py:

  • test_load_expr_telemetry_emits_session_correlated_spans — extended to assert the three write-side fields on the firstpull.summary_stats span.

Both confirmed failing against pre-fix source (KeyError on the new attrs) and passing with the fix. Full relevant suite green (102 passed across the stats, load_expr, perf-log and telemetry-sink files); ruff + paddy-format clean (pre-commit + pre-push).

🤖 Generated with Claude Code

The xorq snapshot cache did its I/O with no operator-visible signal. The
firstpull.summary_stats span (#943) carried only cache_status/hits/misses/
secs, and _log_cache_stats emitted the write side (snapshots/bytes/
write_errors) at log.info to a server log file no deployment reads — so a
cache that stopped writing, or a run with write_errors > 0 (#910), was
invisible without counting .parquet files by hand.

- xorq_buckaroo: add cache_snapshots/cache_bytes/cache_write_errors to the
  firstpull.summary_stats span attrs (already available from
  cache_run_stats()), so a write_errors > 0 run reaches a telemetry consumer.
- xorq_stat_pipeline: attach the per-run cache outcome to the stat.xorq.total
  telemetry span so every process_table run — not just the firstpull — routes
  its cache outcome through a bound sink, independent of log level. The
  log.info summary line stays for local perf/debug sessions.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

📦 TestPyPI package published

pip install --index-strategy unsafe-best-match --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ buckaroo==0.15.3.dev28542295615

or with uv:

uv pip install --index-strategy unsafe-best-match --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ buckaroo==0.15.3.dev28542295615

MCP server for Claude Code

claude mcp add buckaroo-table -- uvx --from "buckaroo[mcp]==0.15.3.dev28542295615" --index-strategy unsafe-best-match --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ buckaroo-table

📖 Docs preview

🎨 Storybook preview

@paddymul paddymul closed this Jul 1, 2026
@paddymul paddymul reopened this Jul 1, 2026
@paddymul paddymul enabled auto-merge July 1, 2026 19:25
@paddymul paddymul deployed to testpypi July 1, 2026 19:27 — with GitHub Actions Active
@paddymul paddymul added this pull request to the merge queue Jul 1, 2026
Merged via the queue into main with commit 3313534 Jul 1, 2026
52 of 53 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant