feat(row-cache): RowCacheController request lifecycle (7b orchestration, parallel)#945
feat(row-cache): RowCacheController request lifecycle (7b orchestration, parallel)#945paddymul wants to merge 2 commits into
Conversation
Record the decisions resolved for the consumer cutover and plan 7b-7e in detail: - A: getRowsByRowid is in v1 (positional populate can't fetch post-sort rows) - B: every view is an ordered list of original_row_id; sort/filter unify and compose client-side; cache key is the generation (content-changing ops), not the sort - Forward compatibility: auto-cleaning as a future column-delta on the same generation (original_col_id mirrors original_row_id), the two guardrails, and the one v1 simplification (row-granular RowStore) v2 revisits - Rewrote "What's left" into a sequenced 7b-7e plan Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The orchestration layer RowCache (pure state) lacks: fire request, park the AG-Grid callback, replay on response, dedupe in-flight, eager head/tail prefetch. Built net-new and runs PARALLEL to KeyAwareSmartRowCache — no cutover until the rowid system is ready to merge. JS-only. Lifecycle (reactive drive() loop): resolve the active view (fire sort/filter and park if its permutation/subset isn't here — AG-Grid shows its loading overlay because the callback doesn't fire); once built, missingAt the window and either resolve+gc or fetch the missing rowids (by rowid, or positionally via populate for the identity view). Sort is two round-trips. Filter+sort compose client-side by intersection (free — both key off original_row_id). Eager head/tail fires after a sort lands, like the existing second response. 9 lifecycle tests; the test commit was seen failing locally first (the tsc pre-commit gate blocks committing a spec that imports a missing module, so spec + impl land together). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
📦 TestPyPI package publishedpip install --index-strategy unsafe-best-match --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ buckaroo==0.15.2.dev28179768503or with uv: uv pip install --index-strategy unsafe-best-match --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ buckaroo==0.15.2.dev28179768503MCP server for Claude Codeclaude mcp add buckaroo-table -- uvx --from "buckaroo[mcp]==0.15.2.dev28179768503" --index-strategy unsafe-best-match --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ buckaroo-table📖 Docs preview🎨 Storybook preview |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4eecad44b0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| for (const p of this.pending) { | ||
| if (!this.tryResolve(p)) still.push(p); | ||
| } | ||
| this.pending = still; |
There was a problem hiding this comment.
Avoid dropping reentrant row requests
If a success callback synchronously asks for another range (for example, a grid/request wrapper that requests the next block from inside the fulfilled callback), the nested getRows appends to this.pending while the outer drive() is still running; this assignment then replaces this.pending with the outer still snapshot and drops the newly parked request, so its later response will never replay the callback. Consider processing a snapshot or removing resolved entries before invoking callbacks so reentrant requests survive.
Useful? React with 👍 / 👎.
Summary
The orchestration layer that
RowCache(pure state, #719) lacks: fire a request, park the AG-Grid callback, replay it when data lands, dedupe in-flight work, and eagerly prefetch head/tail. Built net-new asRowCacheControllerand runs parallel toKeyAwareSmartRowCache— nothing is cut over; both implementations coexist until the rowid system is ready to merge. JS-only, fully tested against a mockreqFn.This PR also folds the cutover design decisions into
docs/smart-row-cache-redesign.md(the spec this code implements).Decisions recorded in the doc
getRowsByRowid([…])is in v1. After a sort the visible window maps to an arbitrary, non-contiguous rowid set a positionalpopulatecan't fetch;missingAtreturns those rowids andgetRowsByRowidfetches exactly them.original_row_id.SortView/FilterViewunify; filter+sort composes client-side by intersection (free — shared id namespace).RowCache.original_col_idmirrorsoriginal_row_id), the two guardrails, and the one v1 simplification (row-granularRowStore) v2 revisits.Lifecycle (
RowCacheController)Reactive
drive()loop: resolve the active view (firesort/filterand park if its permutation/subset isn't built — AG-Grid shows its loading overlay because the callback is withheld); once built,missingAtthe window and either resolve +gc, or fetch the missing rowids (rowsByRowid, or positionalpopulatefor the identity view) and park. Every response re-drives all parked requests, so partial overlap and composition fall out.sort→rowidOrder→ rows)sort/filterand row fetches are dedupedPublic surface mirrors
KeyAwareSmartRowCache(getRows/addResponse/addError) so the eventual swap is a drop-in.What's NOT in this PR (kept parallel)
getRowCacheDs) wiring AG-GridIDatasource.getRows+model.send/msg:customto the controller.7chandlers + a pandas/polarstag_with_rowids.Map<generationKey, RowCacheController>wrapper (generation key is a follow-on).SmartRowCache— the old path is untouched.Assumptions baked in
rowsresponses are complete (carry every requested rowid); partial-response handling is deferred (flagged in code).original_row_idnamespace) per controller.Test plan
RowCacheControllerlifecycle tests (identity cold/warm, two-step sort, sort dedupe, eager head/tail, scroll-by-rowid under a built sort, filter, filter+sort composition, error path)🤖 Generated with Claude Code