Skip to content

feat(row-cache): RowCacheController request lifecycle (7b orchestration, parallel)#945

Open
paddymul wants to merge 2 commits into
mainfrom
feat/row-cache-controller
Open

feat(row-cache): RowCacheController request lifecycle (7b orchestration, parallel)#945
paddymul wants to merge 2 commits into
mainfrom
feat/row-cache-controller

Conversation

@paddymul

Copy link
Copy Markdown
Collaborator

Summary

The orchestration layer that RowCache (pure state, #719) lacks: fire a request, park the AG-Grid callback, replay it when data lands, dedupe in-flight work, and eagerly prefetch head/tail. Built net-new as RowCacheController and runs parallel to KeyAwareSmartRowCache — nothing is cut over; both implementations coexist until the rowid system is ready to merge. JS-only, fully tested against a mock reqFn.

This PR also folds the cutover design decisions into docs/smart-row-cache-redesign.md (the spec this code implements).

Decisions recorded in the doc

  • A — getRowsByRowid([…]) is in v1. After a sort the visible window maps to an arbitrary, non-contiguous rowid set a positional populate can't fetch; missingAt returns those rowids and getRowsByRowid fetches exactly them.
  • B — every view is an ordered list of original_row_id. SortView/FilterView unify; filter+sort composes client-side by intersection (free — shared id namespace).
  • The cache key is the generation, not the sort — only row-content-changing ops key the cache; sort/filter become views inside one RowCache.
  • Forward-compatibility note — auto-cleaning as a future column-delta on the same generation (original_col_id mirrors original_row_id), the two guardrails, and the one v1 simplification (row-granular RowStore) v2 revisits.

Lifecycle (RowCacheController)

Reactive drive() loop: resolve the active view (fire sort/filter and park if its permutation/subset isn't built — AG-Grid shows its loading overlay because the callback is withheld); once built, missingAt the window and either resolve + gc, or fetch the missing rowids (rowsByRowid, or positional populate for the identity view) and park. Every response re-drives all parked requests, so partial overlap and composition fall out.

  • loading overlay during the gap = callback withheld until rows present
  • sort = two round-trips (sortrowidOrder → rows)
  • in-flight sort/filter and row fetches are deduped
  • filter + sort compose client-side by intersection
  • eager head/tail fires after a sort lands (the existing "second request" pattern)

Public surface mirrors KeyAwareSmartRowCache (getRows / addResponse / addError) so the eventual swap is a drop-in.

What's NOT in this PR (kept parallel)

  • The JS datasource adapter (getRowCacheDs) wiring AG-Grid IDatasource.getRows + model.send/msg:custom to the controller.
  • Python 7c handlers + a pandas/polars tag_with_rowids.
  • The Map<generationKey, RowCacheController> wrapper (generation key is a follow-on).
  • Any cutover or deletion of SmartRowCache — the old path is untouched.

Assumptions baked in

  • rows responses are complete (carry every requested rowid); partial-response handling is deferred (flagged in code).
  • One generation (one original_row_id namespace) per controller.

Test plan

  • CI: JS suite green — 149 tests incl. 9 new RowCacheController lifecycle tests (identity cold/warm, two-step sort, sort dedupe, eager head/tail, scroll-by-rowid under a built sort, filter, filter+sort composition, error path)
  • CI: lint green — no consumers touched, old path untouched, so no regression risk to existing widget flows

🤖 Generated with Claude Code

paddymul and others added 2 commits June 25, 2026 10:27
Record the decisions resolved for the consumer cutover and plan 7b-7e in
detail:
- A: getRowsByRowid is in v1 (positional populate can't fetch post-sort rows)
- B: every view is an ordered list of original_row_id; sort/filter unify and
  compose client-side; cache key is the generation (content-changing ops),
  not the sort
- Forward compatibility: auto-cleaning as a future column-delta on the same
  generation (original_col_id mirrors original_row_id), the two guardrails,
  and the one v1 simplification (row-granular RowStore) v2 revisits
- Rewrote "What's left" into a sequenced 7b-7e plan

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The orchestration layer RowCache (pure state) lacks: fire request, park the
AG-Grid callback, replay on response, dedupe in-flight, eager head/tail
prefetch. Built net-new and runs PARALLEL to KeyAwareSmartRowCache — no
cutover until the rowid system is ready to merge. JS-only.

Lifecycle (reactive drive() loop): resolve the active view (fire sort/filter
and park if its permutation/subset isn't here — AG-Grid shows its loading
overlay because the callback doesn't fire); once built, missingAt the window
and either resolve+gc or fetch the missing rowids (by rowid, or positionally
via populate for the identity view). Sort is two round-trips. Filter+sort
compose client-side by intersection (free — both key off original_row_id).
Eager head/tail fires after a sort lands, like the existing second response.

9 lifecycle tests; the test commit was seen failing locally first (the tsc
pre-commit gate blocks committing a spec that imports a missing module, so
spec + impl land together).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

📦 TestPyPI package published

pip install --index-strategy unsafe-best-match --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ buckaroo==0.15.2.dev28179768503

or with uv:

uv pip install --index-strategy unsafe-best-match --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ buckaroo==0.15.2.dev28179768503

MCP server for Claude Code

claude mcp add buckaroo-table -- uvx --from "buckaroo[mcp]==0.15.2.dev28179768503" --index-strategy unsafe-best-match --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ buckaroo-table

📖 Docs preview

🎨 Storybook preview

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4eecad44b0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

for (const p of this.pending) {
if (!this.tryResolve(p)) still.push(p);
}
this.pending = still;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Avoid dropping reentrant row requests

If a success callback synchronously asks for another range (for example, a grid/request wrapper that requests the next block from inside the fulfilled callback), the nested getRows appends to this.pending while the outer drive() is still running; this assignment then replaces this.pending with the outer still snapshot and drops the newly parked request, so its later response will never replay the callback. Consider processing a snapshot or removing resolved entries before invoking callbacks so reentrant requests survive.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant