Skip to content

keyspace: load keyspace by ID through gRPC#10911

Merged
ti-chi-bot[bot] merged 9 commits into
tikv:masterfrom
rleungx:keyspace-load-by-id
Jun 23, 2026
Merged

keyspace: load keyspace by ID through gRPC#10911
ti-chi-bot[bot] merged 9 commits into
tikv:masterfrom
rleungx:keyspace-load-by-id

Conversation

@rleungx

@rleungx rleungx commented Jun 18, 2026

Copy link
Copy Markdown
Member

What problem does this PR solve?

Issue Number: ref #10912

What is changed and how does it work?

Add a LoadKeyspaceByID gRPC handler that loads keyspace metadata by ID and applies the same region-bound readiness check as LoadKeyspace.

Expose LoadKeyspaceByID on the Go PD client with request metrics, and add unit coverage for successful responses and header errors.

Update kvproto to the merged pingcap/kvproto#1491 version for the new RPC definition.

Check List

Tests

  • Unit test

Related changes

Release note

Support loading keyspace metadata by keyspace ID through the gRPC keyspace service.

Summary by CodeRabbit

  • New Features

    • Added ability to load keyspace metadata by keyspace ID with comprehensive error handling and region validation.
    • Enhanced monitoring with new metrics for keyspace loading operations, tracking both successful and failed request durations.
  • Chores

    • Updated dependency versions across modules for improved compatibility.

@ti-chi-bot ti-chi-bot Bot added do-not-merge/needs-linked-issue dco-signoff: yes Indicates the PR's author has signed the dco. release-note Denotes a PR that will be considered when it comes time to generate release notes. labels Jun 18, 2026
@coderabbitai

coderabbitai Bot commented Jun 18, 2026

Copy link
Copy Markdown

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds LoadKeyspaceByID end-to-end: kvproto is bumped across four go.mod files to expose the new proto types; a gRPC server handler is added with region-bound validation and a failpoint bypass; the PD client interface, implementation, and Prometheus metrics are extended; and unit tests plus an integration test cover success, header-error, RPC-error, and cluster-ID-mismatch paths.

Changes

LoadKeyspaceByID feature

Layer / File(s) Summary
kvproto dependency bump
go.mod, client/go.mod, tests/integrations/go.mod, tools/go.mod
Updates github.com/pingcap/kvproto to v0.0.0-20260622063236-b41e86365ce0 across all four module files, exposing the new LoadKeyspaceByIDRequest and LoadKeyspaceResponse proto types.
Server gRPC handler
server/keyspace_service.go
Adds KeyspaceServer.LoadKeyspaceByID, which validates the request header, loads keyspace metadata by ID via KeyspaceManager, applies a skipKeyspaceRegionCheck failpoint bypass, and checks CheckKeyspaceRegionBound—returning ErrKeyspaceNotFound for unsplit regions or a success response otherwise.
Client metrics observers
client/metrics/metrics.go
Registers CmdDurationLoadKeyspaceByID and CmdFailedDurationLoadKeyspaceByID Prometheus observers, initialized from cmdDuration and cmdFailedDuration histogram vectors using the load_keyspace_by_id label.
Client interface and implementation
client/keyspace_client.go
Adds LoadKeyspaceByID to the KeyspaceClient interface and implements (*client).LoadKeyspaceByID with tracing, metrics recording, timeout enforcement, proto RPC invocation, and RPC/header error handling including ScheduleCheckMemberChanged on RPC failure.
Client unit tests
client/keyspace_client_test.go
Defines testKeyspaceServer (in-process gRPC with atomic ID capture and configurable error header/RPC error), startTestKeyspaceServer and newTestKeyspaceClient helpers, and four tests covering success, header-error, proto-client-unavailable, and RPC-error paths.
Server integration test
tests/server/keyspace/keyspace_service_test.go
Adds TestLoadKeyspaceByIDGRPC exercising multiple scenarios: cluster-ID mismatch returns an error, non-existent keyspace ID returns ENTRY_NOT_FOUND, initial correct-ID returns ENTRY_NOT_FOUND until region boundaries are written, successful load after region boundaries are inserted, and successful load with the skipKeyspaceRegionCheck failpoint enabled.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related issues

Possibly related PRs

  • tikv/pd#10556: Calls LoadKeyspaceByID from a keyspace-group delete handler, making it a direct consumer of the RPC this PR introduces.

Suggested labels

size/XL, lgtm

Suggested reviewers

  • JmPotato
  • lhy1024
  • bufferflies

🐇 A hop, a skip, a keyspace found,
By ID now, on hallowed ground.
The region's split, the failpoint's set,
No keyspace lost that we've once met.
Metrics gleam with every call—
LoadKeyspaceByID serves them all! 🎉

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately and concisely describes the main change: adding a LoadKeyspaceByID gRPC functionality to load keyspace metadata by ID.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description check ✅ Passed The PR description covers the problem statement, detailed changes, includes a commit message, specifies unit tests, and provides a release note.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@ti-chi-bot ti-chi-bot Bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Jun 18, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@client/keyspace_client_test.go`:
- Around line 71-80: The errors from server.Serve and listener.Close are being
silently dropped with blank identifiers, which hides lifecycle failures. In the
goroutine where server.Serve is called, capture the error and log it or store it
for test diagnostics. Similarly, in the cleanup function, capture the error from
listener.Close() and handle it appropriately by logging or asserting on it.
Alternatively, if these errors should legitimately be ignored, add explicit
errcheck exclusions to .golangci.yml for these specific error cases.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 18be4bc5-9e8e-450b-b85f-2873c964d74a

📥 Commits

Reviewing files that changed from the base of the PR and between 8179ce7 and bc54da8.

⛔ Files ignored due to path filters (4)
  • client/go.sum is excluded by !**/*.sum
  • go.sum is excluded by !**/*.sum
  • tests/integrations/go.sum is excluded by !**/*.sum
  • tools/go.sum is excluded by !**/*.sum
📒 Files selected for processing (8)
  • client/go.mod
  • client/keyspace_client.go
  • client/keyspace_client_test.go
  • client/metrics/metrics.go
  • go.mod
  • server/keyspace_service.go
  • tests/integrations/go.mod
  • tools/go.mod

Comment thread client/keyspace_client_test.go Outdated
@ti-chi-bot ti-chi-bot Bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jun 18, 2026
@rleungx

rleungx commented Jun 18, 2026

Copy link
Copy Markdown
Member Author

/test pull-unit-test-next-gen-3

@codecov

codecov Bot commented Jun 18, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 79.18%. Comparing base (4ae1408) to head (c95b03d).

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #10911      +/-   ##
==========================================
+ Coverage   79.15%   79.18%   +0.02%     
==========================================
  Files         540      540              
  Lines       74801    74847      +46     
==========================================
+ Hits        59212    59266      +54     
+ Misses      11405    11402       -3     
+ Partials     4184     4179       -5     
Flag Coverage Δ
unittests 79.18% <100.00%> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
server/keyspace_service_test.go (1)

58-65: ⚡ Quick win

Assert the exact failure contract for mismatched cluster ID.

At Line 62, re.Error(err) is too broad and can pass on unrelated failures. This case should verify mismatch-cluster validation specifically, and that no response is returned.

Proposed test tightening
-	_, err = service.LoadKeyspaceByID(ctx, &keyspacepb.LoadKeyspaceByIDRequest{
+	resp, err := service.LoadKeyspaceByID(ctx, &keyspacepb.LoadKeyspaceByIDRequest{
 		Header: testutil.NewRequestHeader(leader.GetClusterID() + 1),
 		Id:     created.GetId(),
 	})
 	re.Error(err)
+	re.ErrorContains(err, "mismatch cluster id")
+	re.Nil(resp)
 
-	resp, err := service.LoadKeyspaceByID(ctx, &keyspacepb.LoadKeyspaceByIDRequest{
+	resp, err = service.LoadKeyspaceByID(ctx, &keyspacepb.LoadKeyspaceByIDRequest{
 		Header: testutil.NewRequestHeader(leader.GetClusterID()),
 		Id:     created.GetId() + 1000,
 	})
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@server/keyspace_service_test.go` around lines 58 - 65, The test assertion at
line 62 uses a generic re.Error(err) check which is too broad and could pass for
any error condition, not specifically validating the cluster ID mismatch
contract. Replace the generic error assertion with specific checks that verify
both the error is for cluster ID mismatch (check error type, code, or message
for cluster validation failure) and that the response returned from the
LoadKeyspaceByID call is nil when the cluster ID in the request header does not
match the leader cluster ID.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@server/keyspace_service_test.go`:
- Around line 58-65: The test assertion at line 62 uses a generic re.Error(err)
check which is too broad and could pass for any error condition, not
specifically validating the cluster ID mismatch contract. Replace the generic
error assertion with specific checks that verify both the error is for cluster
ID mismatch (check error type, code, or message for cluster validation failure)
and that the response returned from the LoadKeyspaceByID call is nil when the
cluster ID in the request header does not match the leader cluster ID.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 14b3632b-8117-4218-b264-0c369e04efa8

📥 Commits

Reviewing files that changed from the base of the PR and between bc54da8 and 7929b60.

⛔ Files ignored due to path filters (4)
  • client/go.sum is excluded by !**/*.sum
  • go.sum is excluded by !**/*.sum
  • tests/integrations/go.sum is excluded by !**/*.sum
  • tools/go.sum is excluded by !**/*.sum
📒 Files selected for processing (1)
  • server/keyspace_service_test.go

@rleungx

rleungx commented Jun 18, 2026

Copy link
Copy Markdown
Member Author

/test pull-unit-test-next-gen-3
/test pull-integration-realcluster-test

@rleungx

rleungx commented Jun 18, 2026

Copy link
Copy Markdown
Member Author

/test pull-unit-test-next-gen-3

1 similar comment
@rleungx

rleungx commented Jun 18, 2026

Copy link
Copy Markdown
Member Author

/test pull-unit-test-next-gen-3

@rleungx

rleungx commented Jun 18, 2026

Copy link
Copy Markdown
Member Author

/test pull-integration-realcluster-test

@rleungx

rleungx commented Jun 18, 2026

Copy link
Copy Markdown
Member Author

/test pull-unit-test-next-gen-3

1 similar comment
@rleungx

rleungx commented Jun 18, 2026

Copy link
Copy Markdown
Member Author

/test pull-unit-test-next-gen-3

@rleungx

rleungx commented Jun 18, 2026

Copy link
Copy Markdown
Member Author

/retest

Comment thread go.mod Outdated
// kvproto at the same time. You can run `go mod tidy` to make it replaced with go-mod style specification.
// After the PR to kvproto is merged, remember to comment this out and run `go mod tidy`.
// replace github.com/pingcap/kvproto => github.com/$YourPrivateRepo $YourPrivateBranch
replace github.com/pingcap/kvproto => github.com/pingcap/kvproto v0.0.0-20260618032004-5ec421b79217

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why use replace?

@bufferflies bufferflies left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rest lgtm

@ti-chi-bot ti-chi-bot Bot added needs-1-more-lgtm Indicates a PR needs 1 more LGTM. approved labels Jun 18, 2026
rleungx added 6 commits June 22, 2026 14:34
Signed-off-by: Ryan Leung <rleungx@gmail.com>
Signed-off-by: Ryan Leung <rleungx@gmail.com>
Signed-off-by: Ryan Leung <rleungx@gmail.com>
Signed-off-by: Ryan Leung <rleungx@gmail.com>
Signed-off-by: Ryan Leung <rleungx@gmail.com>
Signed-off-by: Ryan Leung <rleungx@gmail.com>
Signed-off-by: Ryan Leung <rleungx@gmail.com>
@rleungx rleungx force-pushed the keyspace-load-by-id branch from 9483e13 to b02d6fb Compare June 22, 2026 06:49
@ti-chi-bot ti-chi-bot Bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Jun 22, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/server/keyspace/keyspace_service_test.go`:
- Around line 78-88: The failpoint test at the LoadKeyspaceByID call (around
line 82-88) is not actually exercising the skipKeyspaceRegionCheck failpoint
because it reuses the success state from the earlier LoadKeyspaceByID call on
lines 64-69. To fix this, you need to modify the test state between the two
calls so that LoadKeyspaceByID would fail the region check without the failpoint
enabled. This could involve removing or invalidating the keyspace regions after
the first successful call, then enable the failpoint and verify that
LoadKeyspaceByID succeeds anyway, which proves the skipKeyspaceRegionCheck
failpoint is properly bypassing the region validation logic.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: b8278253-094f-495f-a6a2-06d69374de29

📥 Commits

Reviewing files that changed from the base of the PR and between 9483e13 and b02d6fb.

⛔ Files ignored due to path filters (4)
  • client/go.sum is excluded by !**/*.sum
  • go.sum is excluded by !**/*.sum
  • tests/integrations/go.sum is excluded by !**/*.sum
  • tools/go.sum is excluded by !**/*.sum
📒 Files selected for processing (9)
  • client/go.mod
  • client/keyspace_client.go
  • client/keyspace_client_test.go
  • client/metrics/metrics.go
  • go.mod
  • server/keyspace_service.go
  • tests/integrations/go.mod
  • tests/server/keyspace/keyspace_service_test.go
  • tools/go.mod
🚧 Files skipped from review as they are similar to previous changes (3)
  • client/keyspace_client.go
  • server/keyspace_service.go
  • client/metrics/metrics.go

Comment thread tests/server/keyspace/keyspace_service_test.go Outdated
Signed-off-by: Ryan Leung <rleungx@gmail.com>
@rleungx

rleungx commented Jun 22, 2026

Copy link
Copy Markdown
Member Author

/retest

@rleungx

rleungx commented Jun 22, 2026

Copy link
Copy Markdown
Member Author

/test pull-unit-test-next-gen-3

1 similar comment
@rleungx

rleungx commented Jun 22, 2026

Copy link
Copy Markdown
Member Author

/test pull-unit-test-next-gen-3

Signed-off-by: Ryan Leung <rleungx@gmail.com>
@ti-chi-bot

ti-chi-bot Bot commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

@bufferflies: Your lgtm message is repeated, so it is ignored.

Details

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@ti-chi-bot ti-chi-bot Bot added the lgtm label Jun 23, 2026
@ti-chi-bot

ti-chi-bot Bot commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bufferflies, JmPotato

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:
  • OWNERS [JmPotato,bufferflies]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot Bot removed the needs-1-more-lgtm Indicates a PR needs 1 more LGTM. label Jun 23, 2026
@ti-chi-bot

ti-chi-bot Bot commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

[LGTM Timeline notifier]

Timeline:

  • 2026-06-18 11:43:35.602901272 +0000 UTC m=+1651516.673218652: ☑️ agreed by bufferflies.
  • 2026-06-23 09:02:15.496723512 +0000 UTC m=+24619.159949025: ☑️ agreed by JmPotato.

@ti-chi-bot ti-chi-bot Bot merged commit 17e9771 into tikv:master Jun 23, 2026
41 of 43 checks passed
@rleungx rleungx deleted the keyspace-load-by-id branch June 23, 2026 09:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved dco-signoff: yes Indicates the PR's author has signed the dco. lgtm release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants