Repository navigation
Conversation
Add caller-owned purge eligibility and dry-run preview without changing container lifecycle or enabling purge by default. Scan MySQL purge pages in primary-key order, wait for metadata cleanup before paging, and stop on deletion failures or repeated versions. Keep deployment and migration-specific policy outside the generic cleaner. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #3305 +/- ##
=============================================
- Coverage 64.24% 50.70% -13.54%
+ Complexity 10398 8711 -1687
=============================================
Files 840 939 +99
Lines 71755 80881 +9126
Branches 8611 9768 +1157
=============================================
- Hits 46099 41014 -5085
- Misses 23004 36465 +13461
- Partials 2652 3402 +750 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Metadata cleanup timestamps use System.currentTimeMillis(), whereas named blob GET evaluates deletion against the injected MockTime. Reset the shared test clock at the start and after cleanup so the Deleted assertion does not depend on test ordering or elapsed suite time. Preserve all existing purge paging and deletion assertions. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
zichengl
marked this pull request as ready for review
October 13, 2026 17:00
nicolaslopezbravo
approved these changes
Oct 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Durability risk: high when enabled. Purge intentionally deletes live/latest versions. Callers must explicitly approve each container, fence source reads/writes, and verify that the source data is no longer needed. Eligibility is checked before each page and delete, but revocation cannot cancel an already-issued deletion. Deletion or metadata failures stop purge; subsequent runs restart the sweep. Repeated versions after metadata cleanup also stop the sweep, and purge never skips names after query kills.
This draft proposes only the generic OSS mechanism. Deployment configuration, migration/completeness checks, composite-database delegation, and verification of composite-router delete routing remain integration responsibilities. No rollout is included. Expired/already-deleted mappings retain their existing cleanup paths. Primary-key ordering avoids the mixed-order filesort but does not guarantee bounded latency when many index entries are filtered.
Please review the API/runner policy first, then the MySQL paging behavior.
Testing Done
Local validation uses JDK 17; CI uses JDK 11.
./gradlew --no-daemon --console=plain -PdisableShipkit :ambry-frontend:test --tests com.github.ambry.frontend.NamedBlobsCleanupRunnerTest :ambry-named-mysql:test --tests com.github.ambry.named.MySqlNamedBlobDbTest :ambry-named-mysql:compileIntTestJavagit diff --check./gradlew --no-daemon --console=plain -PdisableShipkit :ambry-named-mysql:intTest --tests 'com.github.ambry.named.MySqlNamedBlobDbIntegrationTest.testPurge*'./gradlew --scan --warning-mode=summary intTest -x :ambry-server:intTest codeCoverageReportFull local build/full local test suite not run.