Repository navigation
Remove stale transaction log queues from health metrics - #14263
Closed
mengxu-oai wants to merge 1 commit into
Closed
mengxu-oai wants to merge 1 commit into
mengxu-oai wants to merge 1 commit into
Conversation
mengxu-oai
marked this pull request as ready for review
October 10, 2026 05:22
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem in brief
Detailed health metrics retain a transaction log's last queue value after its tracker becomes invalid or is removed. The aggregate worst queue already excludes that tracker, so the detailed and aggregate metrics can describe different sets of logs indefinitely.
Example
If log A has a queue of 500 bytes and log B has 100 bytes, the worst queue is 500. When A becomes invalid, the worst queue becomes 100, but A's 500-byte entry remains in the detailed health map.
Solution in brief
Rebuild the detailed TLog queue map from the valid trackers on each rate update. This changes health reporting only; rate limits and serialized types are unchanged.
The same example
After A becomes invalid, the detailed map contains only B's 100-byte queue, and the worst queue is 100. Removed trackers are handled the same way.
Validation
/fdbserver/Ratekeeper/TLogHealthMetricsto exercise valid, invalid, revalidated, removed, and empty tracker sets after normal and batch updates.Throttling.tomlwithHealthMetricsApi: passed with buggify enabled (1 test, 4 clients). All four clients received detailed health metrics. The trace contained no severity-40 events.The change removes retained entries in new Ratekeeper health updates. Existing client and proxy caching behavior is unchanged.