perf: add thread-local TraceCache to skip redundant TraceTree lookups#60
Open
azurezene wants to merge 1 commit into
Open
perf: add thread-local TraceCache to skip redundant TraceTree lookups#60azurezene wants to merge 1 commit into
azurezene wants to merge 1 commit into
Conversation
An 8-entry LRU cache per thread, keyed by backtrace hash with IP-level verification, avoids walking the shared TraceTree trie on repeated allocations from the same call site.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
In allocation-heavy workloads, every malloc/free must walk the shared TraceTree trie to resolve its backtrace. This lookup holds a lock and consumes significant critical-section time, causing contention under multi-threaded workloads.
However, high-frequency allocation paths tend to be highly repetitive — the same call site allocating in a loop, the same function calling malloc repeatedly. The backtrace is identical every time, yet we pay the full cost of walking the trie on each call.
This PR inserts a thread-local 8-entry LRU cache in front of TraceTree::index() to skip redundant trie traversals. A cache hit avoids both the trie walk and the associated lock contention entirely.
Implementation:
Key: hash of the backtrace (Murmur-style via 0xbb67ae8584caa73b), with memcmp verification of the first 4 IP frames to eliminate false positives from hash collisions.
Eviction: decay-based rather than exact LRU — each miss decays the tail entry's hit counter by 25%; entries drop below the replacement threshold after ~4 consecutive misses. Hot entries resist eviction naturally.
Cache-line aware layout: key[] in the first cache line (read on every lookup), hitCount/index in the second (touched only on hit), IP storage in colder lines — minimizing cache misses on the fast path.
Lock-free: thread_local storage means zero contention, unlike any shared cache structure.
Performance:
This is the flame graph proportion for /heaptrack/tests/manual/threaded.cpp. It can be clearly seen that the proportion of the TraceTree::index function within the heaptrack_malloc function has decreased from 31.1% to 0.4%.
without cache:

with cache:

I found This idea was originally suggested in #13 but never landed