Skip to content

perf: add thread-local TraceCache to skip redundant TraceTree lookups#60

Open
azurezene wants to merge 1 commit into
KDE:masterfrom
azurezene:opti/cache
Open

perf: add thread-local TraceCache to skip redundant TraceTree lookups#60
azurezene wants to merge 1 commit into
KDE:masterfrom
azurezene:opti/cache

Conversation

@azurezene

@azurezene azurezene commented Jul 2, 2026

Copy link
Copy Markdown

In allocation-heavy workloads, every malloc/free must walk the shared TraceTree trie to resolve its backtrace. This lookup holds a lock and consumes significant critical-section time, causing contention under multi-threaded workloads.

However, high-frequency allocation paths tend to be highly repetitive — the same call site allocating in a loop, the same function calling malloc repeatedly. The backtrace is identical every time, yet we pay the full cost of walking the trie on each call.

This PR inserts a thread-local 8-entry LRU cache in front of TraceTree::index() to skip redundant trie traversals. A cache hit avoids both the trie walk and the associated lock contention entirely.

Implementation:

Key: hash of the backtrace (Murmur-style via 0xbb67ae8584caa73b), with memcmp verification of the first 4 IP frames to eliminate false positives from hash collisions.
Eviction: decay-based rather than exact LRU — each miss decays the tail entry's hit counter by 25%; entries drop below the replacement threshold after ~4 consecutive misses. Hot entries resist eviction naturally.
Cache-line aware layout: key[] in the first cache line (read on every lookup), hitCount/index in the second (touched only on hit), IP storage in colder lines — minimizing cache misses on the fast path.
Lock-free: thread_local storage means zero contention, unlike any shared cache structure.

Performance:

This is the flame graph proportion for /heaptrack/tests/manual/threaded.cpp. It can be clearly seen that the proportion of the TraceTree::index function within the heaptrack_malloc function has decreased from 31.1% to 0.4%.

without cache:
图片

with cache:
图片

I found This idea was originally suggested in #13 but never landed

An 8-entry LRU cache per thread, keyed by backtrace hash with IP-level
verification, avoids walking the shared TraceTree trie on repeated
allocations from the same call site.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant