Run StaticIndex::compute over multiple threads by nicolas-guichard · Pull Request #22530 · rust-lang/rust-analyzer

nicolas-guichard · 2026-06-05T15:40:09Z

Each thread takes care of the files whose index in the precomputed list is congruent to the thread index modulo the thread count and generates a partial StaticIndex. All those StaticIndex instances are then simply concatenated together.

On my laptop with 16 threads this brings the time required to generate an SCIP index of Firefox from 80 seconds with 160% CPU usage down to 30 seconds, with 600% CPU usage.

Fixes #18140

nicolas-guichard · 2026-06-05T15:49:01Z

+        };
+
+        for (def, token) in tokens {
+            lsif.add_token(def, token);


I'm not too happy about this: ideally lsif_contains_generated_constant would be order-independent but is it worth the time?

View changes since the review

nicolas-guichard · 2026-07-22T13:11:05Z

I had to rework this because of the new lifetime in Definition<'db> added in #22548. This reverts to using two maps Definition<'db>→TokenId and TokenId→TokenStaticData so that the final StaticIndex doesn't depend on the Analysis lifetime.

This is actually more efficient in CPU time than my first iteration which built a Definition→TokenStaticData map per thread then merged them (thus doing redundant work).

time says for rust-analyzer scip firefox:

1649637 (old master): 141,50s user 5,38s system 131% cpu 1:51,64 total
0bd261e (old pr rev): 283,77s user 6,31s system 748% cpu 38,769 total
1174734 (new master): 142,98s user 5,12s system 130% cpu 1:53,39 total
55b7509 (new pr rev): 190,18s user 6,03s system 431% cpu 45,470 total

When multi-threading StaticIndex::compute, each thread will get its own Analysis clone.

When multi-threading StaticIndex::compute, we want the list of files to be computed only once and the files to index to be split between threads.

The analysis, db and def_map members of StaticIndex were only used for building the index and are of no use once the index is built. Instead of have an &mut self method which mutates those fields, we can extract the FileId→StaticIndexedFile indexing into a free function and keep the token_store and def_map as locals in StaticIndex::compute. This will make multi-threading StaticIndex::compute easier.

Thanks to the previous commits, everything is in place to start multiple threads in StaticIndex::compute. Each thread takes care of the files whose index in the precomputed list is congruent to the thread index modulo the thread count and generates its StaticIndexedFile. The Definition→TokenId and TokenId→TokenStaticData maps are shared across all threads and are now DashMaps. TokenStore now uses an AtomicUsize to generate new TokenIds.

rustbot · 2026-07-27T12:41:15Z

This PR was rebased onto a different master commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

rustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jun 5, 2026

nicolas-guichard commented Jun 5, 2026

View reviewed changes

Veykril reviewed Jun 6, 2026

View reviewed changes

Comment thread crates/ide/src/folding_ranges.rs

Veykril reviewed Jun 6, 2026

View reviewed changes

Comment thread crates/ide/src/static_index.rs Outdated