Support the GCC 12.1+ gcov format in the built-in reader, and read a gcno's gcda in parallel - #1517
Open
michieldwitte wants to merge 5 commits into
Open
Support the GCC 12.1+ gcov format in the built-in reader, and read a gcno's gcda in parallel#1517michieldwitte wants to merge 5 commits into
michieldwitte wants to merge 5 commits into
Conversation
michieldwitte
force-pushed
the
gcc12-gcov-format
branch
from
August 12, 2026 13:44
97789b7 to
286f148
Compare
The built-in gcno/gcda reader only understood the format up to GCC 11, so every file from a newer compiler failed to parse and coverage had to go through gcov instead. Three things changed in GCC 12.1: - the gcno and gcda header carry a checksum after the stamp - lengths count bytes instead of 4-byte words, and strings lost their padding, so records are no longer 4-byte aligned - a negated record length marks counters that are all zero and left out Read integers from their bytes rather than transmuting a pointer, which assumed an alignment the unpadded format no longer guarantees, and let a record end exactly at the end of the buffer. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Merging many runs into one gcno leaves the single producer thread reading one gcda per run, which starves the parsing threads: 501 translation units over 174 runs spent 1.06s reading 87k gcda, and varying --threads from 1 to 16 made no difference at all. Read them with rayon when the archives are plain directories, and stop opening every file found while walking a directory, since only gcno, info, dat, xml and out are ever sniffed. The same 681 MB of gcda now takes 0.64s. Size rayon's global pool from --threads, so that the reads stay within the thread budget that was asked for instead of always spanning every core. This makes the path rewriting and the profraw aggregation honour it too, as both were using the global pool as well. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
michieldwitte
force-pushed
the
gcc12-gcov-format
branch
from
August 12, 2026 14:12
286f148 to
5d41f40
Compare
Three lints fire on code that predates them: collapsible_match on a match arm whose whole body is an if, which becomes a guard, byte_char_slices on the comparison against an array of byte chars, and iter_kv_map on a map iterated for its values only. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
gcov writes its json output before it bails out on a gcno whose version it does not support, and that output holds a garbage working directory string. gcov 12 and 15 name it after the gcno stem instead of after the whole file name, so grcov does not find it where it looks for it and walks the working dir instead, which took the leftover for the output of the next gcno: the unwrap of its json then killed the parser thread and made grcov exit 1. Remove what a failed gcov run left behind, and report a malformed json as a parsing error rather than panicking on it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
michieldwitte
force-pushed
the
gcc12-gcov-format
branch
from
August 12, 2026 14:40
5d41f40 to
20e0b65
Compare
gcov only knows the format of its own major version, and GCC 12.1 having moved record lengths from 4-byte words to bytes makes gcov 12 and newer misread every length of an older gcno: it then grinds through garbage for 15 to 20 seconds before it segfaults. The 27 gcno of test/ cost gcov 12 62s and gcov 15 80s that way, a price the two tests running grcov over the whole repository each pay twice. Read the version stamp that follows the magic and skip a gcno of another major before spawning gcov on it, which takes running grcov over the repository from 90s to 0.9s. Only the major is compared, as a differing minor is just a warning of gcov, which does read the file. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The built-in gcno/gcda reader only understood the format up to GCC 11, so anything newer failed to parse and coverage had to go through gcov instead. On GCC 15.2 output, master errors with Not enough data in buffer: cannot skip 10804527104 bytes and reports nothing at all.
GCC 12.1 changed three things: the gcno and gcda header carry a checksum after the stamp, lengths count bytes instead of 4-byte words and strings lost their padding (so records are no longer 4-byte aligned), and a negated record length marks counters that are all zero and left out. The reader now handles all three, keyed off the version it parsed, so older files are read exactly as before. Since records can now be unaligned, integers are read from their bytes rather than through a transmuted pointer.
The second half is performance, on that same reader. With one gcda per test run against a single build, the producer read them one at a time while the parsing threads waited: 501 translation units over 174 runs spent 1.06s reading 87k gcda (681 MB), and varying --threads from 1 to 16 made no difference at all. They are now read with rayon when the archives are plain directories, and the directory walk no longer opens every file it finds, since only gcno, info, dat, xml and out are ever sniffed. The same 681 MB now takes 0.64s. rayon was already a dependency.
Tests: gcc-11, gcc-12 and gcc-15 reader fixtures in the existing style plus one for the omitted zero counters, and version assertions on both sides of the 12.1 boundary. The golden dumps for gcc-9 through gcc-15 are byte-identical, so the same program parses to the same model across the format change. The parallel read has unit tests for buffer order and for falling back when an archive is a zip. cargo fmt --check and clippy are clean, and 142 tests pass — the 4 llvm_tools failures want llvm-profdata installed locally and fail the same way on master.
On a real C++ project (GCC 15.2, coverage from 174 separate test runs) the built-in reader reports the same 64.8% whether the runs are merged with gcov-tool merge first or passed in unmerged, so the merge step can be dropped entirely.