Skip to content

Support the GCC 12.1+ gcov format in the built-in reader, and read a gcno's gcda in parallel - #1517

Open
michieldwitte wants to merge 5 commits into
mozilla:masterfrom
michieldwitte:gcc12-gcov-format
Open

Support the GCC 12.1+ gcov format in the built-in reader, and read a gcno's gcda in parallel#1517
michieldwitte wants to merge 5 commits into
mozilla:masterfrom
michieldwitte:gcc12-gcov-format

Conversation

@michieldwitte

Copy link
Copy Markdown

The built-in gcno/gcda reader only understood the format up to GCC 11, so anything newer failed to parse and coverage had to go through gcov instead. On GCC 15.2 output, master errors with Not enough data in buffer: cannot skip 10804527104 bytes and reports nothing at all.

GCC 12.1 changed three things: the gcno and gcda header carry a checksum after the stamp, lengths count bytes instead of 4-byte words and strings lost their padding (so records are no longer 4-byte aligned), and a negated record length marks counters that are all zero and left out. The reader now handles all three, keyed off the version it parsed, so older files are read exactly as before. Since records can now be unaligned, integers are read from their bytes rather than through a transmuted pointer.

The second half is performance, on that same reader. With one gcda per test run against a single build, the producer read them one at a time while the parsing threads waited: 501 translation units over 174 runs spent 1.06s reading 87k gcda (681 MB), and varying --threads from 1 to 16 made no difference at all. They are now read with rayon when the archives are plain directories, and the directory walk no longer opens every file it finds, since only gcno, info, dat, xml and out are ever sniffed. The same 681 MB now takes 0.64s. rayon was already a dependency.

Tests: gcc-11, gcc-12 and gcc-15 reader fixtures in the existing style plus one for the omitted zero counters, and version assertions on both sides of the 12.1 boundary. The golden dumps for gcc-9 through gcc-15 are byte-identical, so the same program parses to the same model across the format change. The parallel read has unit tests for buffer order and for falling back when an archive is a zip. cargo fmt --check and clippy are clean, and 142 tests pass — the 4 llvm_tools failures want llvm-profdata installed locally and fail the same way on master.

On a real C++ project (GCC 15.2, coverage from 174 separate test runs) the built-in reader reports the same 64.8% whether the runs are merged with gcov-tool merge first or passed in unmerged, so the merge step can be dropped entirely.

Michiel De Witte and others added 2 commits August 12, 2026 16:11
The built-in gcno/gcda reader only understood the format up to GCC 11, so
every file from a newer compiler failed to parse and coverage had to go
through gcov instead. Three things changed in GCC 12.1:

- the gcno and gcda header carry a checksum after the stamp
- lengths count bytes instead of 4-byte words, and strings lost their
  padding, so records are no longer 4-byte aligned
- a negated record length marks counters that are all zero and left out

Read integers from their bytes rather than transmuting a pointer, which
assumed an alignment the unpadded format no longer guarantees, and let a
record end exactly at the end of the buffer.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Merging many runs into one gcno leaves the single producer thread reading
one gcda per run, which starves the parsing threads: 501 translation units
over 174 runs spent 1.06s reading 87k gcda, and varying --threads from 1 to
16 made no difference at all.

Read them with rayon when the archives are plain directories, and stop
opening every file found while walking a directory, since only gcno, info,
dat, xml and out are ever sniffed. The same 681 MB of gcda now takes 0.64s.

Size rayon's global pool from --threads, so that the reads stay within the
thread budget that was asked for instead of always spanning every core.
This makes the path rewriting and the profraw aggregation honour it too,
as both were using the global pool as well.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Michiel De Witte and others added 2 commits August 12, 2026 16:16
Three lints fire on code that predates them: collapsible_match on a match arm
whose whole body is an if, which becomes a guard, byte_char_slices on the
comparison against an array of byte chars, and iter_kv_map on a map iterated
for its values only.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
gcov writes its json output before it bails out on a gcno whose version it
does not support, and that output holds a garbage working directory string.
gcov 12 and 15 name it after the gcno stem instead of after the whole file
name, so grcov does not find it where it looks for it and walks the working
dir instead, which took the leftover for the output of the next gcno: the
unwrap of its json then killed the parser thread and made grcov exit 1.

Remove what a failed gcov run left behind, and report a malformed json as a
parsing error rather than panicking on it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
gcov only knows the format of its own major version, and GCC 12.1 having moved
record lengths from 4-byte words to bytes makes gcov 12 and newer misread every
length of an older gcno: it then grinds through garbage for 15 to 20 seconds
before it segfaults. The 27 gcno of test/ cost gcov 12 62s and gcov 15 80s that
way, a price the two tests running grcov over the whole repository each pay
twice.

Read the version stamp that follows the magic and skip a gcno of another major
before spawning gcov on it, which takes running grcov over the repository from
90s to 0.9s. Only the major is compared, as a differing minor is just a warning
of gcov, which does read the file.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant