Repository navigation
Gate CI on doctest's JUnit report instead of scraping stdout - #80
Merged
Merged
Conversation
The preflight and per-distro build scripts grepped the human-oriented "[doctest] assertions:" console summary, which is not a stable interface across doctest versions. Add ci/check-doctest-junit.sh: rerun the test binary with doctest's machine-readable JUnit reporter and parse the XML with python3. JUnit separates assertion failures (<failure>) from test cases aborted by environment exceptions (<error>), so the long-standing container policy carries over: failures gate, environment errors are tolerated and reported. An empty or unreadable report fails loudly. ctest still runs the full suite first for console diagnostics and the shell-based CLI tests; its exit code stays informational. Fixes #60
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The preflight and per-distro build scripts grepped the human-oriented
[doctest] assertions:console line, which is not a stable interface across doctest versions.ci/check-doctest-junit.sh: reruns the test binary with doctest's machine-readable JUnit reporter and parses the XML with python3. JUnit natively separates assertion failures (<failure>) from test cases aborted by environment exceptions (<error>), so the long-standing container policy carries over exactly: failures gate, environment errors are tolerated and reported. Empty or unreadable reports fail loudly.cteststill runs the full suite first for console diagnostics and the shell-based CLI tests; its exit code stays informational.FAIL()from failure counts (DOCTEST_FAIL does not mark test as failed in junit report. doctest/doctest#755), which would false-green the gate (Codex review catch; the current lock resolves 2.4.12, so no version churn).Validated end to end in the almalinux:10 preflight container: the new gate ran against the real suite and reported
1265 reported test(s), 0 assertion failure(s), 1 environment error(s) tolerated, exit 0. The missing-report and injected-failure paths were exercised manually and both fail the gate. Codex review: clean.Fixes #60