Repository navigation
Reporting test cases marked with "pending" Kaocha tag as "skipped" in JUnit XML output #20
Description
Background
Kaocha allows tests to be marked with a pending tag. This allows tests to be filtered when running tests using Kaocha but still have the data reported in the output. This is useful for building a suite of tests for a pending, yet-to-be-implemented feature. The following is an example of a simple test marked with this tag:
(ns sample.project.sample-test
(:require [clojure.test :refer [deftest is]]))
(deftest ^:kaocha/pending pending-test
(is (= 1 1)
"This should be marked as 'skipped' by JUnit!"))I am working in a Polylith codebase using polylith-kaocha as the test runner Running tests yield the following output:
1 test, 0 assertions, 1 pending, 0 failures.
PENDING sample.project.pending-test (:)
We've configured our polylith-kaocha tests to generate JUnit XML test results using this plugin in our Kaocha config. This is being generated however we're hitting a snag because the pending tests aren't actually running (as expected), but in the JUnit XML test results they look like they are running and passing:
- Run the sample-test suite
clj -M:poly test - Open the generated JUnit XML test report:
<?xml version='1.0' encoding='UTF-8'?>
<testsuites>
<testsuite errors="0" package="" tests="1" name="sample-test" time="0.545124" hostname="localhost" id="0" timestamp="2025-11-11T10:51:50" failures="0">
<properties/>
<testcase name="sample.project.sample-test/pending-test" classname="sample.project.sample-test" time="0.000000" line="4" column="1" file="/sample/project/sample_test.clj">
</testcase>
<system-out/>
<system-err/>
</testsuites>- Note the
pending-testresult in the JUnit XML test report is missing the<skipped />element that would usually be nested within<testcase>element to indicate that the test has been skipped.
Desired behaviour
- Run the sample-test suite
clj -M:poly test - Open the generated JUnit XML test report:
<?xml version='1.0' encoding='UTF-8'?>
<testsuites>
<testsuite errors="0" package="" tests="1" name="sample-test" time="0.545124" hostname="localhost" id="0" timestamp="2025-11-11T10:51:50" failures="0">
<properties/>
<testcase name="sample.project.sample-test/pending-test" classname="sample.project.sample-test" time="0.000000" line="4" column="1" file="/sample/project/sample_test.clj">
<skipped/>
</testcase>
<system-out/>
<system-err/>
</testsuites>- Observe the test result for
sample-testis correctly tagged asskipped
Please let me know if I can provide any more detail, as this is my first time raising an issue on GitHub. Thanks in advance for any help you can provide!
Activity
plexus commentedon Nov 11, 2025
Thanks for the report, I suspect it's just missing a filter and these can be straightforwardly elided. A pull request with a fix would be much appreciated.
Looking at the code it's probably easiest to already filter these out in the
leaf-testsfunction: https://github.com/lambdaisland/kaocha-junit-xml/blob/main/src/kaocha/plugin/junit_xml.clj#L32-L35If this is your first time hacking on kaocha, start here for an overview of the data model: https://cljdoc.org/d/lambdaisland/kaocha/1.91.1392/doc/9-extending
samcarver4 commentedon Nov 12, 2025
Thanks for the quick reply @plexus , I'll check it out and have a look!
samcarver4 commentedon Nov 17, 2025
Hi @plexus , I have a PR candidate ready on a new branch, however it looks like I don't have permission to push this - please let me know if this is something you can help with! Thanks again in advance.
samcarver4 commentedon Dec 1, 2025
Hi @plexus , just checking if there's any update on this access issue? 👆
I have a fix ready to raise as a Pull Request, but am getting blocked by GitHub when trying to do so with an error:
plexus commentedon Dec 1, 2025
You normally don't push to the upstream repo, you push to your own fork of the repo, you can then make a PR from there.
samcarver4 commentedon Dec 2, 2025
Thanks for the update @plexus - I have a PR available for review here. Thanks in advance!