GitHub - lambdaisland/kaocha-junit-xml: JUnit XML output for Kaocha (original) (raw)

lambdaisland/kaocha-junit-xml

Kaocha plugin to generate a JUnit XML version of the test results.

Support Lambda Island Open Source

Thank you! kaocha-junit-xml is made possible thanks to our generous backers. Become a backer on OpenCollective so that we can continue to make kaocha-junit-xml better.

kaocha-junit-xml is part of a growing collection of quality Clojure libraries and tools released on the Lambda Island label. If you are using this project commercially then you are expected to pay it forward bybecoming a backer on Open Collective, so that we may continue to enjoy a thriving Clojure ecosystem.

Usage

Add kaocha-junit-xml as a dependency

;; deps.edn {:aliases {:test {:extra-deps {lambdaisland/kaocha {...} lambdaisland/kaocha-junit-xml {:mvn/version "1.17.101"}}}}}

;; project.clj (defproject ,,, :dependencies [,,, [lambdaisland/kaocha-junit-xml "1.17.101"]])

Enable the plugin and set an output file

;; tests.edn #kaocha/v1 {:plugins [:kaocha.plugin/junit-xml] :kaocha.plugin.junit-xml/target-file "junit.xml"}

Or from the CLI

bin/kaocha --plugin kaocha.plugin/junit-xml --junit-xml-file junit.xml

Optionally you can omit captured output from junit.xml

;; tests.edn #kaocha/v1 {:plugins [:kaocha.plugin/junit-xml] :kaocha.plugin.junit-xml/target-file "junit.xml" :kaocha.plugin.junit-xml/omit-system-out? true}

Or from the CLI

bin/kaocha --plugin kaocha.plugin/junit-xml --junit-xml-file junit.xml --junit-xml-omit-system-out

Requirements

Requires at least Kaocha 0.0-306 and Clojure 1.9.

CI Integration

Some CI tooling supports the junit xml output in various flavours.

CircleCI

One of the services that can use this output is CircleCI. Your.circleci/config.yml could look like this:

version: 2 jobs: build: docker: - image: circleci/clojure:tools-deps-1.9.0.394 steps: - checkout - run: mkdir -p test-results/kaocha - run: bin/kaocha --plugin kaocha.plugin/junit-xml --junit-xml-file test-results/kaocha/results.xml --junit-xml-add-location-metadata --junit-xml-use-relative-path-in-location - store_test_results: path: test-results

GitHub Actions

The following configuration will create annotations for test failures on files of the relevant commit/PR. First enable the plugin with the add-location-metadata?flag in your tests.edn:

#kaocha/v1 {:plugins [:kaocha.plugin/junit-xml] :kaocha.plugin.junit-xml/target-file "junit.xml" :kaocha.plugin.junit-xml/add-location-metadata? true}

Then, an example .github/workflows/build.yml may look like:

name: Build on: [push]

jobs: build: runs-on: ubuntu-latest container: image: clojure:openjdk-8-tools-deps-1.11.1.1113 steps: - uses: actions/checkout@v2 - name: test run: | bin/kaocha - name: Annotate failure if: failure() uses: mikepenz/action-junit-report@41a3188dde10229782fd78cd72fc574884dd7686 with: report_paths: junit.xml

Gitlab

Configuring Gitlab to parse JUnit XML is easy; just add a report artifact that points to the XML file:

test: only: -tags script: - make test artifacts: reports: junit: junit.xml

See the Gitlab documentation on reports using JUnit for more information.

Caveats

For timing information (timestamp and running time) this plugin relies on thekaocha.plugin/profiling plugin. If the plugin is not present then a running time of 0 will be reported.

For output capturing the kaocha.plugin/capture-output must be present. If it is not present <system-out> will always be empty.

Resources

It was hard to find a definitive source of the Ant Junit XML format. I mostly went with this page for documentation.

For information on how to configure CircleCI to use this information, seestore_test_results.

After reports that the output was not compatible with Azure Devops Pipeline the output was changed to adhere to this schema.

The --junit-xml-add-location-metadata flag was added to enhance testcaseoutput with test location metadata à lapytest. This allows for integration with various tools on GitHub Actions for producing annotations on files in commits/PRs with test failure data. For example, theJUnit Report Action.

The --junit-xml-use-relative-path-in-location flasg was added to enhance the --junit-xml-add-location-metadataoutput for test ecosystems that rely on relative file-pathing. Notably, CircleCI's parallelization assumes file paths in JUnit files are relative to the runner's working directory.

Contributing

Everyone has a right to submit patches to kaocha-junit-xml, and thus become a contributor.

Contributors MUST

adhere to the LambdaIsland Clojure Style Guide
write patches that solve a problem. Start by stating the problem, then supply a minimal solution. *
agree to license their contributions as EPL 1.0.
not break the contract with downstream consumers. **
not break the tests.

Contributors SHOULD

update the CHANGELOG and README.
add tests for new functionality.

If you submit a pull request that adheres to these rules, then it will almost certainly be merged immediately. However some things may require more consideration. If you add new dependencies, or significantly increase the API surface, then we need to decide if these changes are in line with the project's goals. In this case you can start by writing a pitch, and collecting feedback on it.

* This goes for features too, a feature needs to solve a problem. State the problem it solves, then supply a minimal solution.

** As long as this project has not seen a public release (i.e. is not on Clojars) we may still consider making breaking changes, if there is consensus that the changes are justified.

License

Available under the terms of the Eclipse Public License 1.0, see LICENSE.txt