Linux Lighting Group - JUnit XML reporting file format for Jenkins (original) (raw)

The JUnit testing framework has introduced a XML file format to report about the test suite execution. These XML files can be processed by programs like Jenkins with the JUnit plugin to display results of the tests. However there doesn't seem to exist a good description of the XML file format if you want to produce such files with your own program. This webpage provides a sample XML file which describes all valid elements and attributes as used by the Jenkins JUnit Plugin. It can also serve as a template for other programs that process JUnit style XML files like the Maven Surefire Plugin.
If you have additional information on these file formats, contact Dirk Jagdmann via email to <doj@cubic.org>. The files shown on this page are licensed withCC0 1.0 Universal (CC0 1.0) Public Domain Dedication.

sample JUnit XML file for Jenkins

<testsuites disabled="" errors="" failures="" name="" tests="" time="" >

<testsuite name="" tests="" disabled="" errors="" failures="" hostname="" id="" package="" skipped="" time="" timestamp="" >

<!-- Properties (e.g., environment settings) set during test execution.
     The properties element can appear 0 or once. -->
<properties>
  <!-- property can appear multiple times. The name and value attributres are required. -->
  <property name="" value=""/>
</properties>

<!-- testcase can appear multiple times, see /testsuites/testsuite@tests -->
<testcase name=""       <!-- Name of the test method, required. -->
      assertions="" <!-- number of assertions in the test case. optional. not supported by maven surefire. -->
      classname=""  <!-- Full class name for the class the test method is in. required -->
      status=""     <!-- optional. not supported by maven surefire. -->
      time=""       <!-- Time taken (in seconds) to execute the test. optional -->
      >

  <!-- If the test was not executed or failed, you can specify one of the skipped, error or failure elements. -->

  <!-- skipped can appear 0 or once. optional -->
  <skipped message=""   <!-- message/description string why the test case was skipped. optional -->
  />

  <!-- error indicates that the test errored.
       An errored test had an unanticipated problem.
       For example an unchecked throwable (exception), crash or a problem with the implementation of the test.
       Contains as a text node relevant data for the error, for example a stack trace. optional -->
  <error message="" <!-- The error message. e.g., if a java exception is thrown, the return value of getMessage() -->
     type=""    <!-- The type of error that occured. e.g., if a java execption is thrown the full class name of the exception. -->
     >error description</error>

  <!-- failure indicates that the test failed.
       A failure is a condition which the code has explicitly failed by using the mechanisms for that purpose.
       For example via an assertEquals.
       Contains as a text node relevant data for the failure, e.g., a stack trace. optional -->
  <failure message="" <!-- The message specified in the assert. -->
       type=""    <!-- The type of the assert. -->
       >failure description</failure>

  <!-- Data that was written to standard out while the test was executed. optional -->
  <system-out>STDOUT text</system-out>

  <!-- Data that was written to standard error while the test was executed. optional -->
  <system-err>STDERR text</system-err>
</testcase>

<!-- Data that was written to standard out while the test suite was executed. optional -->
<system-out>STDOUT text</system-out>
<!-- Data that was written to standard error while the test suite was executed. optional -->
<system-err>STDERR text</system-err>

JUnit XSD file for Jenkins

The following XML Schema Definition is supposed to validate the JUnit format from the previous section.

<xs:schema xmlns:xs="" title="undefined" rel="noopener noreferrer">http://www.w3.org/2001/XMLSchema"> <xs:simpleType name="SUREFIRE_TIME"> <xs:restriction base="xs:string"> <xs:pattern value="(([0-9]{0,3},)*[0-9]{3}|[0-9]{0,3})*(.[0-9]{0,3})?"/>

<xs:complexType name="rerunType" mixed="true"> <!-- mixed (XML contains text) to be compatible with version previous than 2.22.1 -->
    <xs:sequence>
        <xs:element name="stackTrace" type="xs:string" minOccurs="0" /> <!-- optional to be compatible with version previous than 2.22.1 -->
        <xs:element name="system-out" type="xs:string" minOccurs="0" />
        <xs:element name="system-err" type="xs:string" minOccurs="0" />
    </xs:sequence>
    <xs:attribute name="message" type="xs:string" />
    <xs:attribute name="type" type="xs:string" use="required" />
</xs:complexType>

<xs:element name="failure">
    <xs:complexType mixed="true">
        <xs:attribute name="type" type="xs:string"/>
        <xs:attribute name="message" type="xs:string"/>
    </xs:complexType>
</xs:element>

<xs:element name="error">
    <xs:complexType mixed="true">
        <xs:attribute name="type" type="xs:string"/>
        <xs:attribute name="message" type="xs:string"/>
    </xs:complexType>
</xs:element>

<xs:element name="skipped">
    <xs:complexType mixed="true">
        <xs:attribute name="type" type="xs:string"/>
        <xs:attribute name="message" type="xs:string"/>
    </xs:complexType>
</xs:element>

<xs:element name="properties">
    <xs:complexType>
        <xs:sequence>
            <xs:element ref="property" minOccurs="0" maxOccurs="unbounded"/>
        </xs:sequence>
    </xs:complexType>
</xs:element>

<xs:element name="property">
    <xs:complexType>
        <xs:attribute name="name" type="xs:string" use="required"/>
        <xs:attribute name="value" type="xs:string" use="required"/>
    </xs:complexType>
</xs:element>

<xs:element name="system-err" type="xs:string"/>
<xs:element name="system-out" type="xs:string"/>
<xs:element name="rerunFailure" type="rerunType"/>
<xs:element name="rerunError" type="rerunType"/>
<xs:element name="flakyFailure" type="rerunType"/>
<xs:element name="flakyError" type="rerunType"/>

<xs:element name="testcase">
    <xs:complexType>
        <xs:sequence>
            <xs:choice minOccurs="0" maxOccurs="unbounded">
                <xs:element ref="skipped"/>
                <xs:element ref="error"/>
                <xs:element ref="failure"/>
                <xs:element ref="rerunFailure" minOccurs="0" maxOccurs="unbounded"/>
                <xs:element ref="rerunError" minOccurs="0" maxOccurs="unbounded"/>
                <xs:element ref="flakyFailure" minOccurs="0" maxOccurs="unbounded"/>
                <xs:element ref="flakyError" minOccurs="0" maxOccurs="unbounded"/>
                <xs:element ref="system-out"/>
                <xs:element ref="system-err"/>
            </xs:choice>
        </xs:sequence>
        <xs:attribute name="name" type="xs:string" use="required"/>
        <xs:attribute name="time" type="xs:string"/>
        <xs:attribute name="classname" type="xs:string"/>
        <xs:attribute name="group" type="xs:string"/>
    </xs:complexType>
</xs:element>

<xs:element name="testsuite">
    <xs:complexType>
        <xs:choice minOccurs="0" maxOccurs="unbounded">
            <xs:element ref="testsuite"/>
            <xs:element ref="properties"/>
            <xs:element ref="testcase"/>
            <xs:element ref="system-out"/>
            <xs:element ref="system-err"/>
        </xs:choice>
        <xs:attribute name="name" type="xs:string" use="required"/>
        <xs:attribute name="tests" type="xs:string" use="required"/>
        <xs:attribute name="failures" type="xs:string" use="required"/>
        <xs:attribute name="errors" type="xs:string" use="required"/>
        <xs:attribute name="group" type="xs:string" />
        <xs:attribute name="time" type="SUREFIRE_TIME"/>
        <xs:attribute name="skipped" type="xs:string" />
        <xs:attribute name="timestamp" type="xs:string" />
        <xs:attribute name="hostname" type="xs:string" />
        <xs:attribute name="id" type="xs:string" />
        <xs:attribute name="package" type="xs:string" />
        <xs:attribute name="file" type="xs:string"/>
        <xs:attribute name="log" type="xs:string"/>
        <xs:attribute name="url" type="xs:string"/>
        <xs:attribute name="version" type="xs:string"/>
    </xs:complexType>
</xs:element>

<xs:element name="testsuites">
    <xs:complexType>
        <xs:sequence>
            <xs:element ref="testsuite" minOccurs="0" maxOccurs="unbounded" />
        </xs:sequence>
        <xs:attribute name="name" type="xs:string" />
        <xs:attribute name="time" type="SUREFIRE_TIME"/>
        <xs:attribute name="tests" type="xs:string" />
        <xs:attribute name="failures" type="xs:string" />
        <xs:attribute name="errors" type="xs:string" />
    </xs:complexType>
</xs:element>

An alternativeXSD is used by themaven surefire plugin. This software supports additional XML elements not shown above like , , , , .

Another alternative XSD is the github JUnit-Schema repository with a XSD for Apache Ant's JUnit output.

XSLT transformations to JUnit XML format

The Jenkins source code contains several XSLT files to convert the result of other test suites (aunit, boost, cpptest, ppunit, ctest, embunit, fpcunit, gtester, mbunit, mstest, nunit, phpunit, qtestlib, tusar, unittest, valgrind, xunitdotnet) into JUnit XML format, see jenkins xunit plugin source code.