Breadcrumbs

Page History

Versions Compared

Key

This line was added.
This line was removed.
Formatting was changed.

...

Importing JUnit XML reports

Xray supports Junit XML imports and you read more here.

Below is a simplified example of a JUnit XML report containing a Test Suite with one Test Case.

...

Condition	Overall status of the Test Run
If all the mapped results of the Test Case was PASS	PASS
If any of the mapped results of the Test Case was FAIL	FAIL
Other cases	TODO

Notes and Limitations

attachments (e.g. screenshots and other files) are not supported/imported as they are not embedded in the XML report; it seems to be possible to add references to their local paths in the <system-out/> element but these cannot be imported as they are external to the report

Xray extended JUnit format

Test issue id/key

Two scenarios are supported to specify an existing test to import the JUnit test case to:

A test issue id is passed as a test_id property on the testcase element.
A test issue key is passed as a test_key property on the testcase element.

If both properties exist in a testcase, the test_id will be used. If the given test issue id or key does not exist, an error will be thrown.

Code Block

language	xml

<?xml version="1.0" ?>
<testsuites>
    <testsuite errors="0" failures="0" id="0" name="my test suite" tests="1">
        <!-- no test explicitly specified, will try to be matched using the default generic test definition -->
        <testcase classname="some.class.name" name="Test1" requirement="CALC-123" time="123.345000"/>
 
        <testcase classname="some.class.name" name="Test2" time="123.345000">
            <properties>
                <!-- using a custom "test_id" property -->
                <property name="test_id" value="10001" />
            </properties>
        </testcase>
 
        <testcase classname="some.class.name" name="Test3" time="123.345000">
            <properties>
                <!-- using a custom "test_key" property -->
                <property name="test_key" value="CALC-123" />
            </properties>
        </testcase>
    </testsuite>
</testsuites>

Test requirements

Three scenarios are supported to link a test with requirements:

A requirement key is passed as a requirement attribute on the testcase element.
A requirement key is passed on a requirement property element inside the testcase element.
Multiple requirement keys are passed on a requirements property element inside the testcase element, separated by "," (comma).

Code Block

language	xml

<?xml version="1.0" ?>
<testsuites>
    <testsuite errors="0" failures="0" id="0" name="my test suite" tests="1">
        <!-- using a custom "requirement" attribute -->
        <testcase classname="some.class.name" name="Test1" requirement="CALC-123" time="123.345000"/>
 
        <testcase classname="some.class.name" name="Test2" time="123.345000">
            <properties>
                <!-- using a custom "requirement" property -->
                <property name="requirement" value="CALC-123" />
            </properties>
        </testcase>
 
        <testcase classname="some.class.name" name="Test3" time="123.345000">
            <properties>
                <!-- using a custom "requirements" property -->
                <property name="requirements" value="CALC-123,CALC-456" />
            </properties>
        </testcase>
    </testsuite>
</testsuites>

Test summary

Use a test_summary property element inside the testcase element to explicitly set the issue summary. This summary will be used both to create or update the test.

If importing to a new test and the summary is not explicitly defined, it will default to the name attribute of the testcase.

Code Block

language	xml

<?xml version="1.0" ?>
<testsuites>
    <testsuite errors="0" failures="0" id="0" name="my test suite" tests="1">
        <!-- if a new test, summary will default to "Test1" -->
        <!-- if an existing test, summary will stay as is -->
        <testcase classname="some.class.name" name="Test1" time="123.345000"/>
 
        <testcase classname="some.class.name" name="Test2" time="123.345000">
            <properties>
                <!-- if a new test, summary will be "A custom summary" -->
                <!-- if an existing test, summary will be updated to "A custom summary" -->
                <property name="test_summary" value="A custom summary" />
            </properties>
        </testcase>
    </testsuite>
</testsuites>

Test description

Use a test_description property element inside the testcase element to set the issue description. This description will be used both to create or update the test.

Code Block

language	xml

<?xml version="1.0" ?>
<testsuites>
    <testsuite errors="0" failures="0" id="0" name="my test suite" tests="1">
        <testcase classname="some.class.name" name="Test1" time="123.345000">
            <properties>
                <property name="test_description">
                    <![CDATA[Some custom description for the test issue.]]>
                </property>
            </properties>
        </testcase>
    </testsuite>
</testsuites>

Test labels

Use a tags property element inside the testcase element to add labels to the issue. Multiple labels must be separated by "," (comma).

Code Block

language	xml

<?xml version="1.0" ?>
<testsuites>
    <testsuite errors="0" failures="0" id="0" name="my test suite" tests="1">
        <testcase classname="some.class.name" name="Test1" time="123.345000">
            <properties>
                <property name="tags" value="label1,label2" />
            </properties>
        </testcase>
    </testsuite>
</testsuites>

Test run comment

Use a testrun_commentproperty element inside the testcase element to set the overall comment of the test run.

Code Block

language	xml

<?xml version="1.0" ?>
<testsuites>
    <testsuite errors="0" failures="0" id="0" name="my test suite" tests="1">
        <testcase classname="some.class.name" name="Test1" time="123.345000">
            <property name="testrun_comment">
                <![CDATA[Lorem ipsum dolor sit amet,
consectetur adipiscing elit,
sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.]]>
            </property>
        </testcase>
    </testsuite>
</testsuites>

Test run evidence

Use a testrun_evidence property element inside the testcase element to add files as global evidence on the test run. Each evidence must be an item element inside the property, with the filename in the name attribute and with Base64 encoded content.

Code Block

language	xml

<?xml version="1.0" ?>
<testsuites>
    <testsuite errors="0" failures="0" id="0" name="my test suite" tests="1">
        <testcase classname="some.class.name" name="Test1" time="123.345000">
            <property name="testrun_evidence">
                <item name="image1.png">iVBORw0KGgoAAAANSUhEUgAAADIAAAAyCAYAAAAeP4ixAAAAQ0lEQVR42u3PQREAAAgDINc/9Mzg14MGZNrOAxERERERERERERERERERERERERERERERERERERERERERERERERERuVh9TJWdbWjzsQAAAABJRU5ErkJggg==</item>
                <item name="image2.png">iVBORw0KGgoAAAANSUhEUgAAADIAAAAyCAYAAAAeP4ixAAAAQ0lEQVR42u3PQREAAAgDINc/9Mzg14MGZNrOAxERERERERERERERERERERERERERERERERERERERERERERERERERuVh9TJWdbWjzsQAAAABJRU5ErkJggg==</item>
            </property>
        </testcase>
    </testsuite>
</testsuites>

Test run custom fields

Two scenarios are supported to set test run custom fields:

Each custom field in a testrun_customfield property element inside the testcase element. The name of the custom field must appear after the "testrun_customfield:" prefix in the name attribute and the value should be in the value attribute.
Multiple custom fields in a testrun_customfields property element inside the testcase element. Each custom field should be an item element inside the property, with the custom field name in the name attribute and value in the element content.

In both scenarios, multiple select custom fields should have their values separated by ";" (semicolon).

Code Block

language	xml

<?xml version="1.0" ?>
<testsuites>
    <testsuite errors="0" failures="0" id="0" name="my test suite" tests="1">
        <testcase classname="some.class.name" name="Test1" time="123.345000">
            <property name="testrun_customfield:cf1" value="lorem ipsum" />
            <property name="testrun_customfield:cf2" value="option 1;option 2" /> <!-- multi select must have values separated by ";" -->
        </testcase>
 
        <testcase classname="some.class.name" name="Test2" time="123.345000">
            <properties>
                <property name="testrun_customfields">
                    <item name="cf1">
                        <![CDATA[lorem ipsum]]>
                    </item>
                    <item name="cf2">
                        <![CDATA[option 1;option 2]]> <!-- multi select must have values separated by ";" -->
                    </item>
                </property>
            </properties>
        </testcase>
    </testsuite>
</testsuites>

References

Content

Space Tools

Versions Compared

Old Version 2

New Version 20

Key

Importing JUnit XML reports

Notes and Limitations

Xray extended JUnit format

Test issue id/key

Test requirements

Test summary

Test description

Test labels

Test run comment

Test run evidence

Test run custom fields