Page History
...
The simplified tags hierarchy of these reports can be represented in the following diagram:
Entities
JUnit’s Test Cases are identified by the pair of attributes “classname” and “name” attributes.
Test Cases JUnit test cases are imported to Xray’s Generic Test test issues, and the “classname” classname
and “name” name
attributes are concatenated and mapped to the Generic Test Definition field of the Generic Test.test.
Test cases can be imported to an already existing test by explicitly defining the test issue id or key or by a matching generic test definition (if a test If a Test already exists with the same Generic Test Definitiongeneric test definition, then a duplicate is will not be created).
If no test issue id or key is specified and no test exists with the same generic test definition, then a new test will be created.
Test cases Test Cases are imported to a new (or user-specified) Test Execution in the context of some project, along with their respective execution results.
...
Condition | Overall status of the Test Run |
---|---|
If all the mapped results of the Test Case were PASS | PASS |
If any of the mapped results of the Test Case was FAIL | FAIL |
Other cases | TODO |
...
Xray extended JUnit format
Anchor | ||||
---|---|---|---|---|
|
Although this is not yet included by JUnit, Xray supports a customized version of the JUnit report format for linking Requirement issues with Test cases. that uses custom properties/attributes to import Xray related information.
Test issue id/key
Anchor | ||||
---|---|---|---|---|
|
Two scenarios are supported to specify an existing test to import the JUnit test case to:
- A test issue id is passed as a
test_id
property on thetestcase
element. - A test issue The requirement key is passed as an attribute on the "testcase" XML element named "requirement"
- The requirement key is passed as a new property element beneath the "testcase" element, on a dedicated properties element.
- a
test_key
property on thetestcase
element.
If both properties exist in a testcase
, the test_id
will be used. If the given test issue id or key does not exist, an error will be thrown.
Code Block | ||
---|---|---|
| ||
<?xml version="1.0" ?>
<testsuites>
<testsuite errors="0" failures="0" id="0" name="my test suite" tests="1">
<!-- no test explicitly specified, will try to be matched using the default generic test definition -->
<testcase classname="some.class.name" name="Test1" requirement="CALC-123" time="123.345000"/>
<testcase classname="some.class.name" name="Test2" time="123.345000">
<properties>
<!-- using a custom "test_id" property -->
<property name="test_id" value="10001" />
</properties>
</testcase>
<testcase classname="some.class.name" name="Test3" time="123.345000">
<properties>
<!-- using a custom "test_key" property -->
<property name="test_key" value="CALC-123" />
</properties>
</testcase>
</testsuite>
</testsuites> |
Test requirements
Three scenarios are supported to link a test with requirements:
- A requirement key is passed as a
requirement
attribute on thetestcase
element. - A requirement key is passed on a
requirement
property element inside thetestcase
element. - Multiple requirement keys are passed on a
requirements
property element inside thetestcase
element, separated by "," (comma).
Code Block | ||
---|---|---|
| ||
<?xml version="1.0" ?> <testsuites> <testsuite errors="0" failures="0" id="0" name="my test suite" tests="1"> <!-- using a custom "requirement" attribute --> <testcase classname="some.class.name" name="Test1" requirement="CALC-123" time="123.345000"/> <testcase classname="some.class.name" name="Test2" time="123.345000"> <properties> <!-- using a custom "requirement" property --> <property name="requirement" value="CALC-123" /> </properties> </testcase> <testcase classname="some.class.name" name="Test3" time="123.345000"> <properties> <!-- using a custom "requirements" property --> <property name="requirements" value="CALC-123,CALC-456" /> </properties> </testcase> </testsuite> </testsuites> |
Test summary
Use a test_summary
property element inside the testcase
element to explicitly set the issue summary. This summary will be used both to create or update the test.
If importing to a new test and the summary is not explicitly defined, it will default to the name
attribute of the testcase
.
Code Block | ||
---|---|---|
| ||
<?xml version="1.0" ?> <testsuites> <testsuite errors="0" failures="0" id="0" name="my test suite" tests="1"> <!-- if a new test, summary will default to "Test1" --> <!-- if an existing test, summary will stay as is --> <testcase classname="some.class.name" name="Test1" time="123.345000"/> <testcase classname="some.class.name" name="Test2" time="123.345000"> <properties> <!-- if a new test, summary will be "A custom summary" --> <!-- if an existing test, summary will be updated to "A custom summary" --> <property name="test_summary" value="A custom summary" /> </properties> </testcase> </testsuite> </testsuites> |
Test description
Use a test_description
property element inside the testcase element to set the issue description. This description will be used both to create or update the test.
Code Block | ||
---|---|---|
| ||
<?xml version="1.0" ?> <testsuites> <testsuite errors="0" failures="0" id="0" name="my test suite" tests="1"> <testcase classname="some.class.name" name="Test1" time="123.345000"> <properties> <property name="test_description"> <![CDATA[Some custom description for the test issue.]]> </property> </properties> </testcase> </testsuite> </testsuites> |
Test labels
Use a tags
property element inside the testcase
element to add labels to the issue. Multiple labels must be separated by "," (comma).
Code Block | ||
---|---|---|
| ||
<?xml version="1.0" ?>
<testsuites>
<testsuite errors="0" failures="0" id="0" name="my test suite" tests="1">
<testcase classname="some.class.name" name="Test1" time="123.345000">
<properties>
<property name="tags" value="label1,label2" />
</properties>
</testcase>
</testsuite>
</testsuites> |
Test run comment
Use a testrun_comment
property element inside the testcase
element to set the overall comment of the test run.
Code Block | ||
---|---|---|
| ||
<?xml version="1.0" ?>
<testsuites>
<testsuite errors="0" failures="0" id="0" name="my test suite" tests="1">
<testcase classname="some.class.name" name="Test1" time="123.345000">
<property name="testrun_comment">
<![CDATA[Lorem ipsum dolor sit amet,
consectetur adipiscing elit,
sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.]]>
</property>
</testcase>
</testsuite>
</testsuites> |
Test run evidence
Use a testrun_evidence
property element inside the testcase
element to add files as global evidence on the test run. Each evidence must be an item
element inside the property, with the filename in the name
attribute and with Base64 encoded content.
Code Block | ||
---|---|---|
| ||
<?xml version="1.0" ?> <testsuites> <testsuite errors="0" failures="0" id="0" name="my test suite" tests="1"> <testcase classname="some.class.name" name="Test1" time="123.345000"> <property name="testrun_evidence"> <item name="image1.png">iVBORw0KGgoAAAANSUhEUgAAADIAAAAyCAYAAAAeP4ixAAAAQ0lEQVR42u3PQREAAAgDINc/9Mzg14MGZNrOAxERERERERERERERERERERERERERERERERERERERERERERERERERuVh9TJWdbWjzsQAAAABJRU5ErkJggg==</item> <item name="image2.png">iVBORw0KGgoAAAANSUhEUgAAADIAAAAyCAYAAAAeP4ixAAAAQ0lEQVR42u3PQREAAAgDINc/9Mzg14MGZNrOAxERERERERERERERERERERERERERERERERERERERERERERERERERuVh9TJWdbWjzsQAAAABJRU5ErkJggg==</item> </property> </testcase> </testsuite> </testsuites> |
Test run custom fields
Two scenarios are supported to set test run custom fields:
- Each custom field in a
testrun_customfield
property element inside thetestcase
element. The name of the custom field must appear after the "testrun_customfield:" prefix in thename
attribute and the value should be in thevalue
attribute. - Multiple custom fields in a
testrun_customfields
property element inside thetestcase
element. Each custom field should be anitem
element inside the property, with the custom field name in thename
attribute and value in the element content.
In both scenarios, multiple select custom fields should have their values separated by ";" (semicolon).
Code Block | ||
---|---|---|
| ||
<?xml version="1.0" ?> <testsuites> <testsuite errors="0" failures="0" id="0" name="my test suite" tests="1"> <testcase classname="some.class.name" name="Test1" time="123.345000"> <property name="testrun_customfield:cf1" value="lorem ipsum" /> <property name="testrun_customfield:cf2" value="option 1;option 2" /> <!-- multi select must have values separated by ";" --> </testcase> <testcase classname="some.class.name" name="Test2" time="123.345000"> <properties> <property name="testrun_customfields"> <item name="cf1"> <![CDATA[lorem ipsum]]> </item> <item name="cf2"> <![CDATA[option 1;option 2]]> <!-- multi select must have values separated by ";" --> </item> </property>="requirement" value="CALC-123" /> </properties> </testcase> </testsuite> </testsuites> |
...