Page History
...
Another way of integrating with testing frameworks is by using the NUnit test runner's XML report format. The NUnit report format has more features than JUnit 4.0 XML report, which allows Xray to do things such as automatic linking to issues (e.g., requirements) or assignment of labels to the newly created Test entities.
Besides this, Xray also provides specific integrations for Cucumber, Behave, Robot, Xamarin, and other frameworks.
| Gliffy Diagram | ||||
|---|---|---|---|---|
|
Summary of features per framework
...
Robot framework | JUnit 4 | TestNG | NUnit 2.6/3.x | Cucumber | Xray JSON | |
|---|---|---|---|---|---|---|
Abstract automated test as a Test (map an automated test to a Test issue in Jira) | Yes (as a Generic Test) | Yes (as a Generic Test) | Yes (as a Generic Test) | Yes (as a Generic Test) | Yes (as a Cucumber Test) | No Tests must exist beforehand |
Make Test specification in Jira (specify the Test itself in Jira) | No | No | No | No | Yes | Yes |
Create Tests from results (create Tests whenever importing results) | Yes | Yes | Yes | Yes | No* (but there is and endpoint for importing Cucumber features, which will create Tests for the respective Scenarios/Scenario Outlines) | No |
Uniquely identify Tests (identify existing Tests whenever importing results, avoiding duplication of Test issues) | Yes | Yes | Yes | Yes | Yes | Yes* (based on the provided Test issue keys) |
Import results (importing results by REST API or UI) | Yes | Yes | Yes | Yes | Yes | Yes |
Import “labels” (create labels in the Test issues) | Yes (labels may be specified in the test's source code) | No | Yes (labels may be specified in the test's source code) | Yes (labels may be specified in the test's source code) | Yes* (this is only available when using the endpoint for importing Cucumber features; it's not possible when importing results) | No |
Automatic linking to requirements (create links to requirements) | Yes (requirement's issue key may be specified in the test's source code) | No | Yes (requirement's issue key may be specified in the test's source code) | Yes (requirement's issue key may be specified in the test's source code) | Yes* (this is only available when using the endpoint for importing Cucumber features; it's not possible when importing results) | No |
Semantic on the results (present execution details in the execution screen) | Yes (keyword “steps”) | - | Yes (for every "test" section in the XML config file and parameterized Tests) | Yes (Test Suites and parameterized Tests) | Yes (steps for Scenario/Scenario Outline and Background) | Yes* (semantic is implicit, because the Test must be created beforehand) |
Integration by platform
| Content by Label | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
|
| Content by Label | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
|
| Content by Label | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
|
| Content by Label | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
|
| Content by Label | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
|
) |
| Content by Label | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
|
| Content by Label | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
|
Integration by testing framework report
...
TestNG
...
...
...
| Content by Label | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
...
| Content by Label | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
...
| Content by Label | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
...
| Content by Label | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
Testing by target environment
| Content by Label | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
|
| Content by Label | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
|