Integration with testing frameworks is achieved by processing the results contained in the reports test runners (e.g., in XML, JSON). The results are mapped to the appropriate Test issues; if they don't exist, then they will be created. This flow is explained in Using Generic Tests for Automation. Xray also supports Cucumber tests natively.
Besides Cucumber, there are currently many testing frameworks available for every language you may think of.
Summary of features per framework
The following table presents the available features when importing automated test results.
The Xray JSON format is more generic and its capabilities, if used for importing, are different.
| Cucumber | Xray JSON | |
|---|---|---|
Abstract automated test as a Test (map an automated test to a Test issue in Jira) | Yes (as a Cucumber Test) | No Tests must exist beforehand |
Make Test specification in Jira (specify the Test itself in Jira) | Yes | Yes |
Create Tests from results (create Tests whenever importing results) | No* (but there is an endpoint for importing Cucumber features, which will create Tests for the respective Scenarios/Scenario Outlines) | No |
Uniquely identify Tests (identify existing Tests whenever importing results, avoiding duplication of Test issues) | Yes | Yes* (based on the provided Test issue keys) |
Import results (importing results by REST API or UI) | Yes | Yes |
Import “labels” (create labels in the Test issues) | Yes* (this is only available when using the endpoint for importing Cucumber features; it's not possible when importing results) | No |
Automatic linking to requirements (create links to requirements) | Yes* (this is only available when using the endpoint for importing Cucumber features; it's not possible when importing results) | No |
Semantics on the results (present execution details in the execution screen) | Yes (steps for Scenario/Scenario Outline and Background) | Yes* (semantics is implicit because the Test must be created beforehand) |
Integration by platform
|
|
|
|---|---|---|
Integration by testing framework report
 |
|---|