eXray for Jira Server/Data Center has been available in the Atlassian Marketplace since 2014.
Xray for Jira Cloud is an offshoot of the highly successful Xray for Jira Server. While there are many similarities between the two, there are some subtle differences due to the unique nature of Jira Cloud.
These are, in fact, two distinct products with separate roadmaps and release cycles. However, our goal is to make Xray for Jira Cloud as similar to Xray for Jira Server/Data Center as possible, while improving it where we can.
Feature Comparison
Below are the core differences between Xray for Jira Server/Data Center and Xray for Jira Cloud.
Xray Standard | Xray Enterprise * | |||
Feature | Xray for JIRA Server/Data Center (7.2.x) | Xray for Jira Cloud | Xray Enterprise Data Center | Xray Enterprise for Jira Cloud |
Test & Test Type (Manual, Cucumber, Generic) | Xray Custom fields and Test Steps can only be added after Issue creation (edit view). | Xray Custom fields and Test Steps can only be added after Issue creation (edit view). | ||
Parameterized Tests | ||||
Modular Tests | ||||
Test Case Version | ||||
Precondition | ||||
Test Case Designer | ||||
Test Set | Export to Cucumber available in a different menu "..." Ability to create ad-hoc Tests | Export to Cucumber available in a different menu "..." Ability to create ad-hoc Tests | ||
Test Execution | Export to Cucumber available in a different menu "..." At the coverable issue, select Test with a specific test execution status to start a new execution. Ability to create ad-hoc Tests | Export to Cucumber available in a different menu "..." At the coverable issue, select Test with a specific test execution status to start a new execution. Ability to create ad-hoc Tests | ||
Sub-Test Execution | Export to Cucumber available in a different menu "..." Ability to create ad-hoc Tests | Export to Cucumber available in a different menu "..." Ability to create ad-hoc Tests | ||
Test Run | Test Run archive is done by archiving the Test Execution. | Archive options available under the Test Run Table. | Test Run archive is done by archiving the Test Execution. | Archive options available under the Test Run Table |
Test Plan | Interface & UI differences. Ability to create ad-hoc Tests | Interface & UI differences. Ability to create ad-hoc Tests | ||
Dynamic Test Plans | ||||
Cucumber Tests/ Preconditions | ||||
Test Environments | Test Environment Management not available as an Xray feature (controlled by dedicated label custom field where you can apply a label gadget for more control) | Ability to manage Test Environments by project and globally | Test Environment Management not available as an Xray feature (controlled by dedicated label custom field where you can apply a label gadget for more control) | Ability to manage Test Environments by project and globally |
Test Case Importer | ||||
Test Repository | Ability to filter Preconditions and to set views by Test Set There are navigation & UI differences with no impact on features Unable to create or add Test Set / Execution / Plan from selected Tests | Ability to filter Preconditions and to set views by Test Set There are navigation & UI differences with no impact on features Unable to create or add Test Set / Execution / Plan from selected Tests | ||
Test Plan Board | Navigation & UI differences with no impact on features | Navigation & UI differences with no impact on features | ||
Testing Board |
|
| ||
Test Step Custom FIelds | ||||
Test Run Custom Fields | ||||
Automated Step Library / BDD Step Library | ||||
Remote Jobs Trigger | ||||
Reports | Traceability Overall Requirement Coverage Historical Requirement Coverage Tests Plans Report Test Executions Report Test Runs Report Test List Test Sets List Test Executions List Test Plan List Cross-project reports not available on Xray Standard. | Traceability Overall Test Coverage Historical Requirement Coverage Tests Plans Metrics Test Executions Report Test Runs List Test List Test Sets List Test Executions List Test Plan List Cross-project reports available Visualization changes in some reports. ie.: Hierarchical and Flat view in Test Coverage Report. | Traceability Overall Requirement Coverage Historical Requirement Coverage Tests Plans Report Test Executions Report Test Runs Report Test List Test Sets List Test Executions List Test Plan List Cross-project reports available | Traceability Overall Test Coverage Historical Requirement Coverage Tests Plans Metrics Test Executions Report Test Runs List Test List Test Sets List Test Executions List Test Plan List Cross-project reports available Visualization changes in some reports. ie.: Hierarchical and Flat view in Test Coverage Report. |
Gadgets | Overall Requirements Coverage Historical Daily Requirements Coverage Test Runs Summary Tests List Test Runs List Tests Evolution Gadget Overall Test Results Requirement List Test Set List - obtain the same view with Jira native gadget (filter results) Test Execution List - obtain the same view with Jira native gadget (filter results) Test Plan List - obtain the same view with Jira native gadget (filter results) Tests by Test - obtain the same view with Jira native gadget (pie chart) | Overall Test Coverage Historical Daily Requirements Coverage Test Runs Summary Tests List Test Runs List Tests Evolution Gadget Overall Test Results Requirement List Test Set List Test Execution List Test Plan List Test by Test Differences in designation for some gadgets. Also more gadgets available out-of-the-box in Xray Cloud. | Overall Requirements Coverage Historical Daily Requirements Coverage Test Runs Summary Tests List Test Runs List Tests Evolution Gadget Overall Test Results Requirement List Test Set List - obtain the same view with Jira native gadget (filter results) Test Execution List - obtain the same view with Jira native gadget (filter results) Test Plan List - obtain the same view with Jira native gadget (filter results) Tests by Test - obtain the same view with Jira native gadget (pie chart) | Overall Test Coverage Historical Daily Requirements Coverage Test Runs Summary Tests List Test Runs List Tests Evolution Gadet Overall Test Results Requirement List Test Set List Test Execution List Test Plan List Test by Test Differences in designation for some gadgets. Also more gadgets available out-of-the-box in Xray cloud. |
Fields available for Agile Boards | Requirement Status Test Execution Status Test Plan Status | Requirement Status Test Execution Status Test Plan Status (In Xray Cloud, it's not possible to add calculated Xray fields to Agile Boards, due to Jira Cloud limitation; please check the available fields here) | Requirement Status Test Execution Status Test Plan Status | Requirement Status Test Execution Status Test Plan Status (In Xray Cloud, it's not possible to add calculated Xray fields to Agile Boards, due to Jira Cloud limitation; please check the available fields here) |
JQL Functions | JQL Functions are available for Server/DC here | Most of the JQL Functions are not available in Xray Cloud. Check here which ones are available. | JQL Functions are available for Server/DC here | Most of the JQL Functions are not available in Xray Cloud. Check here which ones are available |
Xray Notification (Xray Events) | Details here | Details here | ||
Storage (Attachments) | N/A | 100 GB | N/A | Unlimited |
Integrations | Xporter, eazyBI, Structure, Confluence, Bamboo, Jenkins, TeamCity | Xporter, Jenkins, TeamCity, GitHub, Git Lab, Bitbucket Pipelines, Azure DevOps, Travis CI, Circle CI, Maven, Gradle, Scriptrunner | Xporter, eazyBI, Structure, Confluence, Bamboo, Jenkins, TeamCity | Xporter, Jenkins, TeamCity, GitHub, Git Lab, Bitbucket Pipelines, Azure DevOps, Travis CI, Circle CI, Maven, Gradle, Scriptrunner |
API | Extends Jira's REST API. Able to import multiple reports and manage all entities. | REST API: able to import multiple reports. GraphQL API: manage all entities. API Rate Limit: When moving to Cloud, you may face API rate limits. As Xray uses Jira API in most of their requests, please contact Atlassian Support to understand what is the limit for your instance. | Extends Jira's REST API. Able to import multiple reports and manage all entities. | REST API: able to import multiple reports. GraphQL API: manage all entities. API rate Limit: When moving to Cloud, you may face API rate limits. As Xray uses Jira API in most of their requests, please contact Atlassian Support to understand what is the limit for your instance. |
Automation frameworks/formats supported | Cucumber, Xray, JUnit, NUnit, TestNG, xUnit, Robot framework, Robot framework v4.0, Behave | Cucumber, Xray, JUnit, NUnit, TestNG, xUnit, Robot Framework, Behave | Cucumber, Xray, JUnit, NUnit, TestNG, xUnit, Robot framework, Robot framework v4.0, Behave | Cucumber, Xray, JUnit, NUnit, TestNG, xUnit, Robot Framework, Behave |
Administration & Configuration | Please refer to documentation to understand differences between Xray Server/DC & Cloud Administration and configuration. |
* Xray Enterprise requires an active Xray Standard License.
Understand the icons:
- Available in Xray Cloud but with some differences or minor UI adjustments
- Not available in Xray Cloud
- Extra option or functionality in Xray Cloud