Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
UI Expand
titleTable of Contents

Table of Contents

Introduction

Xray for Jira Server/Data Center has been available in the Atlassian Marketplace since 2014.

Xray for Jira Cloud is an offshoot of the highly successful Xray for Jira Server. While there are many similarities between the two, there are some subtle differences in their features due to the unique nature of Jira Cloud.

These are, in fact, two distinct products with separate roadmaps and release cycles. However, our goal is to make Xray for Jira Cloud as similar to Xray for Jira Server/Data Center as possible, while improving it where we can.

Message Box
titleMigrating from Xray Server/DC and Xray Cloud
typegeneric

If you are looking to migrate your Jira Server/DC instance to Jira Cloud, please take a look at our Jira Cloud Migration Guide.

Feature Comparison

Below are the core differences between Xray for Jira Server/Data Center and Xray for Jira Cloud.

Info
titleUnderstanding the Icons

warning cloud - Available in Xray Cloud but with some differences or minor UI adjustments

(error) cloud - Not available in Xray Cloud

plus sign cloud - Extra option or functionality in Xray Cloud



Xray StandardXray Enterprise *
Feature

Xray for JIRA Server/Data Center

(7.2.x)

Xray for Jira Cloud

Xray Enterprise Data Center Xray Enterprise for Jira Cloud
Test & Test Type
(Manual, Cucumber, Generic)

Image Modified



Image Modified

warning cloud Xray Custom fields and Test Steps can only be added after Issue creation (edit view).
(error) cloud Unable to filter by labels on predefined

steps

Steps.

Image Modified

Image Modified


warning cloud Xray Custom fields and Test Steps can only be added after Issue creation (edit view).
(error) cloud Unable to filter by labels on predefined

steps

Steps.

Parameterized Tests

Image Modified

Image Modified

Image Modified

Image Modified

Modular Tests

Image Modified

Image Modified

Image Modified

Image Modified

Test Case Versioncross markcross mark

Image Modified

Image Modified

Precondition

Image Modified

Image Modified

Image Modified

Image Modified

Test Case Designercross markcross mark

Image Modified

Image Modified

Test Set

Image Modified

Image Modified

warning cloud Export to Cucumber available in a different menu "..."

plus sign  cloud Ability to create ad-hoc Tests

Image Modified

Image Modified

warning cloud Export to Cucumber available in a different menu "..."

plus sign  cloud Ability to create ad-hoc Tests

Test Execution

Image Modified


Image Modified

warning cloud Export to Cucumber available in a different menu "..."

(error) cloud At the coverable

issue

Issue, select Test with a specific

test execution

Test Execution status to start a new execution.

plus sign  cloud Ability to create ad-hoc Tests

Image Modified

Image Modified

warning cloud Export to Cucumber available in a different menu "..."

(error) cloud At the coverable

issue

Issue, select Test with a specific

test execution

Test Execution status to start a new execution.

plus sign  cloud Ability to create ad-hoc Tests

Sub-Test Execution

Image Modified

Image Modified

warning cloud Export to Cucumber available in a different menu "..."

plus sign  cloud Ability to create ad-hoc Tests

Image Modified

Image Modified

warning cloud Export to Cucumber available in a different menu "..."

plus sign  cloud Ability to create ad-hoc Tests

Test Run

Image Modified

warning  Test Run archive is done by archiving the Test Execution

.

Image Modified

warning cloud Archive options available under the Test Run Table

.

Image Modified

warning  Test Run archive is done by archiving the Test Execution

.

Image Modified

warning cloud Archive options available under the Test Run Table

Test Plan

Image Modified

Image Modified

warning cloud Interface & UI differences

.

plus sign  cloud Ability to create ad-hoc Tests

Image Modified

Image Modified

warning cloud Interface & UI differences

.

plus sign  cloud Ability to create ad-hoc Tests

Dynamic Test Planscross markcross mark

Image Modified

Image Modified

Cucumber Tests/ Preconditions

Image Modified

Image Modified

Image Modified

Image Modified

Test Environments

Image Modified

  warning  Test Environment Management not available as an Xray feature (controlled by dedicated label custom field where you can apply a label gadget for more control)

Image Modified

(tick) cloud Ability to manage Test Environments by project and globally

Image Modified

warning  Test Environment Management not available as an Xray feature (controlled by dedicated label custom field where you can apply a label gadget for more control)

Image Modified

(tick) cloud Ability to manage Test Environments by project and globally

Test Case Importer

Image Modified

Image Modified

Image Modified

Image Modified

Test Repository

Image Modified

Image Modified

plus sign  cloud Ability to filter Preconditions and to set views by Test Set

warning cloud There are navigation & UI differences with no impact on features

(error) cloud Unable to create or add Test Set / Execution / Plan from selected Tests

Image Modified

Image Modified

plus sign  cloud Ability to filter Preconditions and to set views by Test Set

warning cloud There are navigation & UI differences with no impact on features

(error) cloud Unable to create or add Test Set / Execution / Plan from selected Tests 

Test Plan Board

Image Modified

Image Modified

warning cloud Navigation & UI differences with no impact on features

Image Modified

Image Modified

warning cloud Navigation & UI differences with no impact on features

Testing Board

cross mark 

Image Modified


cross mark 

Image Modified

Test Step Custom FIelds

Image Modified

Image Modified

Image Modified

Image Modified

Test Run Custom Fields

Image Modified

Image Modified

Image Modified

Image Modified

Automated Step Library / BDD Step Library

Image Modified

Image Modified

Image Modified

Image Modified

Remote Jobs Triggercross mark cross mark

Image Modified

Image Modified

Reports

(tick) Traceability

(tick) Overall Requirement Coverage 

(tick) Historical Requirement Coverage 

(tick) Tests Plans Report

(tick) Test Executions Report

(tick) Test Runs Report

(tick) Test List

(tick) Test Sets List

(tick) Test Executions List

(tick) Test Plan List

(error) Cross-project reports not available on Xray Standard

.

(tick) Traceability

(tick) Overall Test Coverage

(error)  Historical Requirement Coverage 

(tick) Tests Plans Metrics

(error) Test Executions Report

(tick) Test Runs List

(tick) Test List

(tick) Test Sets List

(tick) Test Executions List

(tick) Test Plan List

(tick) Cross-project reports available 

warning cloud Visualization changes in some reports. ie.: Hierarchical and Flat view in Test Coverage Report

.

(tick) Traceability

(tick) Overall Requirement Coverage 

(tick) Historical Requirement Coverage 

(tick) Tests Plans Report

(tick) Test Executions Report

(tick) Test Runs Report

(tick) Test List

(tick) Test Sets List

(tick) Test Executions List

(tick) Test Plan List

(tick) Cross-project reports available 

(tick) Traceability

(tick) Overall Test Coverage

(error)  Historical Requirement Coverage 

(tick) Tests Plans Metrics

(error) Test Executions Report

(tick) Test Runs List

(tick) Test List

(tick) Test Sets List

(tick) Test Executions List

(tick) Test Plan List

(tick) Cross-project reports available 

warning cloud Visualization changes in some reports. ie.: Hierarchical and Flat view in Test Coverage Report

.

Gadgets

(tick) Overall Requirements Coverage 

(tick) Historical Daily Requirements Coverage 

(tick) Test Runs Summary 

(tick) Tests List

(tick) Test Runs List

(tick) Tests Evolution Gadget

(tick) Overall Test Results

(tick) Requirement List

warning Test Set List - obtain the same view with Jira native gadget (filter results)

warning Test Execution List - obtain the same view with Jira native gadget (filter results)

warning Test Plan List - obtain the same view with Jira native gadget (filter results)

warning Tests by Test - obtain the same view with Jira native gadget (pie chart)

(tick) Overall Test Coverage 

(error) Historical Daily Requirements Coverage 

(tick) Test Runs Summary

(tick) Tests List

(tick) Test Runs List

(error)

(tick)  Tests Evolution Gadget

(tick) Overall Test Results

(tick) Requirement List

(tick) Test Set List 

(tick) Test Execution List 

(tick) Test Plan List 

(tick) Test by Test 

warning cloud Differences in designation for some gadgets. Also more gadgets available out-of-the-box in Xray Cloud.


(tick) Overall Requirements Coverage 

(tick) Historical Daily Requirements Coverage 

(tick) Test Runs Summary 

(tick) Tests List

(tick) Test Runs List

(tick) Tests Evolution Gadget

(tick) Overall Test Results

(tick) Requirement List

warning Test Set List - obtain the same view with Jira native gadget (filter results)

warning Test Execution List - obtain the same view with Jira native gadget (filter results)

warning Test Plan List - obtain the same view with Jira native gadget (filter results)

warning Tests by Test - obtain the same view with Jira native gadget (pie chart)

(tick) Overall Test Coverage 

(error) Historical Daily Requirements Coverage 

(tick) Test Runs Summary

(tick) Tests List

(tick) Test Runs List

(error)

(tick)  Tests Evolution

Gadet

Gadget

(tick) Overall Test Results

(tick) Requirement List

(tick) Test Set List 

(tick) Test Execution List 

(tick) Test Plan List 

(tick) Test by Test 

warning cloud Differences in designation for some gadgets. Also more gadgets available out-of-the-box in Xray

cloud.

Cloud

Fields available for Agile Boards

(tick) Requirement Status

(tick) Test Execution Status

(tick) Test Plan Status

(error) Requirement Status

(error) Test Execution Status

(error) Test Plan Status

(In Xray Cloud, it's not possible to add calculated Xray fields to Agile Boards, due to Jira Cloud limitation; please check the available fields here)

(tick) Requirement Status

(tick) Test Execution Status

(tick) Test Plan Status

(error) Requirement Status

(error) Test Execution Status

(error) Test Plan Status

(In Xray Cloud, it's not possible to add calculated Xray fields to Agile Boards, due to Jira Cloud limitation; please check the available fields here)

JQL FunctionsJQL Functions are available for Server/
DC 
DC here
Most

All of the JQL

Functions

functions are

not

available in Xray Cloud, with one exception. Check here which ones are available.

JQL Functions are available for Server/DC here
Most
All of the JQL
Functions
functions are
not
available in Xray Cloud, with one exception. Check here which ones are available
Xray Notification (Xray Events)

Image Modified

Details here

cross mark

Image Modified

Details here

cross mark
Storage (Attachments)N/A100 GBN/AUnlimited
IntegrationsXporter, eazyBI, Structure, Confluence, Bamboo, Jenkins, TeamCity

Xporter, Jenkins, TeamCity, GitHub, Git Lab, Bitbucket Pipelines, Azure DevOps, Travis CI, Circle CI, Maven, Gradle, Scriptrunner

Xporter, eazyBI, Structure, Confluence, Bamboo, Jenkins, TeamCityXporter, Jenkins, TeamCity, GitHub, Git Lab, Bitbucket Pipelines, Azure DevOps, Travis CI, Circle CI, Maven, Gradle, Scriptrunner
Xray API

Extends Jira's REST API.

Able to import multiple reports and manage all entities.

REST API: able to import multiple reports.

GraphQL API: manage all entities.

API Rate Limit:

When

when moving to Cloud, you may face API rate limits. As Xray uses Jira API in most of their requests, please contact Atlassian Support to understand what is the limit for your instance.

Extends Jira's REST API.

Able to import multiple reports and manage all entities.

REST API: able to import multiple reports.

GraphQL API: manage all entities.

API

rate

Rate Limit:

When

when moving to Cloud, you may face API rate limits. As Xray uses Jira API in most of their requests, please contact Atlassian Support to understand what is the limit for your instance.

Automation frameworks/formats supportedCucumber, Xray, JUnit, NUnit, TestNG, xUnit, Robot framework, Robot framework v4.0, BehaveCucumber, Xray, JUnit, NUnit, TestNG, xUnit, Robot Framework, BehaveCucumber, Xray, JUnit, NUnit, TestNG, xUnit, Robot framework, Robot framework v4.0, BehaveCucumber, Xray, JUnit, NUnit, TestNG, xUnit, Robot Framework, Behave
History Tab

Image Added

The History tab shows all of the Issue's operations/updates history

Image Added

The History tab shows the following Issue’s operations/updates:

  • Changing the Test Type.
  • Changing the Test Definition or Steps.
  • Linking the Test with other entities (Preconditions, Test Plans, Test Sets, etc.).
  • Moving the Test between folders on the Test Repository.
  • Attachment addition.
  • Test Execution.

Image Added

The History tab shows all of the Issue's operations/updates history

Image Added

The History tab shows the following Issue’s operations/updates:

  • Changing the Test Type.
  • Changing the Test Definition or Steps.
  • Linking the Test with other entities (Preconditions, Test Plans, Test Sets, etc.).
  • Moving the Test between folders on the Test Repository.
  • Attachment addition.
  • Test Execution.
Administration & Configuration

Please refer to documentation the article [Xray] What is the difference between Administration & Configuration in Xray Server/DC vs Xray Cloud? for more details on the differences in configuration

&

and administration.



Xray Enterprise requires an active Xray Standard License.

Understand the icons: 

warning cloud - Available in Xray Cloud but with some differences or minor UI adjustments

(error) cloud - Not available in Xray Cloud

plus sign cloud - Extra option or functionality in Xray Cloud

Expand
titleSupport/Troubleshooting

If you have questions or technical issues, please contact the Support team via the Customer Portal (Jira service management) or send us a message using the in-app chat.