This article explains in deep detail the differences between Xray Cloud Administration & Configuration.
First, to understand the differences between Xray Cloud Administration & Xray Data Center, let's clarify some concepts:
Global Settings - Xray settings that apply to all projects that use Xray. Global settings could be access by Jira Administration > Apps.
Project Settings - Xray settings that are aplicable to one specific project. In case of you need to replicate the setting per several projects, you need to apply the configuration to all projects that need the same setting. Xray Projects Settings are accessible at Project > Project Settings.
The main difference between Xray Cloud Administration & Xray Data Center is that in Cloud you have a much better control of the settings at a project level.
Find a comparison view of the settings and features in Server / DC vs Cloud about Administration & Configuration on the above table.
This a comparison on settings that you can control on Xray Administration ( Global & project level). To understand if a specific feature comparison please use the Xray Server/Data Center and Xray Cloud.
Xray Global Settings
Group
Setting
Xray Server/ DC
(Standard & Enterprise)
Xray Cloud
(Standard & Enterprise)
Miscellaneous Settings
The Global Configurations for Xray.
Display the Pre-Conditions section before the Test Details
Display Manual Test Results table in short view
Available under Global by default and per Project after enabling it.
Project admins can configure Test Step Custom Fields
Available by default on the project level through Test Step Fields, no longer a Global Setting
Project admins can configure Test Run Custom Fields
Available by default on the project level through Test Run Custom Fields, no longer a Global Setting
Test Run Options
Set final Test Run status
Inline Execute Tests
Restrict test execution to the assignee
Fail All Steps/Examples
Available however changing the Run to Fail will change all Test Steps to fail and not just the ones in TODO.
Restrict automated Test status
- By default, the automated Test status can be set manually.
Can Edit TestRun Dates
The setting is available under Project Settings → Miscellaneous
Track Time
Set the value of the Time Tracker
Enable Test Run Status history log in Test Executions
Defect Links
Link Defects with Test
Link Defects with Test Execution
Link Defects with Requirement
The menu is "Linked Defects with Covered Issues"
User Groups allowed to add Xray Issue Types
This permission is controlled by Jira permissions (only project administrators can add Xray issue Types to a project)
Notifications
User Groups allowed to add/remove Xray Notifications
- Xray Notifications are not available in Cloud.
Final Test Run statuses that may trigger notifications
Hierarchical Test Organization
Enable Hierarchical Test organization
- By default is set with Hierarchical Organization
Project Roles allowed the edit Test Repository information
- Anyone with access to the project can edit Test Repo info.
Workflow Status
Disallow executions of Tests with workflow statuses
Disallow executions with workflow statuses
In progress test Execution workflow status
Resolved Test Execution workflow status
Resolution for Resolved Test Execution
Clone Issues
Associate a cloned Test with all original Test Sets
Associate a cloned Test with all original Test Plans
Defect Default Values
Environment
- Although the option does not exist, It behaves with the default linked to "Test"
Steps to Reproduce
Version
- By default it behaves as the Affect Version field of the defects created in the context of a test run will be populated by the FixVersion value of Test Execution.
Label
Component
Automated Steps Library
Enable Automated Steps Library
Validate Cucumber Scenarios
Cucumber Export Prefixes
Test
- Available under Global Settings → Cucumber". Unable to create a prefix for a Test Plan.
Pre-Condition
Test Set
Test Plan
Requirement
Cucumber Language
Language
Cucumber Export
Replace dash [ - ] with underscore [ _ ] when exporting issue keys as tags
Generate features by requirements
Cucumber export feature name
Document Generator
Enable Document Generator
Available under Global Settings Miscellaneous & Feature
Max number of issues that can be exported
Requirements Reports
Max number of requirements per report or gadget
Requirements Tests
Maximum number of Links added to a Requirement
Gadgets
Test Evolution Tests Limit
Test Evolution Test Executions Limit
Test Runs list limit
Tests per Test Plan
Tests per Test Plan Maximum allowed
Xray REST API
Xray REST API - Max results per request
Associating a Test Execution to a Test Plan and importing Test results
Bulk operations
Max Test Runs for bulk operations
Datasets
Maximum size for parameter values
Xray Exploratory App
Enable Xray Exploratory App execution links
Usage Metrics
Enable Xray Analytics
Thread Pool Size
Thread Pool Size
Chat Support
Enable In-app Chat Support
Notifications
Enable Xray notifications
Show Xray notifications button
Custom Fields
Custom Fields
Test Run Status Custom Field Version
Test Set Status Custom Field Version
Requirement Status Custom Field Versions
Maintenance
Recalculate Requirement Status
Test Types
Manage Test Types
Manage Test Type
Available under "Global Settings – Test Types" and "Project Settings – Test Types".
In Xray Cloud it's not possible to set different Test Types to Test & Precondition, meaning that it applies to both issue types. Learn more <link>
Manager Pre-Condition Type
Xray Enterprise
Xray Enterprise Settings
Xray Test Case Designer
Issue Type Mapping
Issue Type Mapping
Available Issue Types
- Available under Project Configuration → Test Coverage & Defects Mapping, meaning that you can change your setting per project.
Requirement Issue Types
Defect Issue Types
Sub-Requirements
Sub-Tasks/Links
Epic - Story relation as a Requirement
Issue Link Type
Requirement Coverage
Requirement Coverage Strategy
Ignore Requirements with Status
Ignore Requirements with Label
Ignore Tests with Status
Separation of concerns
Include Previous Version Requirements
Show Flatten Requirements
Document Generator
Document Generator Templates
Requirement Projects
Test Statuses
Test Statuses
Manage custom Test Run statuses for Xray
Preferences
Test Step Statuses
Manage custom Test Step statuses for Xray
Default Column Layouts
Test Execution TestRun Columns
Parameter value lists
Integrity Checker
Integrity Checker
Calculated Custom Field Values
Automated Steps Library
Integrity of Hierarchical Test Organization
License Management
License Information
Cloud Migration
Cloud Migration Pre-Flight Checker
Features
** Setting exclusive to Xray Cloud.
BDD Step Libraries
** Setting exclusive to Xray Cloud.
Cucumber
** Setting exclusive to Xray Cloud.
Test Run Archiving
** Setting exclusive to Xray Cloud.
Test Case Designer
** Setting exclusive to Xray Cloud.
API Keys
** Setting exclusive to Xray Cloud.
Storage
** Setting exclusive to Xray Cloud.
Backup
** Setting exclusive to Xray Cloud.
Health Check
** Setting exclusive to Xray Cloud.
Coverage & Defects Configuration
Available under Project Configuration → Test Coverage & Defects Mapping, meaning that you can change your setting per project.
Sub-Requirements
?
Workflow & Notifications
Migration & Copy
Backup & Restore
Backup
Restore
Xray in-App Notifications
Archiving Xray Issues
Adds new feature: Auto-Archiving
Storage
Not Applicable
Storage is managed on-premise
Adds new feature Storage Management
Miscellaneous Settings
Display the Pre-Conditions section before the Test Details
Display Manual Test Results table in short view
Available under Global by default and per Project after enabling it.
Project admins can configure Test Step Custom Fields
Available by default on project level through Test Step Fields, no longer a Global Setting
Project admins can configure Test Run Custom Fields
Available by default on project level through Test Run Custom Fields, no longer a Global Setting
Set final Test Run status
Inline Execute Tests
Restrict tests execution to the assignee
Fail All Steps/Examples
Available however changing the Run to Fail will change all Test Steps to fail and not just the ones in TODO.
Restrict automated Test status
By default, the automated Test status can be set manually.
Can Edit TestRun Dates
The setting is available under Project Settings → Miscellaneous
Track Time
Set the value of the Time Tracker
Enable Test Run Status history log in Test Executions
Link Defects with Test
Link Defects with Test Execution
Link Defects with Requirement
The menu is "Linked Defects with Covered Issues"
User Groups allowed to add Xray Issue Types
This permission is controlled by Jira permissions (only project administrators can add Xray issue Types to a project)
User Groups allowed to add/remove Xray Notifications
Xray Notifications are not available in Cloud.
Final Test Run statuses that may trigger notifications
Xray Notifications are not available in Cloud.
Enable Hierarchical Test organization
By default is set with Hierarchical Organization
Project Roles allowed the edit Test Repository information
Anyone with access to the project can edit Test Repo info.
Disallow executions of Tests with workflow statuses
Disallow executions with workflow statuses
In progress test Execution workflow status
Resolved Test Execution workflow status
Resolution for Resolved Test Execution
Associate a cloned Test with all original Test Sets
Associate a cloned Test with all original Test Plans
Environment
Although the option does not exist, It behaves with the default linked to "Test"
Steps to Reproduce
Version
By default it behaves as the Affect Version field of the defects created in the context of a test run will be populated by FixVersion value of Test Execution.
Label
Component
Enable Automated Steps Library
Validate Cucumber Scenarios
Cucumber Export Prefixes
Available under Global Settings → Cucumber" but unable to create a prefix for a Test Plan.
Cucumber Language
Setting moved to a dedicated option in the menu tab: Global Settings → Cucumber
Replace dash [ - ] with underscore [ _ ] when exporting issue keys as tags
Generate features by requirements
Cucumber export feature name
Enable Document Generator
Available under Global Settings Miscellaneous & Feature
Max number of issues that can be exported
Max number of requirements per report or gadget
Maximum number of Links added to a Requirement
Test Evolution Tests Limit
Test Evolution Test Executions Limit
Test Runs list limit
Tests per Test Plan Maximum allowed
Xray REST API - Max results per request
Associating a Test Execution to a Test Plan and importing Test results
Max Test Runs for bulk operations
Maximum size for parameter values
Enable Xray Exploratory App execution links
Enable Xray Analytics
Thread Pool Size
Enable In-app Chat Support
Enable Xray notifications
Show Xray notifications button
Test Environments
Possible to manage at a Global level & project level.
Automated / BDD Step Libraries
Not available as a Global setting, only as a project setting under the name "Automated Step Library"
Possible to manage at a Global level.
Parameter value lists
Cucumber
Test Run Archiving
Auto Archiving
Ability to manually archive Test Runs
Adds Auto Archive Feature
API Keys
Not Applicable
Access levels are managed on-premise
Storage
Not Applicable
storage is managed on-premise
Health Check
Features
Feature Toggling
Setting
Server
Cloud
Project Organization
Configuring Xray Test Project
Available under "Project Configuration → Summary"
Requirements per Project
Requirement Coverage Strategy
Available under "Project Configuration → Test Coverage & Defects Mapping"
Ignore Requirements with Status
Ignore Requirements with Label
Ignore Tests with Status
Requirement Historical Chart Days
Include Previous Version Requirements
Show Flatten Requirements
Separation of concerns
Configure Xray with an existent Project
Available under "Project Configuration → Summary"
Using Xray Template
Agile Boards
Available with limitations. We are only able to use 3 Xray Custom Fields, we are unable to showcase the "Requirement Status" field (one of the most used in DC/Server)
Permissions
(Permissions are Jira wise)
Summary
Not Applicable
Issue Types Mapping
Not Applicable
Only Available for Team Managed Projects
Test Coverage
Available under the name Issue Type Mapping
Defect Mapping
Test Types
Test Environments
Ability to manage Test Environment at a project level.
BDD Step Library
Available under the name "Automated Steps Library"
Test Step Fields
Available under Project Settings → Test Step Custom Fields
Test Run Custom Fields
Available under Project Settings → Test Run Custom Fields
Parameter value lists
Default Column Layouts
Test Execution TestRun Columns
Test TestRun Columns
Test PreCondition Columns
Test Test Set Columns
Test Test Plan Columns
Test Set Test Columns
Test Plan Test Columns
Test Plan Test Execution Columns
Test Coverage Columns
Project Test Columns
Project Test Set Columns
Project Test Execution Columns
Project Test Plan Columns
Precondition Tests Columns
Re-Indexing
Understand the icons:
- Available in Xray Cloud but with some differences or minor UI adjustments