Switching test management tools can be cumbersome, with the overhead of moving your assets and relearning the processes. But do not fret, with some help, you can take the steps necessary to streamline the experience significantly.

In this tutorial, we will provide guidance for users migrating from Zephyr Scale to Xray in Jira Cloud, focusing on the testing workflow and best practices.


Note: many of the concepts and flows are similar (if not exactly the same) in Zephyr Scale for Data Center, so this tutorial could be used as high-level guidance for the migration from there to Xray Cloud as well.

Before you migrate

To ensure your data is not lost in transition, you first need to identify all the migration steps. The preferred sequence is:

  1. Evaluate whether any adjustments are needed for Zephyr test cases given their XML export limitations and the desired Xray layout & format (see the Mapping section below).

  2. Export the Zephyr test case data - follow the initial steps in this tutorial.

  3. Disable Zephyr Scale - uninstalling the plugin prevents you from accessing any Zephyr-specific functionality (test step data, test cycle and test execution details).
    1. If you must have Zephyr Scale and Xray plugins active simultaneously, make sure they don’t share the “Test” issue type name (i.e. “Test Case” or “TestZephyr” is ok).

  4. Enable Xray and consider if any elements need to be created/configured before the import of test cases (see the Mapping section below).

  5. Prepare the test cases from step 2 accordingly and import them to Xray following the rest of the tutorial.

Mapping

As you are migrating from one Jira Cloud plugin to the other, the support for Jira’s native fields will be similar - you can assume that the elements we don’t explicitly mention are consistent between the plugins.

So, we will focus our tutorial on the plugin-specific concepts and entities. At the project level:

Zephyr Scale

Xray

Comments

Zephyr Scale link in the left navigation menu

Testing Board link in the left navigation menu

Xray also supports cross-project operations, you have quite a lot of flexibility in structuring you assets (see more here).

Xray entities are native Jira issues, so you can apply operations like JQL, workflows for status transitions, Jira Automation to them.

And as a side note, Xray and Xray Enterprise are also Cloud Fortified.

Test Cases tab

Test Repository Folders/Subfolders

You can move the test cases after the import or create the folder structure in advance then map it during the import. You can also consider using Test Sets.

Test Case

Test

Xray’s Test layout is more similar to the one from standard Jira issue types. Xray supports all types of tests from Zephyr Scale as well as custom ones. 

Related features that Xray supports:

Test Cycle

Test Execution issue 

Similar to Zephyr Scale entities, Xray Test Execution is mandatory for running the tests while Xray Test Plan serves as an optional higher-level layer, especially useful when the same group of tests is run in multiple Test Executions.

The order of creation in Xray does not matter (e.g. you can create an Xray Test Plan first, Xray Test Execution second). You can add tests directly to the Xray Test Plan.

You can have more than 1 Xray Test Execution per Sprint. You can clone both Xray Test Executions and Xray Test Plans.

Xray also supports easy re-running of tests by status.

Test Plan

Test Plan issue

Test Player inside Test Cycle

A line item on the Executions tab in the Test Case view

Test Run Details

In Xray Test Execution issues, you click the “Play” button for each test individually, then the aspects like step-level results or attachments are handled in the Test Run Details view.

Test Cycle environment

Multiple Test Execution issues with different Test Environments

The use of Test Environments in Xray is very broad. This is also a good use case for a Test Plan linked to those multiple Test Executions.

Reports tab

Reports section of the Testing Board

Both plugins provide traceability, coverage, and execution-focused reports. In addition, Xray provides a more comprehensive list of scopes (e.g. requirements, test plans). Also, you can leverage Document Generator or Xporter for additional insights.

Xray Dashboard gadgets are also available out of the box.

Configuration

Project Settings -> Apps -> Xray Settings

Manage Apps -> Xray


REST API

REST API

GraphQL API

You can find some of the integration and automation examples in these sections of Xray documentation:


At the Story and Test level: 

Zephyr Scale

Xray

Comments

Zephyr Scale button at the top of the Story and Zephyr Scale section 

Test Coverage button at the top of the Story and Test Coverage section 

When a test is created from the Story screen, traceability is established automatically in Xray as well.

You can link Xray Test Executions and Test Plans to the Story or see the aggregated execution status in the Test Coverage section. You can learn more about Xray coverage calculations in this article.

Plain Text type of Test Case

Generic type of Test

Keep in mind - Zephyr Scale does not export the test case attachments in the XML file.

Step-by-Step type of Test Case

Manual type of Test

BDD-Gherkin type of Test Case

Cucumber type of Test

Details tab

Jira’s Description and other metadata fields

For Xray, this information is handled in Jira’s settings for issue layouts. I.e. if you are using custom fields in Zephyr, you need to create them for Jira issues and apply that layout to Xray tests.

A few elements to highlight:

  • Labels - for Xray entities, these are Jira labels, not a separate list.
  • Status - for Xray entities, these are tied to Jira workflows and their customizations; not a separate list.
  • Precondition - see below.

Precondition

(Field in Test Case entity)

Precondition

(independent entity)

In Zephyr Scale, the precondition is one field in the test case.

In Xray, a Precondition is a different entity and one Test Case can be linked to several Preconditions.

If you follow the instructions in the import tutorial, the precondition field from Zephyr Scale is converted into a Precondition issue in Xray and linked back to the Test.

Test Script tab

Test Details section

The layout for all 3 test types is similar between the plugins. You can also create custom step-level fields in Xray settings.

Executions tab

Test Runs section


Traceability tab

Linked issues and Web Links sections

Test Coverage section

The way Jira “sees” the links is a bit different between the plugins, but you should be able to map the “Coverage”/”Links” column from the Zephyr export to the “Outward link (tests)” one in the Xray import.

Attachments tab

Attach button in the Jira issue layout

Creating attachments at the test case level and the test step one is supported by both plugins.

Comments and History tabs

Comments and History subsections under the Activity section



To summarize, for the migration, you will need to pay particular attention to:

  • ways of organizing your tests (i.e. recreating folders, test executions, and possibly test plans/test sets);
  • attachments (move them to non-Zephyr issues like associated stories/bugs or transfer to a cloud storage outside Jira then manually reattach to Xray entities);
  • custom fields and statuses.


To compare the visual Story layout (same color = same function):

Zephyr Scale


Xray



To compare the visual Test layout (same color = same function):

Zephyr Scale


Xray


Getting started with Xray after migration

If you've used Zephyr Scale before, adopting Xray is easy. Your high-level workflow with Zephyr Scale was likely this: 

  1. Enable Zephyr Scale for a given project
  2. Open a requirement story
  3. Use the Zephyr Scale section to create new tests (or add existing ones)
  4. Define the Test Case Details and Test Script
  5. Create a Test Cycle for execution and add the test(s)
  6. (Optional) Create a Test Plan to group Test Cycles
  7. Execute
  8. See the results on the requirement story
  9. Utilize the reports and/or dashboards for project-level analysis

In Xray, your actions would be similar:

  1. Enable Xray for a given project
    1. Map your issue types to requirements and/or defects to make the coverable by Xray tests (in global settings - Jira Administration > Manage Apps > Xray > Issue Type Mapping) 
    2. Activate Xray Requirement Coverage for the project
  2. Open a requirement story
  3. Use the Test Coverage section to create new tests (or add existing ones)
  4. Define the Test Details and issue metadata fields
  5. Create an Xray Test Execution issue from the “Test Runs” section of Test Details and add more tests to it, if desired
  6. (optional) Create an Xray Test Plan issue from the respective section of Test Details to group Xray Test Executions
  7. Execute
  8. See the results on the requirement story
  9. Utilize the reports (built-in and/or from Document Generator/Xporter) and/or dashboards for project-level analysis


You can watch our Xray Cloud product demo for a more comprehensive process walkthrough and check out the Xray Enterprise feature set.


Best practices for Xray adoption

For a consolidated collection of useful tips please refer to this Process section, as well as the “Tutorial, Tips, and Tricks” category overall. Here, we highlight just a few best practices:

  • Avoid cloning issues; reuse as much as possible.
    • modular and parameterized test, versioning.
    • Test Environments.
  • You can leverage Jira Automation to make Test issues inherit metadata from the stories.
  • Xray can take advantage of Jira’s built-in workflow mechanism in order to provide greater control over the specification, execution and planning.
  • Restrict data (i.e., filter it) in the reports or gadgets to make the insights more focused.
  • Keep in mind that Xray has both global and project-specific settings. 
  • Leveraging “Health Check” in global settings every 3 months is recommended.
  • You can track storage usage in Xray global settings; we advise you to perform this periodically.
  • Remember that JQL and filters are used extensively throughout Jira, so teams should be aware that they should optimize their queries and use proper JQL functions.
  • Have control over the REST API as it’s easy to “get excited with it” and abuse it.
  • We highly recommend you to take Atlassian University Course Jira Governance & Housekeeping as most of the recommendation do apply to Xray


Xray and its Enterprise version are very comprehensive solutions for test management, so there's much to explore. We recommend checking out the academy courses, product guide, and tutorials.