Switching test management tools can be cumbersome, with the overhead of moving your assets and relearning the processes. But do not fret, with some help, you can take the steps necessary to streamline the experience significantly. 

In this tutorial, we will provide guidance for users migrating from Zephyr Squad to Xray in Jira Cloud, focusing on the testing workflow and best practices.

Before you migrate

To ensure your data is not lost in transition, you first need to identify all the migration steps. The preferred sequence is:

  1. Evaluate whether any adjustments are needed for Zephyr test cases given their Excel export limitations and the desired Xray layout & format (see the Mapping section below).

  2. Export the Zephyr test case data - use XLS for step-based tests and .feature for BDD ones.

  3. Disable Zephyr Squad - uninstalling the plugin prevents you from accessing any Zephyr-specific functionality (test step data, test cycle and test execution details).
    1. If you must have Zephyr Squad and Xray plugins active simultaneously, make sure they don’t share the “Test” issue type name (i.e. “Test Case” or “TestZephyr” is ok).

  4. Enable Xray and consider if any elements need to be created/configured before the import of test cases (see the Mapping section below).

  5. Prepare the test cases from step 2 and import them to Xray. Refer to this tutorial for the specifics about step-based tests. For BDD tests, you will need to use the API method.

Mapping

As you are migrating from one Jira Cloud plugin to the other, the support for Jira’s native fields will be similar - you can assume that the elements we don’t explicitly mention are consistent between the plugins.

So, we will focus our tutorial on the plugin-specific concepts and entities. At the project level:

Zephyr Squad

Xray

Comments

Test Case

Test, Precondition

Xray supports all types of tests from Zephyr Squad as well as custom ones. 

Keep in mind:

  • Zephyr Squad does not export the test case attachments in the Excel file.
  • Zephyr Squad does not export the details of a BDD test case (Scenario Outline/ Scenario) in the Excel file.


Xray’s Test layout is more similar to the one from standard Jira issue types. For common prerequisite steps, you can leverage the dedicated Precondition issue type.

Test Cycle

Test Execution issue (with or without Test Plan issue)

Both Test Cycle and Test Execution serve as mandatory means of organizing your test cases for execution. Test Plans are a flexible optional layer, especially useful when the same group of tests is run in multiple Test Executions.

Test Player inside Test Cycle

A line item on the Executions tab in the Test Case view

Test Run Details

In Test Execution issues, you click the “Play” button for each test, then the aspects like step-level results or attachments are handled in the Test Run Details view.

Test Cycle iteration

Test Cycle environment

Multiple Test Execution issues with different Test Environments

The use of Test Environments in Xray is very broad. This is also a good use case for a Test Plan linked to those multiple Test Executions.

Test Case Library Folders/Subfolders

Test Repository Folders/Subfolders

You can move the test cases after the import or create the folder structure in advance, specify the location in the csv, then map it during the import. You can also consider using Test Sets.

Zephyr Squad link in the left navigation menu

Testing Board link in the left navigation menu


Reports

Reports section of the Testing Board

Both plugins provide traceability and execution-focused reports. In addition, Xray provides a more comprehensive list of scopes (e.g. requirements, test plans). Also, you can leverage Xporter or Document Generator for additional insights.

REST API

REST API

GraphQL API



At the issue (e.g. test case or story) level: 

Zephyr Squad

Xray

Comments

Details tab

Jira’s Description and other metadata fields

For Xray, this information is handled in Jira’s settings for issue layouts. I.e. if you are using custom fields in Zephyr, you need to create them for Jira issues and apply that layout to Xray tests.

Test Script tab

Test Details section

The layout for both step-based and BDD tests is similar. You can also create custom step-level fields in Xray settings.

Executions tab

Test Runs section


Traceability tab

Linked issues and Web Links sections

The way Jira “sees” the links is a bit different between the plugins, but you should be able to map the “Coverage” column from the Zephyr export to the “Outward link (tests)” one in the Xray import.

Attachments tab

Attach button in the Jira issue layout

Attachments at the test case level and the test step one are supported by both plugins.

Comments and History tabs

Comments and History subsections under the Activity section


Prompt to create or add test cases for a story

List of associated test cases

Test Coverage button and the associated section of a story


Test Cycle execution progress

Test Coverage button and the associated section of a story



To summarize, for the migration, you will need to pay particular attention to:

  • ways of organizing your tests (i.e. recreating folders, test executions, and possibly test plans/test sets);
  • attachments (move them to non-Zephyr issues like associated stories/bugs or transfer to a cloud storage outside Jira then manually reattach to Xray entities);
  • custom fields and statuses.


To compare the visual issue layout (same color = same function):

Zephyr Squad

Xray

Getting started with Xray after migration

If you've used Zephyr Squad before, adopting Xray is easy. Your high-level workflow with Zephyr Squad was likely this: 

  1. Enable Zephyr Squad for a given project
  2. Open a requirement story
  3. Use the Zephyr Squad section to create new tests (or add existing ones)
  4. Define the Test Case Details and Test Script
  5. Create a Test Cycle for execution and add the tests
    1. In Zephyr Squad, a specific testing goal for a given cycle can be pretty flexible. Usually, we would create a test cycle for a given time period (e.g. sprint) + the testing scope (e.g. regression, build-verification, end-to-end). Next sprint, for the same scope, we would need to create a new cycle as each one holds only the execution results specific to that time period.
  6. Execute
  7. See the results on the requirement story
  8. Utilize the reports and/or dashboards for project-level analysis


In Xray, your actions would be similar:

  1. Enable Xray for a given project
    1. Map your issue types to requirements and/or defects to make them coverable by Xray tests (in global settings - Jira Administration > Manage Apps > Xray > Issue Type Mapping) 
    2. Activate Xray Requirement Coverage for the project
  2. Open a requirement story
  3. Use the Test Coverage section to create new tests (or add existing ones)
  4. Define the Test Details and issue metadata fields
  5. (optional) Create a Test Plan from the respective section of Test Details
    1. In Xray, Test Plans are also flexible in defining the goal and the scope of testing but not as dependent on the time period - you can have the same Test Plan issue between sprints (associated with different Test Executions) and therefore consolidate the results from multiple iterations/builds.
  6. Create a Test Execution from the “Test Runs” section of Test Details
  7. Execute
  8. See the results on the requirement story
  9. Utilize the reports (built-in or from Xporter/Document Generator) and/or dashboards for project-level analysis


You can watch our Xray Cloud product demo for a more comprehensive process walkthrough.

Best practices for Xray adoption

For a consolidated collection of useful tips please refer to this Process section, as well as the “Tutorial, Tips, and Tricks” category overall. Here, we highlight just a few best practices:

  • Avoid cloning issues; reuse as much as possible.
    • modular and parameterized test, versioning.
    • Test Environments.
  • You can leverage Jira Automation to make Test issues inherit metadata from the stories.
  • Xray can take advantage of Jira’s built-in workflow mechanism in order to provide greater control over the specification, execution and planning.
  • Restrict data (i.e., filter it) in the reports or gadgets to make the insights more focused.
  • Keep in mind that Xray has both global and project-specific settings. 
  • Leveraging “Health Check” in global settings every 3 months is recommended.
  • You can track storage usage in Xray global settings; we advise you to perform this periodically.
  • Remember that JQL and filters are used extensively throughout Jira, so teams should be aware that they should optimize their queries and use proper JQL functions.
  • Have control over the REST API as it’s easy to “get excited with it” and abuse it.
  • We highly recommend you to take Atlassian University Course Jira Governance & Housekeeping as most of the recommendation do apply to Xray


Xray and its Enterprise version are very comprehensive solutions for test management, so there's much to explore. We recommend checking out the product guide, academy courses, and tutorials.


  • No labels