Switching test management tools can be cumbersome, with the overhead of moving your assets and relearning the processes. But do not fret, with some help, you can take the steps necessary to streamline the experience significantly.

In this tutorial, we will provide guidance for users migrating from qTest (2023.6 SaaS version, primarily qTest Manager) to Xray in Jira Cloud, focusing on the testing workflow and best practices.

Disclaimer: concepts and UI elements may be slightly different in other qTest versions, but the high-level process should be consistent.

Before you migrate

To ensure your data is not lost in transition, you first need to identify all the migration steps. The preferred sequence is:

  1. (if you do not already manage requirements in Jira via Pro or Enterprise versions of qTest) Evaluate whether any adjustments are needed for qTest Requirements entities given the desired Jira layout (see the Mapping section below).
  2. (if you do not already manage requirements in Jira via Pro or Enterprise versions of qTest) Migrate the requirements data and make any corrections in Jira, if necessary. Options: 
    1. Requirement Details report
    2. Get All Requirements from qTest API, potentially modify the file, then import via Jira APIs.
  3. Export the qTest test cases
    1. Refer to this tutorial for the verified, recommended flow for all test sub-types
    2. Alternatively:
      1. If you have a qTest Scenario’s Git repository configured, you can import the .feature files with this API method.
      2. You can try getting All Tests from qTest API, modifying the file, then importing via Xray REST API.
  4. Enable Xray and consider if any elements need to be created/configured before the import of test cases (see the Mapping section below).
  5. Prepare the test cases from step 3 and import them to Xray.

Mapping

Even though you are migrating from one test management platform to another, some concepts will still be similar (maybe with different labels) as both tools facilitate Agile and DevOps practices with compatible/consistent approaches. 

We will focus our tutorial on the major differences. At the project level, for both Jira and Xray:

qTest

Jira/Xray

Comments

Project

Jira Project

If desired, you can have requirements in one project and testing entities in another, with cross-project traceability and reporting.

Test Plan tab -> Release

Jira Version

Entities in Jira/Xray can have more than 1 version value.

Test Plan tab -> Build

Jira Sprint


Project Modules in Requirements/Test Cases

Jira Components

Jira Labels or Custom Fields

Jira Components are a common way to have consistent grouping across your asset types (requirements, tests, and defects).

Xray Test Repository can be structured based on your qTest Project Modules as well.

Requirements tab

Jira Backlog

In Jira, you can

  • have multiple issue types as requirements (e.g. epics, stories, tasks) 
  • assign requirements to versions/sprints/components using issue fields;
  • establish traceability between requirements and tests by designating coverable issues and using the “Tests”/”Is Tested By” link type that Xray adds automatically.

In qTest, the structure you create on the Requirements tab will be mirrored on the Test Design tab. In Jira/Xray, you can be more flexible, if desired.

Nested requirements hierarchy can be achieved with Jira add-ons like Structure.

Test Design tab

Xray Test Repository

Xray Test Set

Both Test Repository and Test Set help organize your tests, with differences described in this section.

Test Case

Xray Test

See more in the second table below.

Test Execution tab -> Test Cycle

Xray Test Plan

In Xray, Test Plan is not mandatory for running the tests, only Test Execution is. But Test Plan is recommended for easier asset organization, especially with the dynamic option.

You can view the execution results on each involved entity (traceable through to the requirement story via links).

qTest Quick Run is similar to changing the test status from Xray Test Execution directly.

Test Execution tab -> Test Suite

Xray Test Execution

Test Runs

Test Run of the Test Execution

Test Run in Xray is not a separate Jira entity, it doesn’t have an ID.

Test Run Details in Xray are similar to qTest Test Pad.

In qTest, the Test Run inherits the properties of the Test Cycle or Test Suite at the time the Test Run is added. If you update the property of the Test Suite or Test Cycle after the Test Run has been added, the Test Run properties must be updated manually.

In Xray, Test Run automatically inherits the Test Execution properties and their updates. For Test Case changes, you have a choice - see the Data Consistency section.

Test Run Configurations for Execution

Test Environments

Especially for the use case of testing on different devices/platforms with minimal test case differences, consider Test Environments.

Defects tab

Jira Defect issue type

You can have more custom issue types for this role. Make sure all those types are set as coverable for Xray.

You can create defects directly from Xray Test Runs as well as from the standard issue creation dialog.

Data Query

Jira Query Language (JQL)


qTest Export Reports

qTest Insights

Jira Dashboards and Export

Xray Reports and Dashboard Gadgets

Xray provides a comprehensive list of scopes (e.g. requirements traceability, test plan metrics, test runs summary) in both standalone and dashboard gadget formats. Also, you can leverage Xporter or Document Generator for additional insights.

A couple examples:

  • qTest Requirement Details report or Test Case Detail report = Jira’s Export Issues to CSV
  • qTest Requirement Traceability Matrix = Xray Traceability Report

Explorer

Xray Exploratory App

You can create a custom test type (“exploratory”) in Xray and kick off the execution in Xray Exploratory App directly from the Xray UI.

Schedule and Distribute Test Automation in qTest Launch

Remote Jobs Triggering

Not exactly identical, but Remote Jobs Triggering serves a similar purpose of connecting your test management and automation more seamlessly.

Pulse

Jira Workflows and Jira Automation

Also keep in mind the Xray settings for defects inheriting test details.

Name field

Summary field


APIs

REST API

GraphQL API



Note: the mapping above is recommended for better clarity but, in many cases, is not mandatory. E.g. you could map releases to sprints, builds to components, Test Cycles to Test Sets or directly to Test Executions, etc.


At the test case level for Xray: 

qTest

Xray

Comments

Sub-type

Type


Manual

Manual Test

Pre-condition

Xray Test issue layout overall is the same as any other Jira issue. You can use these Jira Automation tips to inherit data from requirement stories.

Tests can be copied, shared, and approved according to Jira permission schemes and workflow statuses. Xray supports reusing steps via modular tests (aka “call test”).

Robust test case versioning is available as part of Xray Enterprise while basic history can be found on the corresponding tabs of an Xray Test issue.

From the migration perspective, 

Supported:

  • All sub-types
  • Test and Test steps/Scenario content
  • Metadata fields

Not supported:

  • Test case attachments
  • Datasets
  • “Call” step type

For linked Defects, Requirements, Test Plans/Sets/Repository folders, if you have already migrated those entities to Jira (or recreated), you can substitute the values in the qTest export with Jira issue keys, so that the test issue will be linked upon import. Otherwise, you will need to re-link after the import.

Once the test entities are migrated, you will need to recreate any applicable attachments/comments, datasets, and modular (“call”) tests in Xray.

Automation

Performance

Generic Test

Scenario add-on

Cucumber Test and BDD Steps Library

Parameters and datasets

Parameter lists and dataset

Parameterized Tests in Xray


To summarize, for the migration, you will need to pay particular attention to:

  • custom issue types, fields, and statuses;
  • ways of organizing your tests (i.e. recreating test executions and likely test plans/test sets/test repository folders);
  • attachments and datasets (move them to a cloud storage outside qTest then manually reattach to Xray entities).


To compare the visual issue layout for test cases (same color = same function):

qTest



Xray



Getting started with Xray after migration

If you've used qTest before, adopting Xray is straightforward. Your high-level workflow with qTest (focusing on the testing aspects) was likely this: 

  1. Create a project.
  2. Create a Test Plan structure for releases and builds. Use Pulse for workflow rules. 
  3. Create entities on the Requirements tab to establish the project scope and link them to releases/builds.
  4. Create entities on the Test Design tab and link them to requirements.
  5. Plan test execution with Test Cycles and Test Suites.
  6. Execute via Test Runs.
  7. See the results on the Test Execution entities.
  8. Utilize Export Reports and Insights for project-level analysis.

 

In Jira and Xray, your actions would be similar:

  1. Create a project.
  2. Create fix versions and sprints. Define the Jira workflow with statuses and actions.
  3. Create Epics and Stories to establish the project scope and assign fix version and sprint values via issue fields.
  4. Enable Xray for the project
    1. Map your issue types to requirements and/or defects to make them coverable by Xray tests (in global settings - Jira Administration > Manage Apps > Xray > Issue Type Mapping) 
    2. Activate Xray Requirement Coverage for the project
  5. Create Test issues and connect them to the requirements via “tests” link type. Leverage Test Sets or Test Repository folders for structure (optional but recommended).
  6. Plan test execution with Test Plans (optional but recommended) and Test Executions. Assign the necessary tests accordingly.
  7. Execute via Test Runs.
  8. See the results on the Test Execution, Test Plan, Test, or Requirement entities.
  9. Utilize the reports (built-in or from Xporter/Document Generator) and/or dashboards for project-level analysis.

 

You can watch our Xray Cloud product demo for a more comprehensive process walkthrough.

Best practices for Xray adoption

For a consolidated collection of useful tips please refer to this Process section, as well as the “Tutorial, Tips, and Tricks” category overall. Here, we highlight just a few best practices:

  • Avoid cloning issues; reuse as much as possible.
    • modular and parameterized test, versioning.
    • Test Environments.
  • You can leverage Jira Automation to make Test issues inherit metadata from the stories.
  • Xray can take advantage of Jira’s built-in workflow mechanism in order to provide greater control over the specification, execution, and planning.
  • Restrict data (i.e., filter it) in the reports or gadgets to make the insights more focused.
  • Keep in mind that Xray has both global and project-specific settings. 
  • Leveraging “Health Check” in global settings every 3 months is recommended.
  • You can track storage usage in Xray global settings; we advise you to perform this periodically.
  • Remember that JQL and filters are used extensively throughout Jira, so teams should be aware that they should optimize their queries and use proper JQL functions.
  • Have control over the REST API as it’s easy to “get excited with it” and abuse it.
  • We highly recommend you to take Atlassian University Course Jira Governance & Housekeeping as most of the recommendation do apply to Xray

 

Xray and its Enterprise version are very comprehensive solutions for test management, so there's much to explore. We recommend checking out the academy courses, product guide, and tutorials.


  • No labels