Switching test management tools can be cumbersome, with the overhead of moving your assets and relearning the processes. But do not fret, with some help, you can take the steps necessary to streamline the experience significantly.

In this tutorial, we will provide guidance for users migrating from ALM Octane (SaaS versions 23.4-24.1) to Xray in Jira Cloud, focusing on the testing workflow and best practices.

Disclaimer: concepts and UI elements may be slightly different in other ALM Octane versions, but the high-level process should be consistent.

Before you migrate

To ensure your data is not lost in transition, you first need to identify all the migration steps. The preferred sequence is:

  1. (if you do not already manage requirements in Jira) Evaluate whether any adjustments are needed for Octane Requirements entities given the desired Jira layout (see the Mapping section below).
  2. (if you do not already manage requirements in Jira) Migrate the requirements data and make any corrections in Jira, if necessary. Options: 
    1. Manual export then import using Jira
    2. Connect utility
  3. Export the Octane tests
    1. Refer to this tutorial for the specifics about manual tests. 
    2. For Gherkin tests, you will need to download the .feature files and then use the API method in step 5 below.
  4. Enable Xray and consider if any elements need to be created/configured before the import of test cases (see the Mapping section below).
  5. Prepare the test cases from step 3 and import them to Xray.

Mapping

Even though you are migrating from one test management platform to another, some concepts will still be similar (maybe with different labels) as both tools facilitate Agile and DevOps practices with consistent approaches. 

We will focus our tutorial on the major differences. At the project level, for both Jira and Xray:

ALM Octane

Jira/Xray

Comments

Workspace

Project


Teams

Groups

For permission management

Release

Version

Epics, stories, and other entities in Jira/Xray can have more than 1 version value.

Requirements

Backlog

Backlog

There are no separate modules per se for the entities in Jira/Xray. However, you can have requirements in one project and testing entities in another, with cross-project traceability and reporting.

Application module

Jira Component

Xray Test Repository folder

Xray Test Set

For requirements, Jira components are a common way to organize your assets.

For tests, you have multiple, not mutually exclusive options - e.g. you can have a component assigned together with a test repository folder.

The column responsible for this field in the ALM Octane export can be called “product_areas”. Depending on how you want to map it, you may have to split it into several columns before importing to Jira/Xray.

Epic

Feature

Epic

Both tools have 4 levels, so there can be flexibility in terms of mapping.

Story

Story

Task

Task

Task

Sub-Task

Defect

Bug


Backlog coverage (parent-child relations, “linked”, “covering”, etc.)

Links of different types

Xray adds the “Tests”/”Is Tested By” link type, you can also configure custom ones. The linked entity has to already exist in Jira for the link to be established during the import, otherwise it has to be done afterwards.

The column responsible for this field in the ALM Octane export can be called “covered_content”.

Test

Test

See more details in the section below

Test Suite

Test Plan

Test Execution

In Xray, Test Plan is not mandatory for running the tests, only Test Execution is. But Test Plan is recommended for easier asset organization, especially with the dynamic option.

You can view the execution results on each involved entity (i.e. Test Execution, Test Plan, Test, Story).

Suite Run

Manual Runner for multiple tests

Test Execution

Sub Test Execution

Name field

Summary field


Phase field

Workflow Status field

The list values are quite different between the systems, especially with the custom workflow possibilities, so you will need to identify the closest matches and create the missing ones, if necessary.

User tags field

Labels field


Octane Entity Settings Business Rules and Workflow 

Jira workflows (with validators, etc.) and Jira automation

Also keep in mind the Xray settings for defects inheriting test details.

Insights (Dashboard, Document Reports, Requirements Graphs, etc.)

Jira dashboards

Reports section of the Xray Testing Board

Xray provides a comprehensive list of scopes (e.g. requirements traceability, test plan metrics, test runs summary) in both standalone and dashboard gadget formats. Also, you can leverage Xporter or Document Generator for additional insights.

REST API

REST API

GraphQL API



Note: the mapping above is recommended for better clarity but, in many cases, is not mandatory. E.g. you could map features to stories, defects to tasks, custom field A to custom field B.


At the test case level for Xray: 

ALM Octane

Xray

Comments

Manual Test

Manual Test

Pre-condition

Xray Test issue layout is overall the same as any other Jira issue. 

From the migration perspective, 

Supported:

  • Both manual and Gherkin types (via separate import paths)
  • Test and Test steps/Scenario content
  • Metadata fields

Not supported:

  • Test case attachments/comments
  • Datasets
  • “Call” step type

For linked Defects and Requirements, if you have already migrated those entities to Jira, you can substitute the values in the Octane export with Jira issue keys, so that the test issue will be linked upon import.

Once the test entities are migrated, you will need to recreate any applicable attachments/comments, datasets, and modular (“call”) tests in Xray.

Gherkin Test

Cucumber Test

Test Type field

Label

Custom field

Xray Test Repository folder

Xray Test Set

Xray Test Plan

As with the Application Modules, you have different options in Jira/Xray to group tests by their nature (smoke, e2e, regression, etc.)

Manual Run for one test

Test Run of the Test Execution


Data table

Dataset


Automation Status field

Custom field

Label

Creating a custom field would allow a cleaner mapping since it won’t interact with any other labels.


To summarize, for the migration, you will need to pay particular attention to:

  • custom issue types, fields, and statuses;
  • ways of organizing your tests (i.e. recreating test executions and likely test plans/test sets/test repository folders);
  • attachments and datasets (move them to a cloud storage outside Octane then manually reattach to Xray entities).


To compare the visual issue layout for test cases (same color = same function):

ALM Octane


Xray



Getting started with Xray after migration

If you've used ALM Octane before, adopting Xray is straightforward. Your high-level workflow with ALM Octane (focusing on the testing aspects) was likely this: 

  1. Create releases and sprints. Define the workflow with phases.
  2. Create Requirements and Features assigned to release to establish project scope 
  3. Create application modules then tests linked to the requirements
  4. Define the Test details
  5. Create a Test Suite for execution and add the tests
  6. Execute
  7. See the results on the Feature
  8. Utilize reports and dashboards for project-level analysis

 

In Jira and Xray, your actions would be similar:

  1. Create fix versions and sprints. Define the workflow with statuses.
  2. Create Epics and Stories assigned to a fix version to establish project scope
  3. Enable Xray for a given project
    1. Map your issue types to requirements and/or defects to make them coverable by Xray tests (in global settings - Jira Administration > Manage Apps > Xray > Issue Type Mapping) 
    2. Activate Xray Requirement Coverage for the project
  4. Create Test Plans, Test Sets, or Test Repository folders to establish the structure for testing assets (all optional but at least one is recommended), then create Tests linked to the stories.
  5. Define the Test details
  6. Create a Test Execution and add the tests
  7. Execute
    1. You could still utilize UFT for this step, then integrate with Xray via JUnit or NUnit reports
  8. See the results on the requirement Story
  9. Utilize the reports (built-in or from Xporter/Document Generator) and/or dashboards for project-level analysis

 

You can watch our Xray Cloud product demo for a more comprehensive process walkthrough.

 

Best practices for Xray adoption

For a consolidated collection of useful tips please refer to this Process section, as well as the “Tutorial, Tips, and Tricks” category overall. Here, we highlight just a few best practices:

  • Avoid cloning issues; reuse as much as possible.
    • modular and parameterized test, versioning.
    • Test Environments.
  • You can leverage Jira Automation to make Test issues inherit metadata from the stories.
  • Xray can take advantage of Jira’s built-in workflow mechanism in order to provide greater control over the specification, execution, and planning.
  • Restrict data (i.e., filter it) in the reports or gadgets to make the insights more focused.
  • Keep in mind that Xray has both global and project-specific settings. 
  • Leveraging “Health Check” in global settings every 3 months is recommended.
  • You can track storage usage in Xray global settings; we advise you to perform this periodically.
  • Remember that JQL and filters are used extensively throughout Jira, so teams should be aware that they should optimize their queries and use proper JQL functions.
  • Have control over the REST API as it’s easy to “get excited with it” and abuse it.
  • We highly recommend you to take Atlassian University Course Jira Governance & Housekeeping as most of the recommendation do apply to Xray

 

Xray and its Enterprise version are very comprehensive solutions for test management, so there's much to explore. We recommend checking out the academy courses, product guide, and tutorials.

  • No labels