What you'll learn
- How to export test cases from Zephyr Scale
- Use a script to convert the exported XML test cases to an Xray compatible CSV file
- Import different test cases (with different fields and options)
- Validate that the test cases were imported to Xray
Overview
This tutorial explains how to export test cases from Zephyr Scale and import them into Xray.
Zephyr Scale is a test management tool running in Jira with versions available on-premises and in the cloud. In this tutorial, you will learn how to export the test cases from Zephyr Scale in an XML format, how to use a script (available in GitHub) to convert the XML file into a compatible CSV so that it can be imported into Xray using the Test Case Importer.
This tutorial will provide different examples of exporting and importing test cases from Zephyr Scale into Xray.
Features and Limitations
Below is a list of the support features and current limitations.
Migration from Zephyr Scale is limited by the information we can export from Zephyr Scale and how it can be mapped to Xray entities.
Most of the test case information will be migrated seamlessly, but please check the following table for more details.
| Supported Features | Unsupported |
|---|---|
|
|
How it works
Zephyr Scale is an add-on that can be enabled in your Jira project and allows the users to specify and organize test cases into folders. Zephyr Scale has test cycles and plans; these concepts help the organization and execution of test cases. Depending on your testing goals, you can use one or another.
Zephyr Scale can create test cases in plain text, step-by-step, or using a BDD approach (with Gherkin). The test case using plain text allows the user to create a test case with a description only. The step-by-step test case allows the user to define multiple steps with step, test data, and expected result; in this type of tests, you can define a data set and use variables or call other test steps. The BDD approach enables the user to define scenarios in Gherkin.
In Zephyr Scale, to execute tests, you need to associate them to a test cycle.
Zephyr Scale uses different entities to describe tests and the flow, so we have synthesized into the next table the existing mappings between Zephyr Scale and Xray.
| Zephyr Scale | Xray | Comments |
|---|---|---|
| Test Case | Test | All three types of tests from Zephyr are imported into Xray. The test case attachments are not exported using the XML export option in Zephyr Scale |
Precondition (Field in Test Case entity) | Precondition (independent entity) | In Zephyr Scale, the precondition is one field in the test case. In Xray, a Precondition is a different entity and one Test Case can be linked to several Preconditions. The precondition field from Zephyr Scale is converted into a Precondition issue in Xray and linked back to the Test. |
| Test Folder | Test Repository | The test folders and sub-folders of Zephyr Scale are mapped into Xray Test Repository. |
Prerequisites
Examples
In each example, we will show how we obtained the XML file from Zephyr Scale, made the files available in GitHub, how to use the scripts to convert the XML files into compatible CSV files, and finally, how to import the CSV files into Xray.
We will showcase different possibilities when exporting test cases from Zephyr Scale, using a combination of fields and possibilities that should cover most usages.
All of the examples in this tutorial have an XML file exported from Zephyr Scale, a CSV file with the definition of the test cases to import, and a configuration file that will configure all associations and fields for the importation to be successful.
Export to XML from Zephyr Scale and Import into Xray
In Zephyr Scale when you choose to export using XML format, it will export the selected test cases. In this example, we are exporting all test cases.
To export from Zephyr Scale we are using the Export to XML option in the More upper menu entry in Zephyr Scale left menu.
Looking into more detail on what we have in Zephyr Scale, we can see that we have defined 2 folders:
- Tests (with one sub-folder)
- Regression Tests
- Performance
Several test cases are defined, covering the different types available in Zephyr Scale. One behavior-driven development test case with one scenario in Gherkin, one plain text test case that describes the entire test in one description field. And several step-by-step test cases, using a dataset, re-using test cases, and using all the variations in the steps available (tables, images, different formats, etc.). All of the details of each case is described below.
The file generated by Zephyr Scale has the following content:
To import into Xray, we created one script to convert this XML file into a CSV-compatible file to be imported into Xray.
The script will parse the XML file and create a CSV file that can be imported into Xray. To execute the script, use the following command:
python3 zephyrscale2Xray.py -i atm-exporter.xml -o atm-exporter.csv
The output file has all the test steps and, if you have used a precondition in Zephyr Scale, it will create a precondition in Xray and link it to the test.
To import it into Xray, we use Test Case Importer with the recently created CSV file; you can use the configuration file provided with the .json extension (importConfiguration.json).
Choose the Project into which you want to import the Tests and select the option to create the Test Repository folders, as shown below.
When the import operation is complete, you can download the operation details, save the configuration for future use, or check the created issues.
When you click over the link to see the imported issues, we can see that it has created twelve issues, six Preconditions and six Tests with all the properties defined in the CSV.
More Details
Let's look into some specificities of this approach and some test particularities.
Precondition field into Precondition issue
In Zephyr Scale Preconditions are another field in the Test that allows tables, different styles and multiline.
The precondition is exported in a 'precondition' field within the 'testCase' as we can see below.
When converting into a CSV for Xray, we are creating a new issue of type 'precondition' where we convert the exported HTML of the precondition element into Jira markup to preserve (most) the styles and elements.
Text with color or underline are two styles that we cannot import into Xray, the content will be there but without the style.
The next line links the precondition to the Test.
Once imported into Xray we can see that the first line creates a Precondition in Xray.
The Precondition summary in Xray is extracted from the precondition field of Zephyr Scale, we extract a maximum of 254 characters or until the first newline (summaries in Jira do not allow newline).
The next line creates a Test and links the Precondition created in the same operation.
Test Folder into Test Repository
In Zephyr Scale we can organize our tests into folders and subfolders as we can see below:
As we can see in the above picture we have:
- one section called 'Tests' with 2 tests and a sub-section 'Regression Tests' with two tests
- one section called 'Performance' with two tests
Zephyr Scale folders and sub-folders are exported into the XML file using the folder element in the following form:
After running the script that converts the XML file into a CSV format, we can see that it adds, in the Test Repo field, the Test Repository name to be created when uploading the file into Xray.
After importing into Xray, we can see that new directories were created in the Test Repository:
Traceability Issues into Requirement references
Zephyr Scale allows you to insert linked issues in the Traceability tab under Coverage, as we can see below:
Any test case from Zephyr Scale that has these references are exported into the XML into a issues field that can be composed by several issues. The exported XML has the following output:
After running the script we have the referenced issues in the Links field separated by ";". The CSV output is the following:
Once imported into Xray, the issues are added into the Linked issues.
Make sure that you have the right permissions to link the issues referenced otherwise the import will fail.
Manual Tests (With Steps separated and in Text) into Manual Test Case
Zephyr Scale allows you to create Manual Test Cases with steps (using steps, test data, and expected results) or using only a text field to describe the test.
We have created one test case using the steps in a text field and one using the steps separated into fields. The XML output is the following:
After running the script that converts this XML into a CSV file we can see that, as explained above, the Preconditions issues are created and linked back to the Tests. Each Test is created and the steps filled with the information extracted from the XML.
The first Test corresponds to a Manual Test Case with the summary obtained from the CSV, and the description combines the objective and the description extracted from the CSV.
In Zephyr Scale the test described in plain text does not have steps so the Test in Xray will be a manual step without any steps defined.
The second Test has one line per each step. Notice that the format is kept for the table and styles added in Zephyr Scale.
Modular test case from Zephyr Scale are exported with the steps unfolded and with no reference to the original test case.
Tests with data sets defined in Zephyr scale are imported without the data set.
BDD Tests into Cucumber Test Case
Zephyr also allows the creation of BDD test cases, this type define a scenario in Gherkin as we can see in the exported XML file below.
After converting the above file using the available script we get the following output:
Importing into Xray will create a Cucumber Test and add the objective into the Description field.
Other Tests
The script will convert all tests of type Plain Text, Step-by-Step, or BDD-Gherkin; all the other types will be discarded, and no output will be created in the resulting CSV file.
Tips
- Use the CSV file provided to import the examples and the associated JSON configuration to have the setup and mapping correct.
- Ensure you are using the scripts available for the right version of Jira (Cloud or Server).