What you'll learn

  • How to export test cases from Zephyr Scale
  • Use a script to convert the exported XML test cases to an Xray compatible CSV file
  • Import different test cases (with different fields and options)
  • Validate that the test cases were imported to Xray

Source-code for this tutorial

  • All examples are available on GitHub

Overview

This tutorial explains how to export test cases from Zephyr Scale and import them into Xray.

Zephyr Scale is a test management tool running in Jira with versions available on-premises and in the cloud. In this tutorial, you will learn how to export the test cases from Zephyr Scale in an XML format, how to use a script (available in GitHub) to convert the XML file into a compatible CSV so that it can be imported into Xray using the Test Case Importer.

This tutorial will provide different examples of exporting and importing test cases from Zephyr Scale into Xray.

Features and Limitations

Below is a list of the support features and current limitations.

Migration from Zephyr Scale is limited by the information we can export from Zephyr Scale and how it can be mapped to Xray entities.

Most of the test case information will be migrated seamlessly, but please check the following table for more details.

Supported FeaturesUnsupported
  • Test and Test steps
  • Test Folders
  • Test case attachments
  • Preconditions
  • Test Cycles
  • Test Plans
  • Defects and Requirements


How it works

Zephyr Scale is an add-on that can be enabled in your Jira project and allows the users to specify and organize test cases into folders. Zephyr Scale has test cycles and test plans; these two concepts help in the organization and execution of test cases. Depending on your testing goals, you can use one or another.

Zephyr Scale can create test cases in plain text, step-by-step, or using a BDD approach (with Gherkin). The test case using plain text allows the user to create a test case with a description only. The step-by-step test case allows the user to define multiple steps with step, test data, and expected result; in this type of tests, you can define a data set and use variables or call other test steps. The BDD approach enables the user to define scenarios in Gherkin.

In Zephyr Scale to execute tests you need to associate them to a test cycle.

Zephyr Scale uses different entities to describe tests and the flow, so we have synthesized into the next table the existing mappings between Zephyr Scale and Xray.

Zephyr ScaleXrayComments
Test CaseTest

All three types of tests from Zephyr are imported into Xray.

The test case attachments are not exported using the XML export option in Zephyr Scale

Test FolderTest Repository

The test folders and sub-folders of Zephyr Scale are mapped into the Xray Test Repository.  



Prerequisites


For these examples we will use the Test Case Importer tool, which is present when you install Xray in your Jira instance.

 We will need:

  • Access to Zephyr Scale instance to export test cases in XML format
  • Access and usage of scripts to convert Zephyr Scale XML test cases to CSV Xray import files
  • Access to a Jira instance with Xray installed

  • Only users that have the Make Bulk Changes permission will have this option available. Additionally, the user will only be able to import data into projects where he has the Create Issues permission.



Examples

In each example, we will show how we obtained the XML file from Zephyr Scale, made the files available in GitHub, how to use the scripts to convert the XML files into compatible CSV files, and finally, how to import the CSV files into Xray.

We will showcase different possibilities when exporting test cases from Zephyr Scale, using a combination of fields and possibilities that should cover most usages.

All of the examples in this tutorial have an XML file exported from Zephyr Scale, a CSV file with the definition of the test cases to import, and a configuration file that will configure all associations and fields for the importation to be successful.


Export to XML from Zephyr Scale and Import into Xray

In Zephyr Scale when you choose to export using XML format, it will export the selected test cases. In this example, we are exporting all test cases.

To export from Zephyr Scale we are using the Export to XML option in the More upper menu entry in Tests upper option.


Looking into more detail on what we have in Zephyr Scale, we can see that we have defined 1 folder with 3 sub-folders:

  • Tests 
    • Plain
    • Steps
    • BDD


Several test cases are defined, covering the different types available in Zephyr Scale. One behavior driven development test case with one scenario in Gherkin, one plain text test case that describes the entire test in one description field. And one step-by-step test case using all the variations in the steps available (tables, images, different formats, etc). All of the details of each case is described below.

The file generated by Zephyr Scale has the following content:

atm-exporter.xml 
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<project>
    <projectId>10601</projectId>
    <projectKey>CE</projectKey>
    <modelVersion>1.0</modelVersion>
    <jiraVersion>8.22.6</jiraVersion>
    <exportDate>2023-08-28 14:54:30 UTC</exportDate>
    <folders>
        <folder fullPath="Test Cases" index="0"/>
        <folder fullPath="Test Cases/BDD" index="2"/>
        <folder fullPath="Test Cases/Plain" index="0"/>
        <folder fullPath="Test Cases/Steps" index="1"/>
    </folders>
    <testCases>
        <testCase id="3" key="CE-T3">
            <attachments/>
            <confluencePageLinks/>
            <createdBy>Cristiano</createdBy>
            <createdOn>2023-08-28 11:15:59 UTC</createdOn>
            <customFields/>
            <folder><![CDATA[Test Cases/BDD]]></folder>
            <issues/>
            <labels/>
            <name><![CDATA[Zephyr third test case (BDD)]]></name>
            <objective><![CDATA[BDD Test Objective]]></objective>
            <owner>JIRAUSER10700</owner>
            <precondition><![CDATA[BDD Objective]]></precondition>
            <priority><![CDATA[Low]]></priority>
            <status><![CDATA[Draft]]></status>
            <parameters/>
            <testScript type="bdd">
                <details><![CDATA[Given browser is opened to login page
When user "<username>" logs in with password "<password>"
Then error page should be open

Examples:
    | username  | password |
    | invalid   |   mode   |
    | demo      | invalid  |
    | invalid   | invalid  |]]></details>
            </testScript>
            <updatedBy>Cristiano</updatedBy>
            <updatedOn>2023-08-28 11:17:09 UTC</updatedOn>
        </testCase>
        <testCase id="2" key="CE-T2">
            <attachments/>
            <confluencePageLinks/>
            <createdBy>Cristiano</createdBy>
            <createdOn>2023-08-28 11:14:07 UTC</createdOn>
            <customFields/>
            <folder><![CDATA[Test Cases/Plain]]></folder>
            <issues>
                <issue>
                    <key>CE-28</key>
                    <summary><![CDATA[As a user, I can see comic books from the listing pages]]></summary>
                </issue>
            </issues>
            <labels/>
            <name><![CDATA[Zephyr second test case (Plain text)]]></name>
            <objective><![CDATA[Second test Objective]]></objective>
            <owner>JIRAUSER10700</owner>
            <precondition><![CDATA[Zephyr precondition field<br /><br />Bullet list:<br /><br /><ul><li><strong>Bold</strong></li><li><em>Italic</em></li><li><u>Underline</u></li></ul>Colors: <span style="color:rgb(44, 130, 201)">Blue</span> and <span style="color:rgb(184, 49, 47)">Red</span><br /><br />Table:<br /><br /><table><tbody><tr><td><strong>Column1</strong></td><td><strong>Column2</strong></td><td><strong>Column3</strong></td></tr><tr><td>Cell1</td><td>Ceel2</td><td>Cell3</td></tr></tbody></table><br /><br />Normal Text]]></precondition>
            <priority><![CDATA[High]]></priority>
            <status><![CDATA[Draft]]></status>
            <parameters/>
            <testScript type="plain">
                <details><![CDATA[Description of the plain text test<br /><br /><strong>Bold</strong><br /><em>Italic</em><br /><u>Underline</u><br /><br />Colors: <span style="color:rgb(247, 218, 100)">Yellow</span> and <span style="color:rgb(147, 101, 184)">Purple</span><br /><br />Table:<br /><br /><table style="width:100%"><tbody><tr><td style="width:33.3333%"><strong>Column1</strong></td><td style="width:33.3333%"><strong>Column2</strong><br /></td><td style="width:33.3333%"><strong>Column3</strong><br /></td></tr><tr><td style="width:33.3333%">Cell1</td><td style="width:33.3333%">Cell2</td><td style="width:33.3333%">Cell3</td></tr></tbody></table><br />]]></details>
            </testScript>
            <updatedBy>Cristiano</updatedBy>
            <updatedOn>2023-08-28 14:51:02 UTC</updatedOn>
        </testCase>
        <testCase id="1" key="CE-T1">
            <attachments/>
            <confluencePageLinks/>
            <createdBy>Cristiano</createdBy>
            <createdOn>2023-08-28 11:05:34 UTC</createdOn>
            <customFields/>
            <folder><![CDATA[Test Cases/Steps]]></folder>
            <issues>
                <issue>
                    <key>CE-2</key>
                    <summary><![CDATA[Checkout workflow]]></summary>
                </issue>
            </issues>
            <labels/>
            <name><![CDATA[Zephyr Scale Test Case (Steps)]]></name>
            <objective><![CDATA[Zephyr Scale Test Case (Steps) Objective]]></objective>
            <owner>JIRAUSER10700</owner>
            <precondition><![CDATA[Zephyr Scale Test Case (Steps) - <strong>Precondition</strong>]]></precondition>
            <priority><![CDATA[Normal]]></priority>
            <status><![CDATA[Draft]]></status>
            <parameters/>
            <testScript type="steps">
                <steps>
                    <step index="0">
                        <customFields/>
                        <description><![CDATA[First step description<br /><br />Bullet list:<br /><br /><ul><li><strong>Bold</strong></li><li><em>Italic</em></li><li><u>Underline</u></li></ul><br /><table style="width:100%"><tbody><tr><td style="width:50.0000%">Column1</td><td style="width:50.0000%">Column2</td></tr><tr><td style="width:50.0000%">Cell1</td><td style="width:50.0000%">Cell2</td></tr></tbody></table><br />Normal text.]]></description>
                        <expectedResult><![CDATA[Expected Result<br /><br /><br />]]></expectedResult>
                        <testData><![CDATA[Test Data field]]></testData>
                    </step>
                    <step index="1">
                        <customFields/>
                        <description><![CDATA[Second step (description)<br /><br />Image link below:<br /><img src="../rest/tests/1.0/attachment/image/1" style="width:300px" class="fr-fil fr-dib" />]]></description>
                        <expectedResult><![CDATA[Expected result]]></expectedResult>
                        <testData><![CDATA[Data used in second step<br /><br /><table style="width:100%"><tbody><tr><td style="width:50.0000%">Column</td><td style="width:50.0000%">Column2</td></tr></tbody></table><br />]]></testData>
                    </step>
                </steps>
            </testScript>
            <updatedBy>Cristiano</updatedBy>
            <updatedOn>2023-08-28 14:53:05 UTC</updatedOn>
        </testCase>
    </testCases>
</project>


To import into Xray, we created one script to convert this XML file into a CSV-compatible file to be imported into Xray.

The script will parse the XML file and create a CSV file that can be imported into Xray. To execute the script, use the following command:

python3 zephyrscale2Xray.py -i zephyr-scale-tests-export.xml -o zephyr-scale-tests-export.csv


Another option is available for the script: -e that receives the ZephyrScale Endpoint that will be concatenated with the link produced by the Zephyr Scale links exported.

python3 zephyrscale2Xray.py -i zephyr-scale-tests-export.xml -o zephyr-scale-tests-export.csv -e https://zephyrscale.com/


For now, Xray only supports manual Tests to be imported so that the script will filter out all other test types; in this case, the BDD is filtered out, and we only have the plain text and the step-by-step test case in the CSV output.

The output file has all the test steps.

zephyr-scale-tests-export.csv 
Issue ID,Issue Key,Test Type,Test Summary,Test Priority,Action,Data,Result,Test Repo,Issue Type,Unstructured Definition,Labels,Component,Description,Links
1,,Manual,Zephyr second test case (Plain text),2,,,,Test Cases/Plain,,,,,"*Objective*: Second test Objective
*Details*: Description of the plain text test  
  
*Bold*  
_Italic_  
Underline  
  
Colors: Yellow and Purple  
  
Table:  
  



| *Column1* | *Column2* | *Column3* |
| Cell1 | Cell2 | Cell3 |",CE-28
2,,Manual,Zephyr Scale Test Case (Steps),3,"First step description  
  
Bullet list:  
  
* *Bold*
* _Italic_
* Underline

  



| Column1 | Column2 |
| Cell1 | Cell2 |

  
Normal text.",Test Data field,"Expected Result  
  
  
",Test Cases/Steps,,,,,Zephyr Scale Test Case (Steps) Objective,CE-2
2,,Manual,,3,"Second step (description)  
  
Image link below:  
[Link|https://jirahex.cloud.getxray.app//rest/tests/1.0/attachment/image/1]","Data used in second step  
  



| Column | Column2 |

  
",Expected result,,,,,,Zephyr Scale Test Case (Steps) Objective,


To import it into Xray, we use Test Case Importer with the recently created CSV file; you can use the configuration file provided with the .txt extension (XrayCSVImporter-configuration.txt).


Choose the Project to which you want to import the Tests, and select the option to create Folders as shown below.


When the import operation is complete, you can download the operation details, save the configuration for future use, or check the created issues.


When you click over the link to see the imported issues, we can see that it has created two Tests with all the properties defined in the CSV.


More Details

Let's look into some specificities of this approach and some test particularities.

Test Folder into Test Repository

In Zephyr Scale we can organize our tests into folders and subfolders as we can see below:

As we can see in the above picture we have:

  • one section called 'test cases' with 3 sub-folders
    • Plain
    • Steps
    • BDD

Zephyr Scale folders and sub-folders are exported into the XML file using the folder element in the following form:

test section in atm-exporter.xml 
		...
		<testCases>
        <testCase id="3" key="CE-T3">
            <attachments/>
            <confluencePageLinks/>
            <createdBy>Cristiano</createdBy>
            <createdOn>2023-08-28 11:15:59 UTC</createdOn>
            <customFields/>
            <folder><![CDATA[Test Cases/BDD]]></folder>
            <issues/>
			...
		<testCase id="2" key="CE-T2">
            <attachments/>
            <confluencePageLinks/>
            <createdBy>Cristiano</createdBy>
            <createdOn>2023-08-28 11:14:07 UTC</createdOn>
            <customFields/>
            <folder><![CDATA[Test Cases/Plain]]></folder>
			...
		<testCase id="1" key="CE-T1">
            <attachments/>
            <confluencePageLinks/>
            <createdBy>Cristiano</createdBy>
            <createdOn>2023-08-28 11:05:34 UTC</createdOn>
            <customFields/>
            <folder><![CDATA[Test Cases/Steps]]></folder>
		...


After running the script that converts the XML file into a CSV format we can see that it adds, in the Test Repo field, the Test Repository name to be created when uploading the file into Xray.

Test Repo in comic_estore.csv 
Issue ID,Issue Key,Test Type,Test Summary,Test Priority,Action,Data,Result,Test Repo,Issue Type,Unstructured Definition,Labels,Component,Description,Links
1,,Manual,Zephyr second test case (Plain text),2,,,,Test Cases/Plain,,,,,"*Objective*: Second test Objective
*Details*:
...
2,,Manual,Zephyr Scale Test Case (Steps),3,"First step description  
  
Bullet list:  
  
* *Bold*
* _Italic_
* Underline

  



| Column1 | Column2 |
| Cell1 | Cell2 |

  
Normal text.",Test Data field,"Expected Result  
  
  
",Test Cases/Steps,,,,,Zephyr Scale Test Case (Steps) Objective,CE-2

...


After importing into Xray, we can see that new directories were created in the Test Repository:

Traceability Issues into Requirement references

Zephyr Scale allows you to insert linked issues in the Traceability  tab under Coverage, as we can see below:


Any test case from Zephyr Scale that has these references are exported into the XML into a issues field that can be composed by several issues. The exported XML has the following output:

Manual Tests in atm-exporter.xml 
...
<testCase id="2" key="CE-T2">
            <attachments/>
            <confluencePageLinks/>
            <createdBy>Cristiano</createdBy>
            <createdOn>2023-08-28 11:14:07 UTC</createdOn>
            <customFields/>
            <folder><![CDATA[Test Cases/Plain]]></folder>
            <issues>
                <issue>
                    <key>CE-28</key>
                    <summary><![CDATA[As a user, I can see comic books from the listing pages]]></summary>
                </issue>
            </issues>
            <labels/>
...


After running the script we have the referenced issues in the Links field separated by ";". The CSV output is the following:

Manual Tests in comic_estore.csv 
Issue ID,Issue Key,Test Type,Test Summary,Test Priority,Action,Data,Result,Test Repo,Issue Type,Unstructured Definition,Labels,Component,Description,Links
1,,Manual,Zephyr second test case (Plain text),2,,,,Test Cases/Plain,,,,,"*Objective*: Second test Objective
*Details*: Description of the plain text test  
  
*Bold*  
_Italic_  
Underline  
  
Colors: Yellow and Purple  
  
Table:  
  



| *Column1* | *Column2* | *Column3* |
| Cell1 | Cell2 | Cell3 |",CE-28
...


Once imported into Xray, the issues are added to the Linked issues.

Ensure you have the right permissions to link the issues referenced; otherwise, the import will fail.


Manual Tests (With Steps separated and in Text) into Manual Test Case

Zephyr Scale allows you to create Manual Test Cases with steps (using steps, test data, and expected results) or using only a text field to describe the test.

We have created one test case using the steps in a text field and one using the steps separated into fields. The XML output is the following:

Manual Tests in atm-exporter.xml 
...
    <testCases>
        ...
        <testCase id="2" key="CE-T2">
            <attachments/>
            <confluencePageLinks/>
            <createdBy>Cristiano</createdBy>
            <createdOn>2023-08-28 11:14:07 UTC</createdOn>
            <customFields/>
            <folder><![CDATA[Test Cases/Plain]]></folder>
            <issues>
                <issue>
                    <key>CE-28</key>
                    <summary><![CDATA[As a user, I can see comic books from the listing pages]]></summary>
                </issue>
            </issues>
            <labels/>
            <name><![CDATA[Zephyr second test case (Plain text)]]></name>
            <objective><![CDATA[Second test Objective]]></objective>
            <owner>JIRAUSER10700</owner>
            <precondition><![CDATA[Zephyr precondition field<br /><br />Bullet list:<br /><br /><ul><li><strong>Bold</strong></li><li><em>Italic</em></li><li><u>Underline</u></li></ul>Colors: <span style="color:rgb(44, 130, 201)">Blue</span> and <span style="color:rgb(184, 49, 47)">Red</span><br /><br />Table:<br /><br /><table><tbody><tr><td><strong>Column1</strong></td><td><strong>Column2</strong></td><td><strong>Column3</strong></td></tr><tr><td>Cell1</td><td>Ceel2</td><td>Cell3</td></tr></tbody></table><br /><br />Normal Text]]></precondition>
            <priority><![CDATA[High]]></priority>
            <status><![CDATA[Draft]]></status>
            <parameters/>
            <testScript type="plain">
                <details><![CDATA[Description of the plain text test<br /><br /><strong>Bold</strong><br /><em>Italic</em><br /><u>Underline</u><br /><br />Colors: <span style="color:rgb(247, 218, 100)">Yellow</span> and <span style="color:rgb(147, 101, 184)">Purple</span><br /><br />Table:<br /><br /><table style="width:100%"><tbody><tr><td style="width:33.3333%"><strong>Column1</strong></td><td style="width:33.3333%"><strong>Column2</strong><br /></td><td style="width:33.3333%"><strong>Column3</strong><br /></td></tr><tr><td style="width:33.3333%">Cell1</td><td style="width:33.3333%">Cell2</td><td style="width:33.3333%">Cell3</td></tr></tbody></table><br />]]></details>
            </testScript>
            <updatedBy>Cristiano</updatedBy>
            <updatedOn>2023-08-28 14:51:02 UTC</updatedOn>
        </testCase>
        <testCase id="1" key="CE-T1">
            <attachments/>
            <confluencePageLinks/>
            <createdBy>Cristiano</createdBy>
            <createdOn>2023-08-28 11:05:34 UTC</createdOn>
            <customFields/>
            <folder><![CDATA[Test Cases/Steps]]></folder>
            <issues>
                <issue>
                    <key>CE-2</key>
                    <summary><![CDATA[Checkout workflow]]></summary>
                </issue>
            </issues>
            <labels/>
            <name><![CDATA[Zephyr Scale Test Case (Steps)]]></name>
            <objective><![CDATA[Zephyr Scale Test Case (Steps) Objective]]></objective>
            <owner>JIRAUSER10700</owner>
            <precondition><![CDATA[Zephyr Scale Test Case (Steps) - <strong>Precondition</strong>]]></precondition>
            <priority><![CDATA[Normal]]></priority>
            <status><![CDATA[Draft]]></status>
            <parameters/>
            <testScript type="steps">
                <steps>
                    <step index="0">
                        <customFields/>
                        <description><![CDATA[First step description<br /><br />Bullet list:<br /><br /><ul><li><strong>Bold</strong></li><li><em>Italic</em></li><li><u>Underline</u></li></ul><br /><table style="width:100%"><tbody><tr><td style="width:50.0000%">Column1</td><td style="width:50.0000%">Column2</td></tr><tr><td style="width:50.0000%">Cell1</td><td style="width:50.0000%">Cell2</td></tr></tbody></table><br />Normal text.]]></description>
                        <expectedResult><![CDATA[Expected Result<br /><br /><br />]]></expectedResult>
                        <testData><![CDATA[Test Data field]]></testData>
                    </step>
                    <step index="1">
                        <customFields/>
                        <description><![CDATA[Second step (description)<br /><br />Image link below:<br /><img src="../rest/tests/1.0/attachment/image/1" style="width:300px" class="fr-fil fr-dib" />]]></description>
                        <expectedResult><![CDATA[Expected result]]></expectedResult>
                        <testData><![CDATA[Data used in second step<br /><br /><table style="width:100%"><tbody><tr><td style="width:50.0000%">Column</td><td style="width:50.0000%">Column2</td></tr></tbody></table><br />]]></testData>
                    </step>
                </steps>
            </testScript>
            <updatedBy>Cristiano</updatedBy>
            <updatedOn>2023-08-28 14:53:05 UTC</updatedOn>
        </testCase>
    </testCases>
</project>
...


After running the script that converts this XML into a CSV file we can see that each Test is created and the steps filled with the information extracted from the XML.

Manual Tests in comic_estore.csv 
Issue ID,Issue Key,Test Type,Test Summary,Test Priority,Action,Data,Result,Test Repo,Issue Type,Unstructured Definition,Labels,Component,Description,Links
1,,Manual,Zephyr second test case (Plain text),2,,,,Test Cases/Plain,,,,,"*Objective*: Second test Objective
*Details*: Description of the plain text test  
  
*Bold*  
_Italic_  
Underline  
  
Colors: Yellow and Purple  
  
Table:  
  



| *Column1* | *Column2* | *Column3* |
| Cell1 | Cell2 | Cell3 |",CE-28
2,,Manual,Zephyr Scale Test Case (Steps),3,"First step description  
  
Bullet list:  
  
* *Bold*
* _Italic_
* Underline

  



| Column1 | Column2 |
| Cell1 | Cell2 |

  
Normal text.",Test Data field,"Expected Result  
  
  
",Test Cases/Steps,,,,,Zephyr Scale Test Case (Steps) Objective,CE-2
2,,Manual,,3,"Second step (description)  
  
Image link below:  
[Link|https://jirahex.cloud.getxray.app//rest/tests/1.0/attachment/image/1]","Data used in second step  
  



| Column | Column2 |

  
",Expected result,,,,,,Zephyr Scale Test Case (Steps) Objective,


The first Test corresponds to a Manual Test Case with the summary obtained from the CSV, and the description combines the objective and the description extracted from the CSV.

In Zephyr Scale the test described in plain text does not have steps so the Test in Xray will be a manual step without any steps defined. 


The second Test has one line per step. Notice that the format is kept for the table and styles added in Zephyr Scale.

Modular test cases from Zephyr Scale are exported with the steps unfolded and no reference to the original test case. 

Tests with data sets defined in Zephyr scale are imported without the data set.


Other Tests

The script will convert all tests of type Plain Text and Step-by-Step; all the other types will be discarded, and no output will be created in the resulting CSV file.


Tips

  • Use the CSV file provided to import the examples and the associated JSON configuration to have the setup and mapping correct.
  • Ensure you are using the scripts available for the right version of Jira (Cloud or Server).