A Test Set is a collection of Tests. It often contains detailed instructions or goals as well as information on the configuration to be used during testing.

Test Sets are simple, flat lists of Tests that you can use as a basis for creating Test Executions or Test Plans. Since the Tests are grouped in some logical way, such as grouping all Tests related with regression testing or security, or all Tests for a specific component or some high-level feature/business case, a Test can be part of different Test Sets. Test Sets can also be used as a dynamic way to cover requirements.


Learn More

Do you need to understand the differences between using Tests Sets & Test Repository to organize your Tests? Check "Test Repository versus Test Sets" article.


Actions

Create

To create a new Test Set issue:

Step 1: Click the Create at the top of the screen to open the Create issue dialog / page.

Step 2: Select the Project and on Issue Type, select Test Set.

Step 3: Type a Summary for the Test Set and complete all appropriate fields — at least, the required ones marked by an asterisk.

Step 4: When you are satisfied with the content of your Test Set, click the Create button.
 If you selected the Create another checkbox (below), a new Create Issue dialog appears. This issue is automatically pre-populated with your previous Test Set details, while leaving the Summary field blank.

Edit

To edit an existing Test Set issue:

Step 1: Locate the Test Set you want to edit.

Step 2: Place the cursor over the field to edit and click on it.

Step 3: Modify your Test Set's detail in the appropriate field

Step 4: Click out of the field to save your changes.


Clone

When cloning a Test Set issue, all the Test issue associations will also be cloned.

To clone a Test Set issue, select the "Clone" action in the issue page:

After this, an additional dialog will be shown.

And after the issue is created, its page will be opened.

On some occasions the cloned issue page may open before the Xray information is actually cloned and as a result the new Test will appear empty.

When this happens the following information will be shown:

By clicking "Refresh issue", the issue page will be reloaded and the cloned information will be visible.

Configure Test Columns

Xray provides the ability to configure columns for the Tests table. This configuration is specific to each user and can be restored to the default configuration defined in the Default Column Layouts page in the Xray app project settings.

To configure columns for the Tests table,

Step 1: Open the Test Set issue.

Step 2: Click on Columns, select the dropdown that appears in the Tests section above the table in the top-right corner. A list of the current configure columns and all available columns appears.

Step 3: Add or remove columns by clicking the checkboxes next to each field.

Step 4: Click Done when finished or Cancel to abort this operation. The table refreshes with the configured columns.


Restoring Columns to Default Configuration

To restore columns in the Tests table to the default configuration,

Step 1: Open the Test Set issue.

Step 2: Click Columns, select the dropdown that appears in the Tests section, above the table in the top-right corner. A list of the current configure columns and all available columns appears.

Step 3: Click the Restore Columns link above the columns list. The table refreshes with the default configured columns.

Change Columns Order

To reorder columns in the Tests table,

Step 1: Open the Test Set issue.

Step 2: Hover over the desired column header.

Step 3: Click in the column header and drag it to the desired position.

Associate Tests

Tests can be associated with Test Set issues:

From the Test Set issue view screen

Create Test

To create a Test directly associated to the current Test Set from the Test Set issue view screen:

Step 1: Open the Test Set issue view screen you wish to create a Test with.

Step 2: Click the Add Tests button (under the Tests section on the view page) and select New Test to open the Create Issue dialog box.

Step 3: Type a Summary for the Test and complete all appropriate fields — at least, the required ones marked by an asterisk.

Step 4: Click Create.

Associate Test

To associate Tests with a Test Set from the Test Set issue view screen:

Step 1: Open the Test Set you wish to associate a Test with.

Step 2: Click the Add Tests button (under the Tests tab on the view page) and select Existing Tests to open the Add Tests dialog box.

Step 3: Select or search the Test issues to be associated with the Test Set:

To select the Tests to be associated, you can, under the Select tab:

- Input the desired Test Issue Key on the Tests field.
- Click on the Down Arrow on the Tests field and select the Test from its History Search list.

To search for Tests to be associated, you can, under the Search tab:

- Select the desired Project and any other desired field such as Test Type, Contains, Label, Fix Version, Component, Covering, Covering Fix Version, Saved Filters and Workflow Status

- Click on the Search button after filling the browse fields to get a list of matching entries


To search for Tests to be associated, you can, under the JQL tab:

- Write the desired JQL search

- Click on the Search button perform the JQL search and to get a list of the matching entries (more details about JQL here)  


Step 4: Click Add.

From the Test issue view screen

To associate Tests with a Test Set from the Test issue view screen:

Step 1: Open the Test you wish to associate a Test Set with.

Step 2: Click the Add Test Sets button (under the Test Sets tab on the view page) and select Existing test sets to open the Add Test Sets dialog box.


(Note: to create a new test set click New test set.)

Step 3: Select or search the Test Set issues to be associated with the Test Set:

To select the Tests Sets to be associated, you can:

- Input the desired Test Set Issue Key on the Tests Sets field.
- Click on the Down Arrow on the Tests Sets field and select the Test Set from its History Search list.

To search for Test Sets to be associated, you can, under the Search tab:

- Select the desired Test Set Project and any other desired field such as Test Type, Contains, Label, Fix Version, Component, Covering, Covering Fix Version, Saved Filters and Workflow Status
- Click on the Search button after filling the browse fields to get a list of matching entries

To search for Test Sets to be associated you can: under the JQL tab:

- Write the desired JQL search

- Click on the Search button perform the JQL search and to get a list of the matching entries (more details about JQL here)


Step 4: Click Add.

Remove Tests

Tests can be removed from Test Set issues:

From the Test Set issue view screen

        Step 1: Open the Test Set from which you want to remove a Test.

Step 2: On the Tests table, click the ... on the desired Test row and click the Delete option that appears in the drop down menu. You will be prompted with a confirmation dialog.

From the Test issue view screen

Step 1: Open the Test you want to remove the Test Set association.

Step 2: On the Tests Sets table, click the ... on the desired Test Set row and click the Delete option that appears in the drop down menu. You will be prompted with a confirmation dialog.

Reorder Tests

There are two ways of changing the order of Tests within a Test Set:

Drag-and-Drop

To reorder Tests,

Step 1: Open the Test Set issue.

Step 2: Hover over the desired Test row (under the Tests section on the view page) and the drag handle column.

Step 3: Click the drag handle and move the Test up or down in the current page.

Change Rank Action

This action allows users to change the Rank of a Test between pages.

To change the rank of a Test,

Step 1: Open the Test Set issue.

Step 2: Click the ... on the desired Test Set row and click on the Change Rank option that appears in the drop down menu

Step 3: Click the Move To action. The change rank dialog appears.

Step 4: Enter the desired index number (1-based) and click OK.









Bulk Operations

Xray provides the following bulk operations on the Tests table: 

  • Remove - Deletes all the selected Tests associations with a Test Set.

To execute this bulk action:

Step 1: Open the Test Set issue.

Step 2: A checkbox should be displayed on each Test row and also the bulk actions button above the tests table.

Step 3: Select the desired Test issues.

Step 4: Click the bulk actions button located above the tests table and choose one of the available actions.

Export to Cucumber

Cucumber Test Sets in Jira can be exported to feature files ready to be executed in Cucumber. The generated feature files follow the rules defined in Generate Cucumber Features and take into account the context of the Jira Test Set issue.

This action will prompt a FeatureBundle zip file containing a single Cucumber feature file for every Requirement associated with the target Test, with a test type with kind Gherkin, contained on the Test Set, to be generated and downloaded. 

To export a Test Set issue containing Cucumber Tests to feature files, 

Step 1: Open the Test Set issue view page. 

Step 2: Select ... > Xray - Export to Cucumber. The file will be downloaded from the browser.

Xray History

Any changes made to the Test Set data are recorded on the Xray History section. These include:

  • Linking the Test with other entities (Test)