The purpose of a Test Plan is to enable you to have better visibility and management over your Tests.
It provides a table that shows all the Tests added to the Test Plan and their respective Test Executions that are also associated with the Test Plan.
A Test Plan can be defined as:
- Static: by adding Tests explicitly to the Test Plan issue using the Xray issue picker.
- Dynamic: (Xray Enterprise only) by defining a saved filter as the source of Tests for the Test Plan. This way, all Tests on the filter will be added automatically into the Test Plan by Xray. For example, you can define a Test Plan with a filter containing the Tests associated with Stories from a Sprint. Your Test Plan will collect all relevant Tests for that Sprint automatically. If a Test Plan is configured with a saved filter, users can't add or remove Tests explicitly from the Test Plan.
Dynamic Test Plan is a Xray Enterprise Feature
Dynamic Test Plans is a feature of Xray Enterprise. If you do not have Xray Enterprise installed, the Dynamic Test Plans are not available in the Test Plan issue, and it is impossible to configure dynamic test lists. When installing Xray Enterprise for the first time, please make sure to re-enable Xray to load all properties from Xray Enterprise.
If tracking a particular version, the Test Plan will present the latest status for each Test, independent of the number of testing iterations (i.e., Test Executions) you make with them.
You can group a bunch of Test Executions and easily give feedback on the current/latest status of the Tests involved in those Test Executions.
From the Test Plan, it's possible to create one or more Test Executions for all or just a subset of the Tests contained within the Test Plan (e.g., only those Tests that are still failing).
You can also assign one or more Test Plans to a given release and track the progress of those tests within each Test Plan.
The coverage analysis can also be done considering the tests from a given Test Plan and its related Test Executions.
This is a list of actions that you can perform on Test Plans:
Create Test Plan
To create a new Test Plan Issue,
Step 1: Click the Create Issue at the top of the screen to open the Create Issue dialog box/page.
Step 2: Select the Project, and on Issue Type, select Test Plan.
Step 3: Under the General tab, type a Summary for the test execution and fill in all appropriate fields — mandatory fields are marked with an asterisk. If you're working with versions, you should also fill out the Fix Version to indicate that you're tracking the results of the Tests for that version. This will be used to automatically fill out the Fix Version of the planned Test Executions (i.e., the ones linked to/created from this Test Plan).
Step 4: Under the Test Plan Details tab, you can populate the Test Plan custom fields provided by Xray, namely the Begin/End date. In this tab, you can also add tests to the Test Plan.
Step 4a – In this tab, you can select which Tests should be associated with your Test Plan. For a static Test Plan, you should select the Tests at the Test associated with a Test Plan. If you have Xray Enterprise, you can select an existing Saved Filter by name on the Test Plan Tests Filter and create a Dynamic Test Plan.
How can I create a Saved Filter?
Saved filters are one of the most powerful Jira features that can be used to help you find the information you need, but they also can be used in gadgets, reports, or even subscriptions. As Xray is seamlessly integrated with Jira, you can use this feature to create your Dynamic Test Plans. Learn more about Saved Filters at Jira Documentation.
Optional: To create a series of similar Test Plans – within the same Project – select the Create another checkbox at the bottom of the dialog.
Step 5: When you are satisfied with the content of your Test Plan, click the Create button.
Step 5a: If you selected the Create another checkbox, a new Create Issue dialog appears. This issue is automatically pre-populated with your previous Test Plan details while leaving the Summary field blank.
Add Tests to a Test Plan
Aside from adding Tests to the Test Plan using the issue create screen, you can also do it on the Test issue page or directly in the tests section of the Test Plan issue page as follows.
Step 1: Click Add.
Step 2: Click the Tests... option to open the Issue Picker Dialog.
- Note that if you choose Test Sets, you are not actually adding Test Set(s); instead, you are adding the Tests that belong to that Test Set(s).
If you want to create a Dynamic Test Plan (Xray Enterprise only) you need to Edit your Test Plan to add the Tests from Saved Filter or you can click on ... (configure) to change the nature of your Test Plan. Dynamic Test Plans are not available from the "Add" button.
Step 3: Open the Search tab.
How to use de Issue Picker Dialog
This is an example of how to add tests, using some of the features available in the Search tab.
For more information, please visit the Issue Picker Dialog detailed overview.
Step 4: Configure the search.
-Use the Filters dropdown button to include extra fields in the search criteria.
-Fill in the filters values.
-Press the search button.
Step 5: Select and Add tests.
- Once the results are obtained, you can order and add/remove columns to the results list (the columns selection is saved as a preference).
- Check the tests you wish to add from the listed results.
Step 6: Add tests by pressing "Add selected."
Ranking of Tests within the Test Plan can be done in two ways: directly in the Test Plan issue screen (if the Hierarchical Test Organization is disabled) or in the Test Plan Board (if Hierarchical Test Organization is enabled). This means you can rank Tests, whether you use hierarchies or not.
If you use Hierarchical Test Organization, you can and have to rank within the Test Plan Board. More information is here.
The following procedure details how you can change the order of the Tests directly in the Test Plan issue screen.
There are two ways of changing the order of Tests within a Test Plan:
To reorder Tests,
Step 1: Open the Test Plan issue.
Step 2: Hover over the desired Test row (under the Tests section on the view page) and the drag handle column.
Step 3: Click the drag handle and move the Test up or down on the current page.
Change Rank Action
This action allows users to change the Rank of a Test between pages.
To change the rank of a Test,
Step 1: Open the Test Plan issue.
Step 2: Hover over the desired Test (under the Tests section on the view page)
Step 3: Click the gear that appears in the last column. The actions menu should pop up with the available actions.
Step 4: Click the Move To action. The change rank dialog appears.
Step 5: Enter the desired index number (1-based) and click OK.
Associate Test Executions to the Test Plan
Since the table only shows Test Runs for tests that are in the Test Plan and for Test Executions that are associated with the Test Plan, it's necessary to associate Test Executions with the Test Plan.
You can also add existing Test Execution issues directly to a Test Plan.
Test Executions can also be associated with a Test Plan by editing the custom field Test Plan in the Test Execution issue view page or directly in the Create issue screen in the Test Execution Details tab. Note that with this method, only the Test Execution shall be associated with the Test Plan; the Tests will not be associated.
Test Plan table
The Test Plan Test view section on the issue page provides you with a table where you can see all the Tests that were added to the Test Plan and their Test Executions (if they are associated with the Test Plan).
- Add Tests (directly or from Test Sets) (Not available if configured as Dynamic Test Plan)
- Create planned Test Execution (see next Section), so its results be reflected on the current Test Plan
- Test Plan Progress Bar
- Table Filters
- Use a Test Environment to calculate the status for test issues for that environment.
- Configure Test Plan with Saved Filter (see Section Configure Test Plan with Saved Filter) (Only available in Xray Enterprise)
It is possible to filter tests by status in Test Plan Progress Bar as shown in the image below:
Each row represents a Test with configurable columns, with the exception of the Latest Status. Each row also contains an inner table with information about the Test Execution. Click on the blue arrow to see the inner table for a particular test.
This inner table is similar to the Test Run table as it allows for inline execution ( 1 ) and for operations like assignee and remove ( 2 ). This inner columns are not configurable.
Create Test Executions from the Test Plan
There are several ways to create Test Executions directly in the Test Plan issue page.
Create planned Test Execution
The most common and straightforward option to create a Test Execution for all the Tests, or for a subset of those that are currently in some consolidated status, is by using the option available at the top of the Test Plan issue screen.
The Create Test Execution button will open a dropdown with some options that will allow you to select the Tests that will be part of the Test Execution.
In order to create a related Test Execution from the Test Plan, the user must be allowed to the edit the current Test Plan since the results on the Test Execution will affect the consolidated information shown at the Test Plan. The user will only be able to create Test Executions in certain projects, therefore the list of projects shown in the creation dialog will be filtered out accordingly.
Lets you create a Test Execution with all the tests that are associated with the Test Plan.
Note: If the revision field is not configured in the create issue screen of the Test Execution for the selected Project, the field will not appear in the dialog. If there are some mandatory fields, the standard create issue screen will be shown.
Choose the Tests based on their consolidated status in a specific Test Environment. You can choose all Tests using the top left-side option; the Test Environment can be chosen in the top right-side option. For all shown status, you can see the amount of Tests that are currently in that consolidated status. You may choose one or more statuses. In the example below, we will be including just the (1) Test that is currently failing.
You can select some Tests, using the bulk selection (1), and then schedule a Test Execution for them.
Dynamic Test Plan - If the Test Plan is configured with saved filter, it is not possible use the bulk selection.
Clone Test Plan and respective Board
A Test Plan along with its respective hierarchical information (i.e. the Board folder structure) can be cloned.
The new Test Plan will be created in a clean state, containing the original Tests without any execution related information. If hierarchical test organization, the Board folder structure is also cloned.
Configure Test Plan with Saved Filter (Xray Enterprise only)
Xray Enterprise Feature
Configure Test Plan with a Saved Filter is a feature of Xray Enterprise. If you do not have Xray enterprise installed, the Configure Test Plan with a Saved Filter are not available in the Test Plan and it is not possible to configure dynamic test lists. When installing Xray enterprise for the first time, make sure to re-enable Xray to load all properties from Xray enterprise.
In addition to being able to explicitly add tests to the Test Plan, you can use a filter and you can define a custom JQL depending on your needs so that tests will appear automatically on the Test Plan, as follows.
Step 1: Click on dropdown menu button with a "more" icon and then click Configure.
Step 2: Click on option Saved Filter.
- A link button named "View Filter Query" is provided to navigate to the Saved Filter Jira issue page.
- A link button named "New Filter" is provided next to the Saved Filter select field. This button open the Jira issue search page.
Step 3: Select a query from the select field.
Step 4: Click Save button to save the Test Plan configuration.
- Xray update the Tests in the Test Plan automatically and the Tests panel refreshed
How can I create a Saved Filter?
Saved filters are one of the most powerful Jira features, that can be used to help you find the information you need, but also can be used into gadgets, reports, or even subscriptions. As Xray is seamlessly integrated with Jira, you can take advantage of this feature to create your Dynamic test Plans. Learn more about Saved Filters at Jira Documentation.
How does Xray update dynamic Test Plans based on a Jira saved filter?
Xray will update the Test Plan Tests whenever the user enters the Test Plan issue page.
Nevertheless, to keep Test Plan up to date with current results from the JQL filter even if users don't visit the Test Plan issue page, a dedicated service will be run in the background every two hours. This interval can be modified in the Services section of the Jira system administration.
Xray will also update the Test Plan immediately whenever the Test Plan is accessed through the REST API.
Using saved filters to create Dynamic Test Plans will avoid users to delete or add Tests from your Test Plan, as well to change the ranking. If you need to make changes at your dynamic test plan list you need to update the filter settings.