Test Plans can be visualized and manipulated from the Testing Board.
The Test Plan Board is a Test Plan-level test organization feature that allows you to organize Tests in a hierarchical way inside folders. Always keep in mind that each Test Plan has an associated (Test Plan) Board and that a Test can only belong to one folder within the Board.
This page will help you understand the Test Plan Board screen and the available operations within it, such as the ability to easily create Test Executions from a given folder or from selected Tests.
How to Access it
To access Test Plans in the Testing board,
Overview of the Test Plans View UI
A:
- A1: Filter Test Plans by text.
- A2: Toggle filtering Test Plans assigned to the logged in user or not.
- A3: Use custom filters to filter the Test Plans in the project
- A4: Select field to sort Test Plans by.
- A5: Select ascending or descending order to sort the Test Plans.
B:
- B1: Create a new Test Plan.
- B2: Export the Test Plan list.
C:
- C1: Set the Scope of the status for the progress bar.
- C2: Test Plan link to the respective Test Plan View.
- C3: Test Plan progress bar.
Actions
Create Test Plan
Create a new Test Plan.
Export
Export the Test Plans listed either as a CSV or through the Document Generator.
Overview of the Test Plan View UI
The image below depicts the several key areas of the Test Plan Board screen, which include the Test Plan Board itself, along with its sub-folders.
A: Folders section (master)
- A1: the Test Plan Board folder tree, composed of folders and sub-folders with Tests organized within them
- A2: a given folder; CRUD actions for the selected folder (e.g., create, rename, delete) can be accessed by right-clicking on the folder
- A3: information about the number of Tests within the current folder and the total Tests, including sub-folders
- A4: context actions for the currently selected folder; these actions are also accessible by right-clicking on a folder.
B: Tests section (detail)
- B1: filter for the Tests.
- B2: switch from hierarchical to flat view.
- B3: folder progress.
- B4: one test from a ranked list of Tests within the current folder (only direct children)
- B5: status of the Test.
- B6: button to open a dialog with the Test Runs of the given Test in the context of the Test Plan.
- B7: button to open the dataset defined for the respective Test.
C:
- C1: breadcrumb links for the Testing Board.
- C2: Test Plan summary (can be edited if the issue is editable).
- C3: add Tests to the Test Plan.
- C4: open the Jira issue dialog.
- C5: scope of the status of the Tests.
- C6: generate cucumber feature files and export test plan information through the Document Generator.
- C7: Test Plan fields, can be edit inline if the issue is editable.
- C8: overall progress.
- C9: create a Test Execution in the Test Plan
- C10: Link to the Test Executions linked to the Test Plan
How to reach Test Plan in the Testing board from an issue view
To reach the Test Plan Board UI, you can simply use the "View on Board" link in the issue page of a Test Plan issue.
Test Plan fields
The Fields section presents some information regarding the Test Plan, namely its description, its assignee and its fix versions. This section can be collapsed or expanded.
If the Test Plan is editable, the fields can be edited inline. Otherwise, the fields will be displayed in read only mode.
To edit a field, click on it and it will enter in edit mode. The changes will be saved automatically when exiting the field.
Tracking progress
The Board presents the consolidated results for the Tests that are part of the Test Plan; in other words, you're able to see the latest status for each Test that is being tracked on the Test Plan, regardless of the number of runs you make for it. If you schedule and run multiple test cycles (i.e., Test Executions), the Board will group the results from those executions and will show you the calculated status for each Test.
You can then see the overall execution status of the Test Plan, globally taking into account all the planned Test Executions you create in the different environments.
If you need to drill-down and see the current status for a subset of Tests, you can do it at the folder level.
Overall progress
In order to track the overall progress of the Test Plan, use the top progress bar (C3).
Moving the cursor over the bar elements will give information about the number of Tests (absolute and percentage) that are currently in that status.
The overall execution status takes into account the joint results for all the Test Environments in which you have run your Tests (i.e., a Test will appear as PASS only if it passed in all the Test Environments for which you scheduled Test Executions).
Folder progress
By allowing you to see the progress per folder (B2), you're able to analyze the status of a subset of Tests that are part of the Test Plan.
The analysis per folder takes into account the selected Test Environment; by default, it shows the calculated status for "All Environments."
The values shown for each folder only take into account the immediate child Tests (i.e., the Tests that are within that folder). Thus, this bar shows the totals for the Tests that are seen "underneath" it.
On the right side of each Test (B7), you will be able to see the current calculated status for that Test, in the context of the Test Plan and the selected Test Environment. The status that is shown corresponds to the last run made for that Test in the scope of the related Test Executions, taking into account the selected Test Environment.
Progress analysis
Tests can be analyzed by Test Environment (i.e., taking into account the results for some given Test Environment) using the proper action (B3), so you can see the current testing status for that environment.
This way, you're able to analyze on a per Test Environment basis or see the overall status, if you choose "All Environments." For more information on using Test Environments and the calculation of the status for a given environment, please see Working with Test Environments.
Test Executions
Create Test Execution
You can easily create a Test Execution for the Test Plan by using the Create Test Execution menu.
To create a Test Execution with all the Tests in the Test Plan:
Step 1: Click on Create Test Execution.
Step 2: Click on option All tests....
Step 3: Fill out the Test Execution fields and click Create. The new Test Execution is created with all the Tests in the Test Plan.
To create a Test Execution with Tests filtered by status:
Step 1: Click on Create Test Execution.
Step 2: Click on option With status....
Step 3: Choose one or more statuses to filter the Tests.
Step 4: Fill out the Test Execution fields and click Create. The new Test Execution is created with the Tests filtered by the selected statuses.
View associated Test Execution
The following link displays how many Test Executions are associated with the Test Plan.
You can click on that link to view those Test Executions in the Test Executions board.
View Mode: Flat or Hierarchical
You can choose between the traditional, hierarchical/folders view or a flattened view.
If you choose "Folders View" then in the Tests section (B) you will see only the Tests that are direct children of the folder selected on the left side, within the Folders section (A). In this case, you can only select Tests directly within the currently selected folder.
On the other hand, if you prefer you can switch to a "Flat View"; if so, you'll see on the right side (Tests section) all the Tests that belong to that folder and related sub-folders, grouped by folder. Then you can apply a filter if you want, select some of the Tests (no matter in which sub-folder they're in) and do an operation for them.
Folder Actions
A folder can be selected by clicking on it. Once selected, you can perform context actions or use the icons available at the top of the folders' section.
Note: It is not possible to select multiple folders.
Create
To create a new folder:
Step 1: Click on the parent folder.
Step 2: Click on the add icon at the top of the folders section.
Step 3: Type the name for the folder on the create folder dialog.
Step 4: Press "Create" to create the folder.
Rename
To rename an existing folder:
Step 1: Right-click on the folder you wish to rename.
Step 2: Click "Rename Folder."
Step 3: Type the new folder name on the rename dialog.
Step 4: Press "Rename" to rename the folder.
Remove
You can easily remove an existing folder and its descendant folders.
Tests that were in one of the removed folders will not be erased. Tests will be removed from the Test Plan Board but will appear in the Board folder, so they can be organized in the Test Repository again if needed.
In order to remove a folder:
Step 1: Right-click on the folder you wish to rename.
Step 2: Click on Delete Folder.
Step 3: Confirm the removal of the folder.
Expand/Collapse
Folders can be expanded or collapsed easily just by clicking the folder's icon.
Collapse All
The "collapse all folders" action is available in the top folders section.
In order to collapse all the folders of the Test Repository:
Step 1: Click on the "Collapse All" action in the top folders section.
View Issues
You can view the Test issues, from the selected folder and from its sub-folders, in the issue search screen.
In order to view the Test issues:
Step 1: Right-click on the folder.
Step 2: Click on View Issues.
A new tab will be open that will take you to the search issues page
Moving to another folder
A folder, including its descendants, can be moved to another folder just by dragging-and-dropping to the destination folder.
Dragging the folder directly on top of some existing folder will append it to the list of current child folders.
Ordering / Ranking
You can also move directly to a given position/rank by choosing the interim space between two existing folders.
Note: If you wish to rank it as the last one (i.e. move it to the last position) within some parent folder, you just need to drag it to the parent folder.
Creating a Test
You can create a Test and associate it automatically with the selected folder.
In order to create a Test in a folder;
Step 1: Right-click on the folder.
Step 2: Click on Create Test.
Step 3: Fill out the Test fields and click Create. Your brand-new Test and associated with the selected folder.
Add Tests
You can "add" Tests to an existing folder based on some filters or JQL.
This action corresponds to "moving" them from any folder they may already be in (including the Board) to the destination folder. Thus, if you "add" a Test that is currently in some folder within the Test Repository to some destination folder, then it will essentially be moved from the source folder to the destination folder.
In order to add Tests to a folder,
Step 1: Right-click on the folder.
Step 2: Click on the Add Tests.
Step 3: Choose the Tests you want, using the provided filters, a previously saved filter or by JQL.
Creating a Test Set
You can easily create a Test Set using all the Tests in a given folder and its descendant folders.
The ranking of the Tests within the Test Set will respect the ranking they already have in the Test Repository folder.
In order to create a Test Set with all the Tests in a given folder, including its descendants:
Step 1: Right-click on the folder.
Step 2: Click on the Create Test Set with Tests.
Step 3: Fill out the Test Set fields and click Create. Your brand-new Test Set is created with all the Tests, in the proper order.
Add to Test Set(s)
You can add Tests to an existing Test Set with all the Tests in a given folder and its descendant folders.
The ranking of the Tests within the Test Set will respect the ranking they already have in the Test Repository folder. If Tests already exist in the destination Test Set, then their ranking in the Test Set is not changed.
In order to add Tests from a folder, including its descendants, to some existing Test Set(s):
Step 1: Right-click on the folder.
Step 2: Click on the Add Tests to Test Set.
Step 3: Choose the Test Set(s) key(s) and select Add Selected. The Test Set(s) will now be updated.
Create Test Execution
You can easily create a Test Execution using all the Tests in a given folder and its descendant folders.
The ranking of the Tests within the Test Execution will respect the ranking they already have in the Board folder.
In order to create a Test Execution with all the Tests in a given folder, including its descendants:
Step 1: Right-click on the folder.
Step 2: Click on the Create Test Execution with Tests.
Step 3: Fill out the Test Execution fields and click Create. Your brand-new Test Execution is created with all the Tests, in the proper order.
Add to Test Execution(s)
You can add Tests to an existing Test Execution using all the Tests in a given folder and its descendant folders.
The ranking of the Tests within the Test Execution will respect the ranking they already have in the Board folder. If Tests already exist in the destination Test Execution, then their ranking in the Test Execution is not changed.
In order to add Tests from a folder, including its descendants, to some existing Test Execution(s):
Step 1: Right-click on the folder.
Step 2: Click on the Add Tests to Test Execution.
Step 3: Choose the Test Execution(s) key(s) and select "add selected." Test Execution(s) will now be updated.
Test Actions
Tests can be selected by clicking on them. Multiple selections are possible by using the "Ctrl" key (on Windows) or the "Cmd" key (on OS X). If you want to select more than one Test and the Tests are listed consecutively, use the "Shift" key to quickly select an entire group. Click the first Test in the list you want to select, hold down the "Shift" key, scroll down to the last Test you want and click it. This selects the first item, the last item and all items in between.
You can filter the list of visible Tests to make it easier to pick the Tests you need.
After selecting one or more Tests, context actions can then be done on them, such as creating a Test Set, Test Execution, etc.
Filtering visible Tests
The list of visible Tests can be filtered, so you can easily see and obtain the relevant Tests you need. Note that the filter is applied when you change to another folder.
Filtering Tests is quite useful, for example, when creating the structure of the Test Repository based on a subset of Tests from the Board folder.
Moving to another folder
Tests can be moved to another folder just by dragging-and-dropping to the destination folder.
You can also use this operation for "moving" Tests (i.e., adding Tests) from folder to folder within the Test Repository.
Ordering / Ranking
Tests can be ranked within a folder by using drag-and-drop.
Create a Test Set
Use the action Create Test Set with Tests to create a new Test Set containing the selected Tests.
Add to Test Set(s)
Use the action Add Tests to Test Set to add selected Tests to an existing Test Set.
Create Test Execution
Use the context action Create Test Execution with Tests to create a new Test Execution containing the selected Tests.
Add to Test Execution(s)
Use the context action Add Tests to Test Execution to add selected Tests to an existing Test Execution.
Move to Root
You can move Tests to the Board (Root) folder; this is equivalent to removing them from the Test Repository.
There are 2 different ways to move Tests to the Board folder:
First option:
Step 1: Select the Tests you want.
Step 2: Drag the Tests to the Board folder.
Second option:
Step 1: Select the Tests you want.
Step 2: Use the context action Move to Root.
View Issues
You can use the context action View Issues to view the selected Tests in the search issues page.
Step 1: Select the Tests you want.
Step 2: Use the context action View Issues.
How ranking works when creating or adding Tests to Xray entities
When creating a Test Set or Test Execution, or adding Tests to an existing one, Xray will respect the order of the Tests, taking into account the ranking they have in the Test Plan Board.
The algorithm starts by adding the Tests of the folder, respecting their rank in the folder, and then iterates through the sub-folders. Each sub-folder is processed in the same way.
If a Test already exists in the destination entity (e.g. Test Set/Execution), then it is not added again and the current ranking is maintained.