Page History
...
These custom fields are located in the native Jira sections. The revision can be found in the Details section of the Test Execution issue while the Begin and End dates are located in the Dates section.
| Section | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
|
Sections
Xray contributes an additional section to the Test Execution view issue screen.
...
It is possible to filter tests by status in Test Execution Progress Bar as shown in the image below:
Test Run Filters
A filter section is also present above the Test Runs table to allow you to filter Test Runs by Assignee, Status, Test Set, or open text.
...
The #Def column provides the number of Defects created during the Test Execution or already related to the Test issue. For more information on how the calculation is done for this field, please refer to Defects Count.
Actions
| Table of Contents | ||||
|---|---|---|---|---|
|
...
Step 1: Click the Create Issue at the top of the screen to open the Create Issue dialog box / page.
Step 2: Select the Project and on Issue Type, select Test Execution.
...
Step 5a: If you selected the Create another checkbox, a new Create Issue dialog appears. This issue is automatically pre-populated with your previous test execution details, while leaving the Summary field blank.
Edit
The Edit Issue dialog box / page is similar to the Create Issue dialog box / page, and can be accessed through the following steps:
...
Step 2: Click the Edit button (at the top-left of the 'view issue' page) to open the Edit Issue dialog box / page.
Alternatively, hover over a field and click the pencil icon to edit it inline.
Step 3: Modify your Test Execution details in the appropriate fields using either the General or Test Execution Details tab of the Edit Issue view.
Step 4: Click the Update button to save your changes.
Clone
When cloning a Test Execution issue, all the Test issue associations will also be cloned. However, a new Test Run is created for each Test so that the new Test Execution issue has no execution state. Also:
...
Step 2: Select More > Clone. The Clone Issue screen will appear.
Step 3: Edit the clone Test Set's Summary if you wish.
Step 4: Click Create.
Adding Tests
Tests and Test Sets can be associated with Test Execution issues.
...
Step 2: Select Add > Tests (under the Tests tab on the view page) to open the Add Tests dialog box.from the dropdown button in the Tests section.
Step 3: Select or search the Test issues to be associated with the Test Execution:
Select Test Issues - In order to select the Tests to be associated, you can, under the Select tab:
- Input the desired Test Issue Key on the Tests field.
- Click on the Down Arrow on the Tests field and select the Test from its History Search list.
- Click on the + button wich will prompt an Issue Selector Pop-Up.
- Click on the Search tab to use the Find Jira issues browser.
Search Test Issues - In order to search for Tests to be associated, you can, under the Search tab:
- Browse for the desired Test Project, Type or terms it Contains.
- Click on the More button to add more browse fields such as Label, Test Set, Test Set Fix Version, Requirement, Requirement Fix Version, Saved Filter and Workflow Status.
- Click on the Magnifier button after filling out the browse fields to get a list of matching entries
- Click on the Advanced button to perform a JQL search.
Step 4: Click Add.
Add Tests from Test Sets
When adding a Test Set issue to a Test Execution, all the Test issues associated with the Test Set are added to the Test Execution. The Test Set issue is not directly associated with a Test Execution, so changes to the Test Set after it was added will not affect the Test Execution.
Step 1: Open the Test Execution you wish to associate a Test Set with.
Step 2: Select Add > Tests (under the Tests tab on the view page) to open the Add Test Sets dialog box.
Step 3: Select or search the Test Set issues to be associated with the Test Execution.
Select Test Set Issues - In order to select the Test Sets to be associated, you can, under the Select tab:
- Input the desired Test Set Issue Key on the Test Sets field.
- Click on the Down Arrow on the Test Sets field and select the Test Set from its History Search list.
- Click on the + button wich will prompt an Issue Selector Pop-Up.
- Click on the Search tab to use the Find Jira issues browser.
| Info | ||
|---|---|---|
| ||
Notice the setting for ignoring Tests in non-executable statuses at the bottom-left side. By default it will be selected, therefore restricting the shown Tests available for adding accordingly to the configuration "Disallow executions of Tests with workflow statuses". |
Search Test Sets Issues - In order to search for Test Sets to be associ.ated, you can, under the Search tab:
- Browse for the desired Test Set Project or terms it Contains.
- Click on the More button to add more browse fields such as Label, Fix Version, Saved Filter and Workflow Status.
- Click on the Magnifier button after filling out the browse fields to get a list of matching entries.
- Click on the Advanced button to perform a JQL search.
| Info | ||
|---|---|---|
| ||
Notice the setting for ignoring Tests in non-executable statuses at the bottom-left side. By default it will be selected, therefore restricting the shown Tests available for adding accordingly to the configuration "Disallow executions of Tests with workflow statuses". |
Step 4: Click Add.
Associate with a Test Plan
You can associate Test Executions with Test Plans by editing the custom field Test Plan in the Test Execution issue view page, or directly in the create issue screen in the Test Execution Details tab.
If the Test Execution is associated with a Test Plan and contains Tests that are not in the Test Plan, a warning message will be displayed telling you how many tests in the Test Execution are not in the Test Plan with a button to add these tests to the Test Plan.
There will be one warning message for each Test Plan associated with the Test Execution that does not contain all the Tests in the Test Execution.
Remove Tests
Open the Search tab.
| Info | ||
|---|---|---|
| ||
This is an example of how to add tests, using some of the features available in the Search tab. For more information, please visit the Issue Picker Dialog detailed overview. |
Step 4: Configure the search.
-Use the Filters dropdown button to include extra fields in the search criteria.
-Fill in the filters values.
-Press the search button.
| Info | ||
|---|---|---|
| ||
Notice the setting for ignoring Tests in non-executable statuses at the bottom-left side. By default it will be selected, therefore restricting the shown Tests available for adding accordingly to the configuration "Miscellaneous > Workflow Status > ". |
Step 5: Select and Add tests.
- Once the results are obtained you can order, add/remove columns to the results list (the columns selection is saved as a preference).
- Before adding the selected tests you can define the execution assignee by using the upper right corner Assignee dropdown.
| Info | ||
|---|---|---|
| ||
Since Xray 3.4.0, the Assignee dropdown provides you a suggestions list with the user type fields found in the tests. If you choose the "Test Reporter" from the list, the assignee of the resultant Test Run will be the user set in the Test Reporter field of the respective test. |
- Check the tests you wish to add from the listed results.
Step 6: Add tests by pressing "Add selected".
Add Tests from Test Sets
In practical terms this operation is very similar to adding tests, instead you will just be searching and selecting Test Set issues.
Keep in mind that when adding a Test Set issue to a Test Execution, all the Test issues associated with the Test Set are added to the Test Execution.
The Test Set issue is not directly associated with a Test Execution, so changes to the Test Set after it was added will not affect the Test Execution.
Associate with a Test Plan
You can associate Test Executions with Test Plans by editing the custom field Test Plan in the Test Execution issue view page, or directly in the create issue screen in the Test Execution Details tab.
If the Test Execution is associated with a Test Plan and contains Tests that are not in the Test Plan, a warning message will be displayed telling you how many tests in the Test Execution are not in the Test Plan with a button to add these tests to the Test Plan.
There will be one warning message for each Test Plan associated with the Test Execution that does not contain all the Tests in the Test Execution.
Remove Tests
| Section | ||||
|---|---|---|---|---|
|
Assigning Work
| Section | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
There are two ways to distribute work between Testers.
When assigning individual Test Runs, it is also possible to assign based on any user type field present on the Test cases. For instance, you'll be able to assign Test Runs to the Test issue assignee. This allows users that are normally used to execute a specific Test case (that are assigned to the Test case), to be the default assignees on Test Runs within a Test Execution.
|
Change Test Run Rank
| Section | ||||
|---|---|---|---|---|
| ||||
| Section | ||||
|
Assigning Work
...
|
Execute Test
To execute a Test from a Test Execution:
Step 1: Open the Test Execution you wish to run
There are two ways to distribute work between Testers.
- A Test Execution issue can be created per Tester by assigning the issue to a specific Tester, or
- Within a Test Execution issue, you can assign individual Test Runs to Testers. Test Runs are assigned to the Test Execution issue assignee by default. Only users with permission to assign the Test Execution issue can assign Test Runs.
...
To assign individual Test Runs to testers,
...
.
Step 2: Hover over the desired Test Run (under the Tests section on the view page)
Step 3: Click the
...
Execute button that appears in the last column. The actions menu should pop
...
Step 4: Click the Assign action. The assign dialog appears.
Step 5: Start typing to find the user to assign the Test Run and click Assign to complete the action or Cancel to abort it.
| Column |
|---|
|
Change Test Run Rank
...
up with the available actions.
Step 4: Click Run action to open the Execution page to start the execution.
You can see a detailed explanation of the execution screen here.
Note: If you does not have permission to execute the Test Run, the "Run" option will not be available.
| Info | ||
|---|---|---|
| ||
This action may trigger some email notification(s) depending on your Xray settings. For more info on email notifications and how to set-up them, please have a look at Email Notifications. |
Execute Test Inline
To execute Test Runs inline from the Test Execution view screen, this option must be enabled in the Xray administration page. The option for setting the Test Run status manually without having to execute all Test Steps (or Examples, in the case of Cucumber Tests) must also be enabled in the Xray administration page.
Given that the above options are enabled and you have permission to execute the Test, the context menu for executing Test Runs, accessed by the Gear button on each Test, must show the available transitions.
When executing inline Tests, the status of the manual steps (or Examples, in the case of Cucumber Tests) can be changed automatically. The following rules are applied:
- PASS: All step statuses will be changed to PASS independently of their current status.
- FAIL: All step statuses will remain unchanged, even if they are all PASS. This is because a Test might fail due to a Pre-Condition.
- TODO: All step statuses will be changed to TODO.
- ABORTED: The Test Execution status will be changed to ABORTED.
- EXECUTING: The Test Execution status will be changed to EXECUTING.
- BLOCKED: The Test Execution status will be changed to BLOCKED.
- CUSTOM: The Test Execution status will be changed to the chosen custom status.
To execute a Test Run inline:
Step 1: Open the Test Execution issue you wish to run.
Step 2: Click the Execute button located in the last column of the Test Runs table and select one of the available statuses.
| Column |
|---|
There are two ways to change the rank of a Test Run: Drag-and-DropStep 1: Open the Test Execution issue. Step 2: Hover over the desired Test Run row (under the Tests section on the view page) and the drag handle column. Step 3: Click the drag handle and move the Test Run up or down in the current page. Change Rank ActionThis action allows users to change the Rank of a Test run between pages. Step 1: Open the Test Execution issue. Step 2: Hover over the desired Test Run (under the Tests section on the view page) Step 3: Click the gear that appears in the last column. The actions menu should pop up with the available actions. Step 4: Click the Move To action. The change rank dialog appears. Step 5: Enter the desired index number (1-based) and click OK. |
...
Execute Test
To execute a Test from a Test Execution:
Step 1: Open the Test Execution you wish to run.
Step 2: Hover over the desired Test Run (under the Tests section on the view page)
Step 3: Click the Execute button that appears in the last column. The actions menu should pop up with the available actions.
Step 4: Click Run action to open the Execution page to start the execution.
You can see a detailed explanation of the execution screen here.
Note: If you does not have permission to execute the Test Run, the "Run" option will not be available.
| Info | ||
|---|---|---|
| ||
This action may trigger some email notification(s) depending on your Xray settings. For more info on email notifications and how to set-up them, please have a look at Email Notifications. |
...
View Test
...
Run Details
You can always view and modify the Test Run details after execution. If you do not have permission to execute the Test Run, you can still view the execution details in read-only mode, given that you have permission to browse the Test Execution issue.
To view the execution details from a Test Run:
Step 1: Open the Test Execution you wish to view the execution details.
Step 2: Hover over the desired Test Run (under the Tests section on the view page).
Step 3: Click the Execute button that appears in the last column. The actions menu should popup with the available actions.
Step 4: Click the Execution Details action to open the Execution page and view the details.
Bulk Operations
Xray provides the following bulk operations on the Test Runs table.
- Execute Inline - Executes inline all the selected Test Runs.
- Assign - Assigns all the selected Test Runs to a particular user.
- Remove - Deletes all the selected Test Runs.
To execute these bulk actions:
Step 1: Open the Test Execution issue.
Step 2: Select the desired Test Runs. This can be done by using the checkbox that appears on each Test Run row or, since v3.1, by using the Bulk dropdown button options for easy selecting all Test Runs or the ones in the current page.
| Info | ||
|---|---|---|
| ||
As of v3.1, the selection will be preserved even if you navigate between pages. |
Step 3: Click the bulk actions gear located on the actions column header and choose one of the available actions.
| Info | ||
|---|---|---|
| ||
Bulk operations may trigger some email notification(s) depending on your Xray settings. For more info on email notifications and how to set-up them, please have a look at Email Notifications. |
Filter Test Runs
To filter the Test Runs table, select the desired filters in the filters section above the Test Runs table.
- Test Set - show only Test Runs that are associated with a specific Test Set issue.
- Assignee - show only Test Runs that are assigned to a specific user.
- Status - show only Test Runs with a specific status.
- Contains Text - show only Test Runs with Tests containing the entered text. This will query Test issues with the JQL keyword text ~ ...
Since v3.1, the filters will be stored in the user and Test Execution context, and will be applied whenever going into the execution screen from one of the filtered Tests.
The execution screen shows a small notice, so you can quickly identify that the filter has been applied thus affecting the navigation options (Previous/Next) available in the execution screen.
If you decide to come back to the Test Execution issue screen, the filter will remain. If you wish, you can clear the filter in order to see all Test Runs once again.
Configure Test Run Columns
More info in Test Runs.
Order Test Runs
Since v3.4.0, it is now possible to sort test runs by an Issue Field of the Test. Any sortable JIRA field and Custom Field is eligible. Simply add it to the table using the Columns menu and then click it's header cell on the table to sort by that field. Click again to toggle between ascending and descending order. Just as the configured columns a user has selected are stored, the selected order is also stored.
To execute Test Runs inline from the Test Execution view screen, this option must be enabled in the Xray administration page. The option for setting the Test Run status manually without having to execute all Test Steps (or Examples, in the case of Cucumber Tests) must also be enabled in the Xray administration page.
Given that the above options are enabled and you have permission to execute the Test, the context menu for executing Test Runs, accessed by the Gear button on each Test, must show the available transitions.
When executing inline Tests, the status of the manual steps (or Examples, in the case of Cucumber Tests) can be changed automatically. The following rules are applied:
- PASS: All step statuses will be changed to PASS independently of their current status.
- FAIL: All step statuses will remain unchanged, even if they are all PASS. This is because a Test might fail due to a Pre-Condition.
- TODO: All step statuses will be changed to TODO.
- ABORTED: The Test Execution status will be changed to ABORTED.
- EXECUTING: The Test Execution status will be changed to EXECUTING.
- BLOCKED: The Test Execution status will be changed to BLOCKED.
- CUSTOM: The Test Execution status will be changed to the chosen custom status.
To execute a Test Run inline:
Step 1: Open the Test Execution issue you wish to run.
Step 2: Click the Execute button located in the last column of the Test Runs table and select one of the available statuses.
| Info | ||
|---|---|---|
| ||
This action may trigger some email notification(s) depending on your Xray settings. For more info on email notifications and how to set-up them, please have a look at Email Notifications. |
View Test Run Details
You can always view and modify the Test Run details after execution. If you do not have permission to execute the Test Run, you can still view the execution details in read-only mode, given that you have permission to browse the Test Execution issue.
To view the execution details from a Test Run:
Step 1: Open the Test Execution you wish to view the execution details.
Step 2: Hover over the desired Test Run (under the Tests section on the view page).
Step 3: Click the Execute button that appears in the last column. The actions menu should popup with the available actions.
Step 4: Click the Execution Details action to open the Execution page and view the details.
Bulk Operations
Xray provides the following bulk operations on the Test Runs table.
- Execute Inline - Executes inline all the selected Test Runs.
- Assign - Assigns all the selected Test Runs to a particular user.
- Remove - Deletes all the selected Test Runs.
To execute these bulk actions:
Step 1: Open the Test Execution issue.
Step 2: Select the desired Test Runs. This can be done by using the checkbox that appears on each Test Run row or, since v3.1, by using the Bulk dropdown button options for easy selecting all Test Runs or the ones in the current page.
| Info | ||
|---|---|---|
| ||
As of v3.1, the selection will be preserved even if you navigate between pages. |
Step 3: Click the bulk actions gear located on the actions column header and choose one of the available actions.
| Info | ||
|---|---|---|
| ||
Bulk operations may trigger some email notification(s) depending on your Xray settings. For more info on email notifications and how to set-up them, please have a look at Email Notifications. |
Filter Test Runs
To filter the Test Runs table, select the desired filters in the filters section above the Test Runs table.
- Test Set - show only Test Runs that are associated with a specific Test Set issue.
- Assignee - show only Test Runs that are assigned to a specific user.
- Status - show only Test Runs with a specific status.
- Contains Text - show only Test Runs with Tests containing the entered text. This will query Test issues with the JQL keyword text ~ ...
Since v3.1, the filters will be stored in the user and Test Execution context, and will be applied whenever going into the execution screen from one of the filtered Tests.
The execution screen shows a small notice, so you can quickly identify that the filter has been applied thus affecting the navigation options (Previous/Next) available in the execution screen.
If you decide to come back to the Test Execution issue screen, the filter will remain. If you wish, you can clear the filter in order to see all Test Runs once again.
Configure Test Run Columns
More info in Test Runs.
Export to Cucumber
Cucumber Test Executions in JIRA can be exported to feature files ready to be executed in Cucumber. The generated feature files follow the rules defined in Export Rules and take into account the context of the Jira Test Execution issue.
...
Step 2: Select More > Export to Cucumber. The file will be downloaded from the browser.
Export Test Runs to CSV
Xray Test Runs can be exported to CSV. The Xray export action will export to CSV the most relevant fields for Test Runs such as:
...
Step 2: Select More > Export Test Runs to CSV. The resulting CSV file will be downloaded to the browser.
Import
...
Execution Test Results
Importing from Test Execution view issue screen
...
Step 2: Select More > Import Automated Execution Results. The Import Automated Test Results dialog will be prompted.
Step 3: Click the Browse button to navigate to the local disk and select the JSON/XML/ZIP file with the automated execution results for the Test Execution. You can see the support file schemas here.
Step 4: Click Import.
Importing from the Execution Details Page
...
clicking Execution Details if the test has been previously executed).
Step 2: Click the Import Execution Results button.
...
Step 3: Click the Browse button to navigate the local disk and select the JSON/XML/ZIP file with the automated execution results for the Test Execution.
Step 4: Click Import.
...