Page History
...
- Navigation and Actions bar
- Execution Status and Test Execution fields
- Comments
- Defects
- Evidence
- Test issue details
- Custom Fields
- Preconditions
- Test details (script or steps)
- Result contexts
- Activity
Navigation and Actions bar
...
If you are executing an automated Test, you can export its definition right from the execution screen.
Export to Cucumber
If you're executing a Cucumber Test, an option to export the Test as a feature file will appear.
...
If you already have the test results, just use this action to import the report files into Xray and update a specific Test Run.
Dataset
If you want to edit a test run dataset, you can do it by clicking on the Dataset button. To learn more about it click on Parameterized Tests.
Execution Status
The Execution Status describes the current Test progress. The possible (native) status for a Test Run are TODO, EXECUTING, FAIL, ABORTED, and PASS. You can also create custom statuses in the Xray Execute Tests administration page.
...
Info |
---|
This section is only visible if the "Separation of Concerns" option is disabled in Coverage Settings settings page. |
All affected requirement statuses will be updated whenever a step (or example in the case of Cucumber automated Tests) status is changed. The status for any affected requirement can always be changed by clicking on the status and choosing the desired option.
...
The field supports the Jira markup wiki language.
Execution Defects
The Execution Defects field lists the overall defects related to the current execution. It is possible to create a new Defect issue, create a new Sub-task or Add existing Defects by clicking on the respective option.
Execution Evidence
The Execution Evidences field lists the overall attachments related to the current execution and by clicking on the Add Evidences, opens the Attach Files dialog box.
Issue Details
This section includes some fields present on the Test issue. It often contains useful information for testers to execute the Test.
Test Description
This section includes the current Test issue description. The Test issue description often contains useful information for testers to execute the Test.
The field supports the Jira markup wiki language.
Test Issue Links
This section shows all links associated with the Test.
Test Issue Attachments
This section shows all attachments associated with the Test.
Pre-Conditions
The Pre-Condition field shows the Pre-Conditions issues associated with the current Test.
Pre-Condition Attachments
The Pre-Condition field also shows the Pre-Conditions attachment in a inline dialog.
Custom Fields
You define additional custom fields for Test Runs. These fields can be useful to add extra information to Test Runs, usually only available when or after executing Tests.
Test Run custom fields can be configured by project and by Test Type. Therefore, these settings will not affect other projects within your Jira instance. For instance, it is possible to have custom fields just for Manual Tests within a project.
If there are any custom fields defined for the scope of the Test Run, they will appear on the "Custom Fields" section.
It is also possible to define custom fields that are mandatory. If this is the case, users will not be able to set a final status on the Test Run if there is any required custom field that is not populated.
Test Details
Within the Test Details section, you will find the Test script to execute. Depending on the Test Type, this section can display Test Steps, a Generic Definition field (for unstructured Tests), and Gherkin scenarios.
Manual Test Step Results Table
A Manual Test execution screen contains the Test Steps field, listing the Steps, Data, Expected Result and Attachments to be used as reference while testing.
By clicking on each step, the following fields become editable:
- the Actual Result field which contains the actual result of the execution of the step;
- This field can be edited using the wiki markup toolbar from Jira
- It's possible to refer to an Evidence previously added to any Step or Global execution
- the Comment field which contains any notes about the corresponding step;
- the Defects field which contains any defect associated with the corresponding step. It is also possible to input new defects by:
- Clicking on the Create Defect to create a new Defect
- Clicking on the Create Sub-Task to create a new Sub-Task
- Clicking on the Add Defects to add an already existing defect - the Evidences field which contains any attachment associated with the corresponding step. It is also possible to input new attachments by clicking on the + button, which opens the Attach Files dialogue box
- the Status which contains the current step testing status. This can be changed by clicking on any of the status in front of your Step Status.
Note: Changing the Status on the each Step will affect the Execution Status field on the Execution Screen.
Gherkin Test Details
Gherkin Tests always display the Test scenario on the execution page. Usually, these tests are automated and their result can be imported into Xray. However, it is also possible to execute these Tests manually by setting the overall result of the Test Run, or by setting the result of each example (in case of scenario outlines).
Examples
A scenario outline of an automated Cucumber test will normally contain an "examples" table containing several inputs and outputs for the given scenario.
The examples section details these examples, containing an overall status bar in the header and a table with all the examples below. Each example can have the status: PASS, FAIL or TODO.
A scenario outline test is PASS only if all examples passed.
A scenario outline test is FAIL if at least one of the examples failed.
A scenario outline test is TODO if at least one of the examples did not run and all the other examples passed or todo.
Unstructured Test Details
Unstructured Tests only contain the generic definition field on the Test Details section. This field can specify a test script, an ID to an external (automated) test case, or an Exploratory Test charter. Similar to Gherkin Tests, most unstructured test results will be imported or set by another tool. However, it is also possible to set the status of unstructured tests manually.
Global and Test Step Defects
The options menu on the right top corner appears for Manual Tests only, it changes the presentation of the defects list.
By enabling the Show Step Defects option, besides the Global Defects, the Test Step Defects will be shown in this list along with the index of the Step it belongs to.
Execution Evidence
The Execution Evidences field lists the attachments related to the current execution and by clicking on the Add Evidence Icon, opens the Attach Files dialog box.
Global and Test Step Evidence
The options menu on the right top corner appears for Manual Tests only, it changes the presentation of the evidence list.
By enabling the Show Step Evidence option, besides the Global Evidence, the Test Step Evidence will be shown in this list along with the index of the Step it belongs to.
Issue Details
This section includes some fields present on the Test issue. It often contains useful information for testers to execute the Test.
Test Description
This section includes the current Test issue description. The Test issue description often contains useful information for testers to execute the Test.
The field supports the Jira markup wiki language.
Test Issue Links
This section shows all links associated with the Test.
Test Issue Attachments
This section shows all attachments associated with the Test.
Pre-Conditions
The Pre-Condition field shows the Pre-Conditions issues associated with the current Test.
Pre-Condition Attachments
The Pre-Condition field also shows the Pre-Conditions attachment in a inline dialog.
Custom Fields
You define additional custom fields for Test Runs. These fields can be useful to add extra information to Test Runs, usually only available when or after executing Tests.
Test Run custom fields can be configured by project and by Test Type. Therefore, these settings will not affect other projects within your Jira instance. For instance, it is possible to have custom fields just for Manual Tests within a project.
If there are any custom fields defined for the scope of the Test Run, they will appear on the "Custom Fields" section.
It is also possible to define custom fields that are mandatory. If this is the case, users will not be able to set a final status on the Test Run if there is any required custom field that is not populated.
Test Details
Within the Test Details section, you will find the Test script to execute. Depending on the Test Type, this section can display Test Steps, a Generic Definition field (for unstructured Tests), and Gherkin scenarios.
Manual Test Step Results Table
A Manual Test execution screen contains the Test Steps field, listing the Steps, Data, Expected Result and Attachments to be used as reference while testing.
By clicking on each step, the following fields become editable:
- the Actual Result field which contains the actual result of the execution of the step;
- This field can be edited using the wiki markup toolbar from Jira
- It's possible to refer to an Evidence previously added to any Step or Global execution
- the Comment field which contains any notes about the corresponding step;
- the Defects field which contains any defect associated with the corresponding step. It is also possible to input new defects by:
- Clicking on the Create Defect to create a new Defect
- Clicking on the Create Sub-Task to create a new Sub-Task
- Clicking on the Add Defects to add an already existing defect - the Evidences field which contains any attachment associated with the corresponding step. It is also possible to input new attachments by clicking on the + button, which opens the Attach Files dialogue box
- the Status which contains the current step testing status. This can be changed by clicking on any of the status in front of your Step Status.
Note: Changing the Status on the each Step will affect the Execution Status field on the Execution Screen.
Gherkin Test Details
Gherkin Tests always display the Test scenario on the execution page. Usually, these tests are automated and their result can be imported into Xray. However, it is also possible to execute these Tests manually by setting the overall result of the Test Run, or by setting the result of each example (in case of scenario outlines).
Examples
A scenario outline of an automated Cucumber test will normally contain an "examples" table containing several inputs and outputs for the given scenario.
The examples section details these examples, containing an overall status bar in the header and a table with all the examples below. Each example can have the status: PASS, FAIL or TODO.
A scenario outline test is PASS only if all examples passed.
A scenario outline test is FAIL if at least one of the examples failed.
A scenario outline test is TODO if at least one of the examples did not run and all the other examples passed or todo.
Unstructured Test Details
Unstructured Tests only contain the generic definition field on the Test Details section. This field can specify a test script, an ID to an external (automated) test case, or an Exploratory Test charter. Similar to Gherkin Tests, most unstructured test results will be imported or set by another tool. However, it is also possible to set the status of unstructured tests manually.
Iterations
When this is a test run of a data-driven test, this field shows all the iterations corresponding to the rows of the dataset closest to the test run. For each iteration, the preconditions and steps appear with the expected parameter values for it. To learn more about it click on Parameterized Tests.
All operations for test steps are also available within an iteration, namely adding defects, evidence, and the actual result.
This section also features a progress bar displaying the status of all iterations.
Filtering iterations
It is also possible to filter the iterations by status. A filter button is provided next to the iterations progress bar. Clicking on this button will open a panel with all the available statuses. Choose one or more statuses and press Apply.
Status calculation
When a step status is changed, the iteration status will be updated automatically, according to the step statuses and, in turn, the overall test status will also be updated based on all iterations statuses.
Automated Results
...