Page History
...
- if any test step status is "FAIL" then the calculated status for the Test Run will be "FAIL"
- if any of the test steps "contributes negatively" (i.e., is mapped to a Test status associated with the NOK coverage), then the status of Test Run will correspond to the mapped Test status of that step
- the Test Run will have status "PASS" if all the steps are marked as "PASS"
- the calculated status for the Test Run will only be "EXECUTING" if there is at least one step in "EXECUTING" or "TODO" (or a similar custom test step status) and all other steps are in "PASS" or equivalent (i.e., associated to the "OK" coverage status)
Configuration Example 1
Examples
The following table provides some examples given the Test Step Statuses configuration shown above.
...
Let's consider the following configuration.
Example # | Statuses of the steps/contexts (the order of the steps/contexts is irrelevant) | Calculated value for the status of the Test Run | Why? |
|---|---|---|---|
| 1 |
| CUSTOM_PASS2 | We can see that both steps contribute in a "positive way" (i.e., they were successful as ultimately they are linked to successful coverage impact). Both statuses mapped to these test step statuses are associated with the "OK" coverage; as CUSTOM_PASS2 has higher ranking than CUSTOM_PASS, the run will be marked as "CUSTOM_PASS2". |
| 2 |
| CUSTOM_PASS2 | Similary to the previous example. Any status wins the "PASS" status. |
| 3 |
| CUSTOM_FAIL2 | We can see that both steps contribute in a "negative way" (i.e., they were not successful as ultimately they are linked to unsucessful coverage impact). Both statuses mapped to these test step statuses are associated with the "NOK" coverage; as CUSTOM_FAIL2 has higher ranking than CUSTOM_FAIL, the run will be marked as "CUSTOM_FAIL2". |
Calculation of the status for a given Test
...
- calculate the Test status for each Test Environment, based on all the implicit Test Environments from the relevant Test Executions (i.e. Test Executions in version V or Test Executions associated with Test Plan TP)
- calculate the joint value for the Test status
- PASS has lowest ranking (i.e. for the calculated to be PASS, all calculated statuses must be PASS in the different Test Environments)
- if one is FAIL, then the calculated value will be FAIL
- otherwise, use the ranking of Test statuses
Examples
The following table provides some examples given the Test Statuses configuration shown above in the Managing Test Statuses section.
...
| Info | ||
|---|---|---|
| ||
Even if you have sub-requirements, when you have tests that are directly linked to the parent requirement, Xray assumes that you are validating the requirement directly. Thus, it's irrelevant if the sub-requirements are uncovered by tests. |
Examples
The following table provides some examples given the Test Statuses configuration shown above in the Managing Test Statuses section.
...