Page History
...
Test Environments in Xray at a glance
The status of a Test (i.e., TestRunStatus) is Test is generally calculated by looking at the Test's last execution (more detail here). However, this does not work well if you execute the same Test in two different test environments (e.g., devices) and you want the two results to be consolidated.
...
Note: If you don't use Test Environments, then the TestRunStatus is calculated as usualonly the latest result matters for assessing the current status of the Test.
| Info |
|---|
If you start using Test Environments in your Test Executions, then it's not only your test status calculation that will change. All custom fields that depend on it (e.g., Requirement status, Test Sets status) will change. Consequently, the requirement coverage calculation and all associated charts/gadgets are also affected. |
...
Below are example scenarios and shows how the status of the TestRunStatus Test is calculated in each.
If you have a mix of Test Executions (i.e., with and without Test Environment as in scenario D), it wil will be treated the same way as scenarios A, B, and C.
| Scenario | Test Environment(s) of TE 1 | Test Environment(s) of TE 2 | Test run status in TE1 | Test run status in TE2 | Calculated value of "TestRunStatus" custom fieldfor the overall status of the Test |
|---|---|---|---|---|---|
A | Android | iOS | PASS | PASS | PASS |
B | iOS | iOS | PASS | FAIL | FAIL |
C | iOS | iOS | FAIL | PASS | PASS |
D | iOS | - | FAIL | PASS | FAIL |
E | - | - | PASS | FAIL | FAIL |
F | - | - | FAIL | PASS | PASS |
...
| Info | ||
|---|---|---|
| ||
For advanced Test Environment management capabilities, please check our Integration with Apwide Golive. |
Test Environments and the TestRunStatus custom field
The "TestRunStatus" custom field is associated with Test issues and can be used to provide information about the latest status of your test; more info here.
This custom field calculates the status of the test for "all environments" (i.e. the consolidated status), giving you a high-level view; it cannot be configured to show the status for a specific environment.
Internally, this field will store the status of the test for all possible scopes, which besides other things includes the information about the status in all different environments.