The status of a Test is generally calculated by looking at the Test's last execution. However, this does not work well if you execute the same Test in two different test environments (e.g., devices) and you want the two results to be consolidated. Within a Test Execution, you may specify the Test Environment(s) where the tests will be executed. Test Environments can be created by project managers and made accessible for users to specify in Test Execution issues. If you use Test Environments, you may execute the same Test for multiple test environments by creating Test Executions for each one. If you don't use Test Environments and you want to track tests for multiple environments (e.g., devices), then the only way to do it is to create multiple tests, one per each test environment. Let's say that you have executions for two Test Environments: "Android" and "iOS". The test is considered as PASS only if the latest executions for Android and iOS are both PASS; otherwise, it will be FAIL. Xray allows you to calculate the status of Tests and Requirements considering all Environments or a specific Environment.
Learn more here. |