The “latest”/consolidated status of the Test for a specific scope/context
Depends on where/how you’re analyzing your Test
Generally it represents the latest result obtained for that Test in that context
More grained or high-level overview
e.g. status of a Test for version 1.0, depends on consolidating the latest results for Chrome and Firefox
Can involve consolidating results from multiple Test Environments
Info
Note: the TestRunStatus custom field provides a sneak peek for it (explained ahead).
UI Step
Test Status in different scopes and Xray elements
Test Coverage panel in the “requirement” issue screen
“Tests” legacy report (on project tab)
Traceability Report
UI Step
Calculating the status of a Test for a Scope
1 - Obtain Test Runs based on the analysis scope (i.e. the context)
2 - Obtain all the distinct environments involved in the previous Test Runs
3 - For each Test Environment, obtain the latest Test Run
if final statuses have precedence then
…. pick most recent Test Run based on its “finished on” date, having a status marked as “final”; if no T.R. exists, then the latest intermediate one (see following rule)
else,
pick most recent Test Run based on its “created” date
4 - If results from multiple environments are involved, then a joint status must be calculated:
FAIL status wins over all other status
Any status wins over PASS
Based on their ranking, in the Test Statuses settings
Info
Check your "Final statuses have precedence over non-final statuses” setting (default=true).
UI Step
Analysis Scope – What is it?
The analysis scope is our context. Defines the criteria for finding out the relevant Test Runs.
Version “V”
Only Test Runs from Test Executions assigned to version V are considered
Test Plan “T”
Only Test Runs from Test Executions assigned to Test Plan T are considered
Latest results / No version
“Latest” Test Runs from ”recent” Test Executions are considered
And optionally…
Test Environment“E”
Only Test Runs from Test Executions assigned to Test Environment E are considered
Info
If no Test Environment is specified, then Xray calculates the status based on the joint values for all the different Test Environments having scheduled executions
Lets see some examples....
UI Step
Example 1 - "Final statuses have precedence over non-final statuses” setting as true
UI Step
Example 2 - "Final statuses have precedence over non-final statuses” setting as false
UI Step
Example 3 - Multiple Environments
When Final statuses have precedence over non-final statuses” setting astrue.
When Final statuses have precedence over non-final statuses” setting asfalse.