What is the Test Run Status
The state or result that was reported against a Test Run
- Represents the status of the Test in the context of some Test Execution
- Can be set explicitly/inline or automatically, based on the step level results (i.e. statuses)
The Test Run Status is not the TestRunStatus custom field (this is a totally different thing explained ahead).
Test Status Configuration
Xray provides some built-in Test Step statuses (which can’t be modified nor deleted).
But you can add based on your own preference.
- Custom statuses may be created in Xray administration settings
- A status may be marked as “final”
- Able to specify how the status contributes to the calculation of the requirement/coverage status
- Ranking is used to compare statuses under certain circumstances (explained ahead)
Where can you view it?
Usually referred as “Status”
- Execution screen itself (“execution status”)
- Test Runs panel (in Test issue screen, Test Execution issue screen)
- Traceability report
- Test Runs List report/gadget
|
|
Computing the overall status of a Test Run
- Use existing Test Run status if it has been enforced manually, otherwise proceed
- Obtain the test status mapped to each reported test step status
- Compare all the the previous mapped test statuses together
- Any status wins over PASS
- FAIL wins over any status
- Statuses marked as “final” win over intermediary ones
- Use ranking of mapped test statuses
How ranking end up being applied.
If all mapped statuses are only a mix of TODO and PASS, then consider Test Run to be EXECUTING