The “latest”/consolidated status of the coverable issue (e.g. requirement) for a specific context
Depends on where/how you’re analyzing your ”requirement"
Generally, it represents the latest result obtained for that Test in that context
More grained or high-level overview
Can involve consolidating results from multiple Test Environments
e.g. status of a Test for version 1.0, depends on consolidating the latest results for Chrome and Firefox
Info
the “Requirement Status” custom field provides a sneak peek for it (explained ahead)
UI Step
Where is it present
Overall Requirement Coverage (indirectly)
Requirements List gadget
Traceability report
UI Step
How and issue/requirement can be covered
UI Step
Understanding Sub-requirements
e
UI Step
Requirements Hierarchy
Xray fully supports two level of requirements
Traceability and Overall Coverage Reports/gadgets are limited to two levels
It is possible to have more levels using one of the following combinations (although this hierarchy won’t be fully visible throughout Xray)
Epic⬄Story relation + specific issue link
Epic⬄Story relation + sub-task issues
Info
Coverage - Which Tests are considered?
By default, the list of Tests that cover an issue/requirement is not dependent on the version, i.e. all “linked” Tests are considered
If you wish to control which Tests are ”valid” for each version, then you must choose the “Use versioned Test Sets for Requirement Coverage” in the Requirement Coverage Xray settings
For each version, you must create a Test Set and assign it to the version you want, and only those Tests will be considered for the coverage calculation
UI Step
Coverage Status – Simplified overview
OK All the tests associated with the issue (e.g. requirement) are PASSED
NOKAt least one test associated with the issue isFAILED
NOT RUN At least one test associated with the issue is TODO,ABORTED or EXECUTING and there are no tests with status FAILED
UNKNOWN At least one test associated with the issue is UNKNOWN and there are no Tests with status FAILED
UNCOVERED The issue has no tests associated to it
UI Step
Calculation of coverage status in given scope
Obtain the list of Tests that cover the issue, directly or indirectly through sub-requirements
Depends on the Coverage configuration in the Xray settings
Calculate the Test status for all the Tests individually, in the given scope
Considers Test Runs in version V (as a result of Test Executions in version V) or Test Runs in Test Plan TP (within Test Executions associated with Test Plan TP)
If a specific Environment is also chosen, then only Test Runs from Test Executions with this Environment will be considered. In case no Environment is specified then all Test Executions are considered
Calculate the coverage status mapped to the previously calculated Test statuses
Compare the previously calculated coverage statuses together
UI Step
Assessing the coverage status of an issue
UI Step
Sub-requirements in Xray
The coverage status of the parent requirement will consider the sub-requirements status
Info
you would obtain the same value for the calculation of the status of the parent requirement ifyou consider that it is being covered by all the explicitly linked Tests and also the ones linked tosub-requirements.
UI Step
Multiple examples of Coverage
Info
Requirement has tests and its "UNCOVERED"
Test
Test status
Calculated coverage status
-
-
No Tests cover the issue.
Info
Requirement has tests and its "Ok"
Test
Test status
Calculated coverage status
1
PASS
OK
All Tests are passed
(all contribute to the OK status of the requirement)
2
PASS
3
PASS
Info
Requirement has tests and its "NOT RUN"
Test
Test status
Calculated coverage status
1
PASS
NOT RUN
One of the Tests (3) is TODO which is mapped to NOT RUN,that has higher ranking than OK.
2
PASS
3
TODO
Info
Requirement has tests and its NOK
Test
Test status
Calculated coverage status
1
PASS
NOK
One of the Tests (3) is FAIL, whichmaps to NOK, that higher rankingthan OK (of the mapped PASS tests)
2
PASS
3
FAIL
Info
Sub-Requirement NOK
Test
Test status
Calculated coverage status
1
PASS
NOK
One of the Tests (3b) is FAIL, thus subReq2 will beconsidered as NOK. Since it is NOK, then the parentrequirement will be NOK.
2a
(subReq1)
PASS
3a
(subReq2)
PASS
3b
(subReq2)
FAIL
Info
In this case, it is assumed that the parent requirement is NOK since one of the sub-requirements is NOK.
Sub-Requirement NOT RUN
Test
Test status
Calculated coverage status
1
PASS
NOT RUN
One of the sub-requirements (subReq1) is NOT RUN,thus the calculated status, whenever doing theconjunction with the parent requirement status, willbe NOT RUN.
2a
(subReq1)
TODO
3a
(subReq2)
PASS
3b
(subReq2)
PASS
Sub-Requirements is UNCOVERED
One Test directly linked to the parent requirement. Sub–requirement without tests.
Test
Test status
Calculated coverage status
1
(parent requirement)
PASS
OK
Since all sub-requirements are uncovered and the parentrequirement is covered directly by one Test (1), which is currentlyPASS, then the calculated "OK" status will be based on that Test.
Note: subReq1 is considered as being UNCOVERED
-
(subReq1)
-
Info
In this case, it is assumed that the parent requirement is completely validated by the Testdirectly linked to it.