Page History

...

Requirements may be validated directly or indirectly, through the related sub-requirements and associated Test cases.

Image RemovedImage Added

Thus, how do all these factors contribute to the calculation of a coverage status? How is evaluated the status of some Test?

...

1. Obtain the list of Tests that directly or indirectly through "child" issues (e.g. sub-requirements) cover the issue
   • This depends on the Test Coverage Hierarchy related settings, defined in Project Settings: Test Coverage 
2. Calculate the Test status for all the Tests individually, in version V or Test Plan TP
   • This takes into account Test Runs in version V (as a result of Test Executions in version V) or Test Runs in Test Plan TP (within Test Executions associated with Test Plan TP)
   • If a specific Environment is also chosen, then only Test Runs from Test Executions with this Environment will be considered. In case no Environment is specified then all Test Executions are considered (more info on Test Environments here).
3. Calculate the "joint" status of all the previous Test statuses (i.e. by comparing together each Test status)
4. Calculate the requirement status coverage status mapped to the previous calculated Test status

...

What affects the calculation:

...

• this affects how the status of a Test Run contributes to the status of each linked requirement; if disabled, users can enforce a specific mapping for each requirement, otherwise the Test Run status will contribute in the same way to all the linked requirements
indirectly, the flag "Final statuses have precedence over non-final statuses" (enabled by default)
• the existence of Test Runs for different Test Environments, in case the analysis is made for "All Environments"

...

1. I want to skip some Tests and proceed as they didn't exist
   1. create a "Test Step Status"  (e.g. "SKIP"), mapped to the Test Status "PASS"
2. I want to fail a Test Run but I don't want to mark the requirement as being NOT OK because this failure can be discarded
   1. create a "Test Status" (e.g. "FAIL_DISCARD") , non-final and mapped to the coverage status "UNKNOWN"; setting the status as non-final will give priority to other Test Runs you may have for that Test, If “Final Statuses have precedence over non-final” flag is enabled
   2. create a "Test Step Status" (e.g. "IRRELEVANT_FAIL") and map it to the Test Status created in the previous step
3. I want to always see, for a given Test, the status of Test based on the last run scheduled for it, no matter if it was completed (i.e. in a final status) or not
   1. just uncheck the flag “Final Statuses have precedence over non-final”
4. I want to execute some steps, set them as failed or passed, but I don't want them to reflect immediately in the status of the Test Run
   1. create custom, non-final, Test statuses for passing and failure (e.g. "MYPASS", "MYFAIL"), mapped to the OK and NOK requirement coverage statuses, respectively
   2. create your own custom Test Step statuses for passed and failure (e.g. "PASS_CONTINUE" and FAIL_CONTINUE"), mapped to the previously created Test statuses 

...