Page History
...
| Info | ||
|---|---|---|
| ||
Whenever we're speaking about the status associated with a requirement or with a Test (and even with a Test Set), we may be talking about different things:
The current page details the first one, i.e. the status of the entities based on the executions made in some context. The "Requirement Status", "TestRunStatus" and "Test Set Status" custom fields are calculated fields, usable in the context of requirement, Test and Test Set issues, respectively, that calculate the "status" (not the workflow status) of the requirement/Test/Test Set for a specific version, considering the executions (i.e., test runs) for that version. The version for which it calculates the status depends on the behavior defined in a global configuration (see configuration details here). These fields will show the calculated status for the respective entity, for some specific version; they're just used to have a quick glimpse of the status of each entity, for example right in the view issue screen; therefore their usage is limited. More information on these and other custom fields can be found here . |
Test status
The status of a Test tells you information about its current consolidated state (e.g. latest record result, if existent). Was it is executed? Successfully? In which version?
...
| Info | ||
|---|---|---|
| ||
Do not mix up the status of a given test with the "TestRunStatus" custom field, which shows the status of a given Tests for a specific version, depending on the configuration under Custom Field PreferencesFields. More info on this custom field here. The "TestRunStatus" custom field is a calculated field that belongs to the Test issue and that takes into account several Test Runs; the "TestRunStatus" does not affect the calculation of the status of requirements. |
...
| Info | ||
|---|---|---|
| ||
| Do not mix up the status of a given requirement with the "Requirement Status" custom field, which shows the status of a given requirement for a specific version, depending on the configuration under Custom Field Preferencess. More info on this custom field here. |
...
- Obtain the list of Tests that directly or indirectly through Sub-Requirements (info here) cover the requirement
- This depends on the Requirement Coverage configuration
- Calculate the Test status for all the Tests individually, in version V or Test Plan TP
- This takes into account Test Runs in version V (as a result of Test Executions in version V) or Test Runs in Test Plan TP (within Test Executions associated with Test Plan TP)
- If a specific Environment is also chosen, then only Test Runs from Test Executions with this Environment will be considered. In case no Environment is specified then all Test Executions are considered (more info here).
- Calculate the "joint" status of all the previous Test statuses (i.e. by comparing together each Test status)
- Calculate the requirement status mapped to the previous calculated Test status
...
Setup information for some possible use cases
- I want to skip some Tests steps and proceed as they didn't exist
- create a "Test Step Status" (e.g. "SKIP"), mapped to the Test Status "PASS"
- I want to fail a Test Run but I don't want to mark the requirement as being NOT OK because this failure can be discarded
- create a "Test Status" (e.g. "FAIL_DISCARD") , non-final and mapped to the requirement status "UNKNOWN"; setting the status as non-final will give priority to other Test Runs you may have for that Test, If “Final Statuses have precedence over non-final” flag is enabled
- create a "Test Step Status" (e.g. "IRRELEVANT_FAIL") and map it to the Test Status created in the previous step
- I want to always see, for a given Test, the status of Test based on the last run scheduled for it, no matter if it was completed (i.e. in a final status) or not
- just uncheck the setting “Final Statuses have precedence over non-final”
- I want to execute some steps, set them as failed or passed, but I don't want them to reflect immediately in the status of the Test Run
- create custom, non-final, Test statuses for passing and failure (e.g. "MYPASS", "MYFAIL"), mapped to the OK and NOK requirement statuses, respectively
- create your own custom Test Step statuses for passed and failure (e.g. "PASS_CONTINUE" and FAIL_CONTINUE"), mapped to the previously created Test statuses
...