Page History
...
| Info | ||
|---|---|---|
| ||
Do not mix up the status of a given test with the "TestRunStatus" custom field, which shows the status of a given Tests for a specific version, depending on the configuration under Configuring Statuses Custom Fields. More info on this custom field here. The "TestRunStatus" custom field is a calculated field that belongs to the Test issue and that takes into account several Test Runs; the "TestRunStatus" does not affect the calculation of the status of requirements. |
...
Creating new Test (Run) statuses may be done in the Manage Test Statuses configuration section of Xray.
...
Creating new Test Step statuses may be done in the Manage Test Step Statuses configuration section of Xray.
...
| Info | ||
|---|---|---|
| ||
| Do not mix up the status of a given requirement with the "Requirement Status" custom field, which shows the status of a given requirement for a specific version, depending on the configuration under Custom Fields. More info on this custom field here. |
Calculation of the status for a given Test Run
...
- By Version: For a given Test X, in order to calculate the coverage status for version V, we need to evaluate the related Tests Runs that were executed on that same version V. A special case is whenever you don't have versions or simply don't want to calculate the status based on a version (i.e. "None (latest execution)")
- By Test Plan: For a given Test X, in order to calculate the coverage status for Test Plan TP, we need to evaluate the related Tests Runs that were executed on Test Executions associated with Test Plan TP.
- On a specific Test Environment: For a given Test X, if a specific Environment is also chosen, then only Test Runs from Test Executions with this Environment will be considered. In case no Environment is specified then all Test Executions are considered (more info here).
What affects the calculation:
...
- Obtain the list of Tests that directly or indirectly through Sub-Requirements (info here) cover the requirement
- This depends on the Requirement Coverage configuration
- Calculate the Test status for all the Tests individually, in version V or Test Plan TP
- This takes into account Test Runs in version V (as a result of Test Executions in version V) or Test Runs in Test Plan TP (within Test Executions associated with Test Plan TP)
- If a specific Environment is also chosen, then only Test Runs from Test Executions with this Environment will be considered. In case no Environment is specified then all Test Executions are considered (more info here).
- Calculate the "joint" status of all the previous Test statuses (i.e. by comparing together each Test status)
- Calculate the requirement status mapped to the previous calculated Test status
...
- the requirement coverage strategy defined in the respective setting, in Xray administration settings, in the tab "Requirement Coverage" (more info here)
- the strategy is used to find out the relevant Tests considered for the calculation
- the separation of concerns setting, in Xray administration settings, in the tab "Requirement Coverage" (more info here)
- this affects how the status of a Test Run contributes to the status of each linked requirement; if disabled, users can enforce a specific mapping for each requirement, otherwise the Test Run status will contribute in the same way to all the linked requirements
- indirectly, the setting "Final statuses have precedence over non-final statuses" in Xray administration settings, in the tab "Test Statuses" (enabled by default)
- the existence of Test Runs for different Test Environments, in case the analysis is made for "All Environments"
...