Page History
This gadget provides a way to see the evolution of a group of Tests in some version of the system by taking into account some given Test Executions.
| Table of Contents |
|---|
Aim
See the evolution/trend of the current status of a group of Tests, taking into account the results from those Tests in some given Test Executions.
...
| Info | ||
|---|---|---|
| ||
Please note that only the selected/filtered Test Runs from the given Test Executions are taken into account for the calculation of the status of the Tests for a given moment in time. This means that only those Test Runs are considered. Therefore, as an example, if some Test was PASS (because you already had executed it somewhere in the past and the latest status of it was PASS) but you're only taking into account some more recent runs, then it will start by showing as TODO at the start of the chart. |
How does it work
The Tests Evolution gadget alows you to see the evolution/trend of Tests over time, which may occur due to new executions made for those Tests or to changes made to the reported statuses in the existing Test Runs.
Threrefore, this gadget does its calculations based on the historical changes for the Test Runs being considered. Note that for a given Test Run you may or not change the Test Run statuses several times; this gadget will consider it.
It also takes into account the setting "Final statuses have precedence over non-final statuses", available under "Manage Test Statuses" in Xray settings.
General rules for the calculation of the status at a given day
- changes made over time to the status of a Test Run will be reflected in the status calculated for that Test (no matter if it's a final or non-final status)
- a status will "persist" (be valid) from the moment it has been recorded until the next status change in that Test Run or in another Test Run
- the "Started on" or the "Finished on" dates of the Test Runs are not taken into account for the calculation
"Final statuses have precedence over non-final statuses" checked
| scenario | yesterday | today 10am | today 11am | |
|---|---|---|---|---|
| 1 | TR1 | TR1 | - | Test Run |
| PASS | TO DO | - | status change made in that day | |
| PASS | TO DO | calculated Test status for that day | ||
2 | TR1 | TR1 | TR1 | Test Run |
| PASS | FAIL | TO DO | status change made in that day | |
| PASS | TO DO | calculated Test status for that day | ||
| 3 | TR1 | TR2 | - | Test Run |
| PASS | TO DO | - | status change made in that day | |
| PASS | PASS | calculated Test status for that day | ||
| 4 | TR1 | TR2 | Test Run | |
| PASS | FAIL | status change made in that day | ||
| PASS | FAIL | calculated Test status for that day | ||
| 5 | - | TR1 | Test Run | |
| - | PASS | status change made in that day | ||
| - | PASS | calculated Test status for that day | ||
"Final statuses have precedence over non-final statuses" unchecked
In this case, all statuses are considered independently if they are final or not. The key differentiator is that latest test runs are always considered, just by taking into account their creation date.
| scenario | yesterday | today 10am | today 11am | |
|---|---|---|---|---|
| 1 | TR1 | TR1 | TR1 | Test Run |
| PASS | FAIL | TO DO | status change made in that day | |
| PASS | TO DO | calculated Test status for that day | ||
| 2 | TR1 | TR2 | TR2 | Test Run |
| PASS | FAIL | TO DO | status change made in that day | |
| PASS | TO DO | calculated Test status for that day | ||
| 3 | - | TR1 | TR2 | Test Run |
| - | FAIL | PASS | status change made in that day | |
| - | PASS | calculated Test status for that day | ||
| 4 | TR1 | TR2 | - | Test Run |
| TO DO | PASS | - | status change made in that day | |
| TO DO | PASS | calculated Test status for that day | ||