Subtle differences of statuses
- Reported statuses for Test Run Steps and, implicitly, Test Runs are static; they are not dependent on any other variables or analysis criteria
- The status of a Test or the coverage status of a “requirement” depend on how you analyze them (thus, implicitly on the Test Runs considered for that scope)
Status of a Test
How a given Test is “currently” performing in some scope/context
Was it executed? Successfully? In which version?
- Whenever speaking about the "status of a Test" we need to give it some additional context (e.g. "In which version?") since it depends on "where" and how you want to analyze it.
- Depends on the latest consolidated results obtained for that Test, within some scope
- Provides real-time info on the “status” of coverable issues (e.g. requirements, user stories, epics), for some context; thus, it has no relation whatsoever with the workflow status
- The same Test can be PASS for some context, FAIL for another one and even NOTRUN for another, for example
Status of a Test – no environments
The Test foo…
Status | Condition |
... is PASS on v3.0 | If the latest Test Run for that Test, in version 3.0, was PASS |
... is PASS on v4.0 | If the latest Test Run for that Test, in version v4.0, was PASS |
... is FAIL on v4.0 | If the latest Test Run for that Test, in version v4.0, was FAIL |
Test Run executed in version X means that the Test Run is part of a Test Execution assigned to version X
Status of a Test – runs in many environments
The Test foo…
Condition | |
---|---|
... is FAIL on v4.0 in environment X |
|
... is PASS on v3.0 |
|
... is FAIL on v4.0 |
|
- Test Run executed in version X means that the Test Run is part of a Test Execution assigned to version X
- Test Run executed in version X and environment X, means that the Test Run is part of a Test Execution assigned to version X and Test Environment “X”