You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 2 Next »

Subtle differences of statuses

  • Reported statuses for Test Run Steps and, implicitly, Test Runs are static; they are not dependent on any other variables or analysis criteria
  • The status of a Test or the coverage status of a “requirement” depend on how you analyze them (thus, implicitly on the Test Runs considered for that scope)

Status of a Test

How a given Test is “currently” performing in some scope/context

Was it executed? Successfully? In which version?

  • Whenever speaking about the "status of a Test" we need to give it some additional context (e.g. "In which version?") since it depends on "where" and how you want to analyze it.
  • Depends on the latest consolidated results obtained for that Test, within some scope
    • Provides real-time info on the “status” of coverable issues (e.g. requirements, user stories, epics), for some context; thus, it has no relation whatsoever with the workflow status 
  • The same Test can be PASS for some context, FAIL for another one and even NOTRUN for another, for example


Status of a Test – no environments

The Test foo

StatusCondition
... is PASS on v3.0

If the latest Test Run for that Test, in version 3.0, was PASS

... is PASS on v4.0

If the latest Test Run for that Test, in version v4.0, was PASS

... is FAIL on v4.0

If the latest Test Run for that Test, in version v4.0, was FAIL

Test Run executed in version X means that the Test Run is part of a Test Execution assigned to version X


Status of a Test – runs in many environments 

The Test foo


Condition
... is FAIL on v4.0 in environment X
  • If the latest Test Run for that Test, in version v4.0 in environment X, was FAIL
... is PASS on v3.0
  • If, for all  environments with Test Runs (for that Test) executed in v3.0, the latest Test Run was PASS
... is FAIL on v4.0
  • If, in any environment with Test Runs (for that Test) executed in v3.0, the latest Test Run was FAIL








Notes

    • Test Run executed in version X means that the Test Run is part of a Test Execution assigned to version X
    • Test Run executed in version X and environment X,  means that the Test Run is part of a Test Execution assigned to version X and Test Environment “X” 


  • No labels