Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • OK – requirement has been successfully and fully validated; all the Tests associated with the Requirement are PASSED
  • NOK – requirement is unsuccessfully validated; at least one Test associated with the Requirement is FAILED
  • NOTRUN – requirement has not been validated completely; at least one Test associated with the Requirement is TODO or ABORTED and there are no Tests with status FAILED
  • UNKNOWN – requirement is in an unknown state; at least one Test associated with the Requirement is UNKNOWN and there are no Tests with status FAILED
  • UNCOVERED – requirement is not covered with tests; the Requirement has no Tests associated to it

...

The status of a given Test Run is an attribute that most of the times time is calculated automatically on the respective recorded step statuses. User Users can also enforce a specific status for a Test Run, which in turn may implicitly enforce specific step statuses (e.g. setting a Test Run as "FAIL" can set all steps as "FAIL"). 

...

  1. if a step status is mapped to a non-final then Test Run status will be "EXECUTING"
  2. compare all the step statuses, by their order (steps at the bottom of the list will have higher ranking)
    1. the step status "PASS" has the lowest ranking 

The order of the steps is indifferent for the purpose of the overall Test Run status value.

...

Example #

Statuses of the Test Runs

(ordered by time of execution/creation, ascending)

Final statuses have precedence over non-final statusesCalculated value for the status of the TestWhy?
1a
  1. PASS
  2. PASS
  3. TODO
truePASSLatest executed Test Run (2) having a final status was PASS.
1b
  1. PASS
  2. PASS
  3. TODO
falseTODOLatest created Test Run (3) was TODO.
2a
  1. PASS (env1)
  2. MYPASS2 (env2)
  3. TODO (env2)
  4. PASS (env3)
trueXPASS

Latest executed final Test Runs on each environment were PASS, MYPASS2, and PASS respectively.

Since MYPASS2 (2) has a higher ranking then the calculated status will be MYPASS2.


2b
  1. PASS (env1)
  2. MYPASS2 (env2)
  3. TODO (env2)
  4. PASS (env3)
falseTODO

Latest created Test Runs on each environment were PASS, TODO, and PASS respectively.

Since PASS has the lowest ranking, then TODO (3) will "win" and then the calculated status will be TODO

3
  1. PASS (env1)
  2. TODO (env2)
  3. PASS (env3)
trueTODO

Latest created Test Runs on each environment were PASS, TODO, and PASS respectively.

Although Test Environment "env2" has only a non-final Test Run, since there is no other Run for that environment, then it will be considered as the calculated status for that environment.

Since PASS has the lowest ranking, then TODO (3) will "win" and then the calculated status will be TODO.

4
  1. PASS (env1)
  2. FAIL (env2)
  3. PASS (env3)
true (or false)FAIL

Latest executed (or created) final Test Runs on each environment were PASS, FAIL, and PASS respectively.

Since the calculated status for one of the environments is FAIL, then the calculated status will be FAIL.

5
  1. PASS (env1)
  2. MYPASS2 (env2)
  3. TODO (env2)
  4. MYFAIL (env3)
trueMYPASS2

Latest executed final Test Runs on each environment were PASS, MYPASS2, and MYFAIL respectively.

MYPASS2 has a higher ranking than the other ones, thus the overall calculated value will be MYPASS2.


6
  1. PASS (env1)
  2. MYPASS2 (env2)
  3. TODO (env2)
  4. MYFAIL (env3)
falseMYFAIL

Latest created Test Runs on each environment were PASS, TODO, and MYFAIL respectively.

MYFAIL has a higher ranking than the other ones, thus the overall calculated value will be MYFAIL.


...