Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • if a step status is mapped to a non-final, then Test Run status will be "EXECUTING"
  • compare all the step statuses based on their order (steps at the bottom of the list will have higher ranking)
  • the step status "PASS" has the lowest ranking 
  • the step status "FAIL" has the highest ranking

The order of the steps is irrelevant for the purpose of the overall Test Run status value.

...

Example #

Statuses of the steps/contexts

(the order of the steps/contexts is irrelevant)

Calculated value for the status of the Test Run

Why?

1
  • PASS
  • PASS
  • PASS
PASSAll steps are PASS, thus the joint value is PASS
2
  • PASS
  • TODO
  • PASS
EXECUTINGAt least one step status (i.e. TODO) is mapped to a non-final Test status
3
  • PASS
  • FAIL
  • PASS
FAILOne of the step statuses (i.e. FAIL) has higher ranking than the other ones
4
  • XPASS
  • FAIL
  • PASS
FAIL

XPASS has higher ranking than the other ones, thus the overall calculated value is based on the mapping for that Test Step statusSince one of the steps is FAIL, then the run will be marked as FAIL.


5
  • FAIL
  • XPASS
  • FAIL
FAILXPASS has higher ranking than the other ones, thus the overall calculated value is based on the mapping for that Test Step statusSince one of the steps is FAIL, then the run will be marked as FAIL.


Calculation of the status for a given Test

...