Versions Compared

Old Version 1

changes.mady.by.user Sergio Freire

Saved on

compared with

New Version 2

changes.mady.by.user Sergio Freire

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Example #

Statuses of the related Tests

(sub-requirements, whenever are present using the notation subReqX)

Calculated value for the status of the requirementWhy?
1
  1. PASS
  2. PASS
  3. PASS
OKAll Tests are passed (it is similar to having just one virtual test that would be considered PASS and thus mapped to the OK status of the requirement)
2
  1. PASS
  2. PASS
  3. TODO
NOT RUNOne of the Tests (3) is TODO, which has higher ranking than PASS.
3
  1. PASS
  2. PASS
  3. FAIL

NOK

One of the Tests (3) is FAIL, which has higher ranking than PASS.
4
  1. PASS
  2. subReq1 => OK
    1. PASS
  3. subReq2 => NOK
    1. PASS
    2. FAIL

NOK

One of the Tests (3b) is FAIL, thus subReq2 will be considered as NOK. Since it is NOK, then the parent req which has higher ranking than PASSrequirement status will be NOK.
5
  1. PASS
  2. subReq1 => NOTRUN
    1. TODO
  3. subReq2 => OK
    1. PASS
    2. PASS
NOT RUNOne of the subRequirements (subReq1) is NOT RUN, thus the calculated status, whenever doing the conjunction with the parent requirement status, will be NOT RUN.
6
  1. PASS
  2. subReq1 => UNCOVERED
    1. (no Tests associated)
  3. subReq2 => UNCOVERED
    1. (no Tests associated)
OKSince all sub-requirements are uncovered and the parent requirement is covered directly by one Test (1), which is currently PASS, then the calculated "OK" status will be based on that Test.

Setup information for some possible use cases

  1. I want to skip some Tests steps and proceed as they didn't exist
    1. create a "Test Step Status"  (e.g. "SKIP"), mapped to the Test Status "PASS"
  2. I want to fail a Test Run but I don't want to mark the requirement as being NOT OK because this failure can be discarded
    1. create a "Test Status" (e.g. "FAIL_DISCARD") , non-final and mapped to the requirement status "UNKNOWN"; setting the status as non-final will give priority to other Test Runs you may have for that Test, If “Final Statuses have precedence over non-final” flag is enabled
    2. create a "Test Step Status" (e.g. "IRRELEVANT_FAIL") and map it to the Test Status created in the previous step
  3. I want to always see, for a given Test, the status of Test based on the last run scheduled for it, no matter if it was completed (i.e. in a final status) or not
    1. just uncheck the setting “Final Statuses have precedence over non-final”
  4. I want to execute some steps, set them as failed or passed, but I don't want them to reflect immediately in the status of the Test Run
    1. create custom, non-final, Test statuses for passing and failure (e.g. "MYPASS", "MYFAIL"), mapped to the OK and NOK requirement statuses, respectively
    2. create your own custom Test Step statuses for passed and failure (e.g. "PASS_CONTINUE" and FAIL_CONTINUE"), mapped to the previously created Test statuses 

...