Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Test Case Designer allows you to adjust testing coverage to focus more thorough coverage on selected, high-priority areas

One straightforward risk-based testing technique (at the value level) is to simply enter a specific value multiple times in the “Parameters” screen. Entering “Male, Male, Female, Male” as Values, for example, would result in “Male” appearing three times as often as “Female.”

A more powerful risk-based testing strategy (at the parameter level) is generating Mixed-Strength test sets, as described in the following example. This feature can be helpful to use in many (if not most) of the models you will create.


Why does such risk-based testing makes sense conceptually?

There are only two possible reasons that a set of 2-way tests could fail to trigger a software defect:

  1. There was a missing test "idea" (e.g., the only way the defect could be found is if the application were tested using a specific operating system and that specific system was not included as one of the parameters/values).

  2. All of the test ideas and test conditions were included as values, but the defect could only be triggered by the scenario that included three or more of those existing test conditions together at the same time.

In our experience working with hundreds of software teams, the first reason (not thinking to include a particular test idea) is responsible for more defects slipping by testing than the second one (specific combinations of 3 or more already-included ideas).

Accordingly, this is what is recommended for testers who are planning to manually execute sets of 3-way scenarios:

  • Don't*

At least not until you first experiment a bit with executing well-thought-through sets of Mixed-Strength scenarios

...

*This advice is for manual testing projects (where the costs of executing extra tests is relatively high), not for automated test execution projects (where the costs of executing extra tests is relatively low).

Examples of good sources for additional testing ideas include these:


Let’s set up the use case we’re trying to address

We have a mission-critical application that includes several large changes in this release. We’ve filled in parameters for our System Under Test, and we see that a 2-way solution based on our variation ideas would require 87 scenarios.

...