Page History
An explanation for how to best handle a common test design challenge related to dynamic cardinality.
The Problem
“Select All That Apply” scenarios occur frequently in web applications and are typically mishandled by test designers. Let’s look at the following example for some context:
...
As you can see, you have many topping choices available. You can choose as many or as few as you would like, and there are no guidelines surrounding the maximum number of toppings you can add (though it may get a bit pricey in reality). When facing a situation like this, the first test design strategy that occurs to many test designers is to create a single parameter called “Pizza Toppings” in DesignWise and Test Case Designer and list each of the 24 available toppings as Values, as shown below.
...
The second problem with this test design strategy is that the number of 2-way tests generated would be unnecessarily large. This is because each of the 24 toppings would need to appear together with every other value in the planmodel. This would result in at least 98 test scenarios. There would be 24 test scenarios involving small-sized pizzas (one with each pizza topping), 24 more tests with medium-sized single-topping pizzas, 24 more tests with large-sized single-topping pizzas, and 24 more tests with extra large single-topping pizzas.
...
- Multiple pizza toppings can appear on the same pizza.
- The parameters are weighted so that a relatively realistic number of toppings (an average of about 3) appear in each scenario.
- It only takes 26 tests to test every 2-way interaction in the system and achieve full pairwise coverage. Obviously we would want to include parameters like Size, Crust Type, and Sauce Type in the final version of this planmodel, but these additions would only minimally, if at all, increase the number of tests.
...
You could simply list out all the pizza toppings for each parameter and be done, but more than likely there is a greater chance of the 1st pizza topping being added to a pizza than a 4th topping. We want to account for this difference in our test planmodel. We can accomplish this by weighting the parameters to fit our exact needs, as shown below.
...
One way to avoid this problem is to use mixed-strength test coverage, utilizing 1-way coverage for each of the pizza toppings while keeping the 2-way coverage for any additional parameters. Mixed-strength test plans allow models allow you to select different levels of coverage for different parameters. This allows you to control the balance between increased test thoroughness with the workload created by additional tests. If we simply adjust the coverage level of each of the 4 pizza toppings to 1-way coverage while keeping the rest of the parameters at 2-way coverage, we are left with only 13 scenarios.
...
Particularly when your underlying business rules treat certain values (or in this situation, toppings) in the same way, you can assemble those values into groups. For instance, if the pizza application makes no internal distinction between pepperoni, ham, and bacon and only cares that each of those toppings are meats, you can easily account for this inside your DesignWise planTCD model. To do so, create a parameter value called “Meat” and include each type of meat inside a value expansion for that value.
An example of a test plan model following this pattern is shown below. It generates only 44 tests to achieve 100% 2-way coverage, significantly limiting the total number of tests without sacrificing test coverage.
...