Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Empyrical data about fault detection

Several studies Studies, such as presented by NIST, PRACTICAL COMBINATORIAL TESTING, 2010,  indicate that the vast majority of defects (67%-84%93%) related to input values are due to either to a problem in a parameter value (single-value fault) or in a combination of two parameter values (2-way interaction fault).

The following chart represents the cumulative error detection rate for fault-triggering conditions. 

Image Added


The key insight underlying t-way combinatorial testing is that not every parameter contributes to every fault and many faults are caused by interactions between a relatively small number of parameters


Single-value faults are mostly probable to typical mistakes, such as the off-by-one bug (e.g., imagine using a loop and using the "<" operator instead of "<="). The interaction of 2 parameters may be to bugs around implementing cascade conditional logic statements (e.g. using if or similarinvolving those parameters/variables.

...