Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Info
titleExample with Xray's Test Case Designer

In Test Case Designer we can . In the following example, we considered an interaction that we need to test due to an hypothetic legislation where some warning must be shown to users who are departing from USA, using the First class, and have more than 1 children. That scenario will be added on the generated ones.




Using pairwise and t-

...

way for scripted testing and exploratory testing

Whenever generating an optimized dataset (i.e., multiple "rows" of values for the parameters) this will be typically used to data-drive a scripted test case (e.g., a "manual" test composed of steps, or an automated test script).

...

Generating these combinations is useful not only for this testing approach though.

Pairwise and t-wise testing way testing don't tell us how to actually perform testing; it just generates the combination of parameters. Therefore, we can use this technique also if choose to adopt a more exploratory testing approach, for example for certain configurations of hardware/software.

Challenges

Pairwise or t-wise way testing even though useful, it's not a silver bullet.

Some challenges or limitations to be aware of, include:

  1. test oracle: this technique doesn't address finding the proper test oracle for the generated scenarios. How do we know the scenario is behaving as expected? How do we know that a given scenario has issues or not?
  2. modeling: depicting a "good" model requires the intervention of testers. Testers with the help of other team members are the ones able to figure out representative and important scenarios to model , their parameters, the values for those parameters, constraints, etc. 

...

Info

Test Case Designer is a feature that is only available to Xray Enterprise users.




Xray's parameterized tests & datasets

(available in all Xray versions)

Xray Test Case Designer

(part of Xray Enterprise only)

Parameters

  • define parameters
xx
  • parameters: enumerate possible values
xx
  • parameters: ranged values
-x
Dataset/scenarios generation

  • custom datasetdatasets
    (i.e., user-defined examples of parameter values)
x-
  • generation of all combinations of parameters/values
x

x

  • generation of a partial combination of parameters
x-
  • generation of scenarios using pairwise (2-way testing)
-x
  • generation of scenarios using t-way testing
-x
  • constraints/rules on generation of scenarios
-x
  • forced interactions
-x
Creation of tests using generated data

  • authoring test cases  (definition of steps) using the generated data
xx
  • generation of test automation code skeleton for multiple testing frameworks, using the generated data
-x
Reporting

  • track t-way coverage
-x

...