Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Info
titleExample with Xray's Test Case Designer

In Test Case Designer we can . In the following example, we considered an interaction that we need to test due to an hypothetic legislation where some warning must be shown to users who are departing from USA, using the First class, and have more than 1 children. That scenario will be added on the generated ones.



Challenges

Pairwise or t-wise testing even though useful, it's not a silver bullet.

Some challenges or limitations to be aware of include:

  1. test oracle: this technique doesn't address finding the proper test oracle for the generated scenarios. How do we know the scenario is behaving as expected? How do we know that a given scenario has issues or not?
  2. modeling: depicting a "good" model requires the intervention of testers. Testers with the help of other team members are the ones able to figure out representative and important scenarios to model , their parameters, the values for those parameters, constraints, etc. 


Using pairwise and t-wise for scripted testing and exploratory testing

...

Pairwise and t-wise testing don't tell us how to actually perform testing; it just generates the combination of parameters. Therefore, we can use this technique also if choose to adopt a more exploratory testing approach, for example for certain configurations of hardware/software.

Challenges

Pairwise or t-wise testing even though useful, it's not a silver bullet.

Some challenges or limitations to be aware of include:

  1. test oracle: this technique doesn't address finding the proper test oracle for the generated scenarios. How do we know the scenario is behaving as expected? How do we know that a given scenario has issues or not?
  2. modeling: depicting a "good" model requires the intervention of testers. Testers with the help of other team members are the ones able to figure out representative and important scenarios to model , their parameters, the values for those parameters, constraints, etc. 

Xray datasets and Xray Test Case Designer

Xray has built-in support for parameterized tests and datasets, supporting user-defined datasets and automatic generation of combinations for the identified parameters.

Test Case Designer (TCD) provides a more comprehensive modeling tool, where it's possible to define parameters, apply constraints, enforce specific scenarios, generate optimized datasets/scenarios using 2-way or n-way to ensure that certain parameters are tested together, up to a certain level. With Test Case Designer it's possible to have a limited and manageabled set of test scenarios to perform and make sure that most combination of parameters are met with the initial scenarios, so that most risk is addressed upfront.

TCD doesn't replace Xray built-in capabilities for parameterized tests and datasets; it's a more evolved approach. Both can be used in a given project.


Info

Test Case Designer is a feature that is only available to Xray Enterprise users.

...


Xray

(available in all Xray versions)

Xray Test Case Designer

(part of Xray Enterprise only)

Parameters

  • define parameters
xx
  • parameters: enumerate possible values
xx
  • parameters:
range of
  • ranged values
-x
Dataset/scenarios generation

  • custom dataset
    (i.e.,
enumeration
  • user-defined examples of parameter values
for all parameters
  • )
x-
  • generation of all combinations of parameters/values
x

x

  • generation of a partial combination of parameters
x-
smart
  • generation of scenarios using pairwise (2-way testing)
-x
smart
  • generation of scenarios using
n
  • t-way testing
-x
  • constraints/rules on generation of scenarios
-x
  • forced interactions
-x
Creation of tests using generated data

  • authoring test cases  (definition of steps) using the generated data
xx
  • generation of test automation code skeleton for multiple testing frameworks, using the generated data
-x
Reporting

tracking n
  • track t-way coverage
-x