Introduction
In certain Scenarios, Test Case Designer allows you to define which Parameter values should never be tested together. This is achieved by applying Constraints.
To guide the model, you can define Invalid Constraints (values that should never be tested together) and/or Bound Constraints (values that must be tested together). Important - regardless of the constraint type, the only function of this feature is the exclusion of certain combinations from the Scenarios screen.
Invalid Constraints
Invalid Constraints prevent parameter values that should never be combined in a Test case.
For example (Figure 1), assume that the browser Internet Explorer (IE) is not supported on Apple computers. In this case, it would be impossible to Test a scenario where IE is launched on a Mac running its native OS. Therefore, you want to prevent the generation of Test cases that combine Mac OS and IE10 or IE11.
Figure 1 - OS and Browser
Creating Invalid Constraints
Go to My Test Models (Figure 2 - 1) and open a Test Model (Figure 2 - 2).
Figure 2 - Test model
You will enter the Parameters screen (Figure 3).
Figure 3 - Parameters
Click Rules (Figure 4 - 1) and then Constraints (Figure 4 - 2) to access the Constraints screen (Figure 4).
Figure 4 - Constraints
You will click two red X’s. Hover over the first value of the Invalid Pair and click the red X that appears (Figure 4 - 3).
Find the second value that can never appear together with the first one, hover over it, and click the red X (Figure 5 - 1) to create your Invalid Pair.
Figure 5 - Invalid
As you enter the Constraints screen (Figure 6), you will see the pairs listed (Figure 6 - 1).
Figure 6 - Pairs
After entering these two Invalid Pairs, clicking the Scenarios button will create a new set of Test cases that exclude those two pairs of values (and only those two pairs of values). Each Test with Mac OS as the operating system will have a browser other than IE.
Figure 7 - Scenarios
As you use the Invalid Pair feature and the related Bound Pair, keep in mind these usage tips:
Adding Many Invalid Constraints
Imagine your app asks, Would you like to add a hotel to your travel reservation? Any Scenarios with Do Not Add a Hotel would require numerous Invalid Constraints between options like Do Not Add a Hotel and Hotel Chain Preference = X. In such cases, the Bound Constraint feature can help you handle constraints more efficiently.
If your model has more than 10 Invalid or Bound Constraints, it may be quicker to export your model into Excel to document paired values. To do this, follow these steps:
- Add multiple paired values in Xray Test Case Designer before exporting, so you can easily copy the formatting for Invalid or Bound Constraints.
- Ensure accurate spelling of values. Copying and pasting is safer than typing.
Not Applicable Values
Be cautious with multiple Not Applicable values in a model. For instance, does it make sense to create a Bound Constraint between Hotel Chain Preference = Not Applicable and Type of Room = Not Applicable ?
Bound Constraints
Bound constraints restrict Parameter values that can only be tested together.
Consider these parameters and values (Figure 8):
Figure 8 - Parameters
You will see the following screen (Figure 9) when you click the Scenarios button (Figure 8 - 1):
Figure 9 - Invalid Row
Even though the Test calls for not adding a hotel (Figure 9 - 1), the data row proceeds to specify the preference and type of room, which is clearly incorrect. To solve this, first, you need to add Not Applicable as parameter values (Figure 10 - 1).
Figure 10 - Conditional Value
This is because something will have to appear in the Test cases that include the value Do Not Add a Hotel (Figure 10 - 2).
We want Not Applicable (Figure 10 - 1) to appear for both Hotel Chain Preference (Figure 10 - 3) and Type of Room (Figure 10 - 3) in every Scenario that includes the value Do Not Add a Hotel (Figure 10 - 2).
The exact syntax of the conditional value and its position in the list don't matter. It could have been N/A as the last value.
Now, we want to make sure that Do Not Add a Hotel (Figure 10 - 2) only gets paired with the Not Applicable values (Figure 10 - 1). You have two options under the Constraints tab.
One option is quick, the other is slow. Let’s see the slow option, first:
Figure 11 - Invalid constraints
Namely, it’s adding a lot of Invalid Constraints as described above.
Creating Bound Constraints
Click the green arrow icon (it appears on hover; Figure 12 - 1).
Figure 12 - Arrow
Click the green arrow icon on the second value (it appears on hover; Figure 13 - 1).
Figure 13 - Arrow
Confirm how you’d like to have the Bound Constraint operate (Figure 14).
Figure 14 - Bound
Most of the time, when you have Not Applicable (Figure 14 - 2) as an option, you will use Mutually Bound Constraints as in this case. Here, we want Do Not Add a Hotel (Figure 14 - 1) to be bound with Type of Room as Not Applicable and we want to have the Type of Room of Not Applicable to be bound with Do Not Add a Hotel so we mark this one as a mutually bound constraint.
When you add a Bound Pair, the Xray Test Case Designer will constrain the first value chosen against all the other values in the parameter of the second value chosen. In the example above, creating a bound constraint of Do Not Add Hotel and Hotel Chain Preference = Not Applicable means you are really invalidating these options:
- Do Not Add Hotel with Hotel Chain Preference = Marriott (this combination will never appear).
- Do Not Add Hotel with Hotel Chain Preference = Hilton (this combination will never appear).
- Do Not Add Hotel with Hotel Chain Preference = Motel One (this combination will never appear).
- Do Not Add Hotel with Hotel Chain Preference = Vivanta by Taj (this combination will never appear).
And because we created a Mutually Bound Constraint, we are also invalidating this combination: Do Add a Hotel with Hotel Chain Preference = Not Applicable (this combination will never appear).
- You can use the Invalid Constraint feature to accomplish anything that the Bound Constraint feature can do.
- The Invalid Constraint feature is frequently less confusing for new users. Don’t hesitate to use the Invalid feature instead of the Bound Constraint feature.
- If you have more than 10 or so Bound Constraints or Invalid Constraints in a model, you might find that it is easier and faster to document your paired values in Excel. If so, we would recommend:
- Adding multiple paired values before you export into Excel.
- Ensuring that you use the exact spelling of values (e.g., cutting and pasting is usually safer than typing).
- Especially watch out for situations where you have multiple related Not Applicable values in a model. Would it make sense to create a Bound Constraint between, say, Hotel Chain Preference = Not Applicable’ and ‘Type of Room = Not Applicable ? In the example above, it would make sense to include that Bound Constraint. Every time Hotel Chain Preference = Not Applicable, we would want Type of Room to also be Not Applicable. Similarly, every time Type of Room = Not Applicable, we would want Hotel Chain Preference to also be Not Applicable. This is an example of a type of constraint that the human brain would handle effortlessly without even consciously applying logical rules.
Bound Constraints are rules for your model algorithm to ensure that two values must appear together. Knowing which type to use depends on your model structure and business and technology requirements.
We will be using this set of values to demonstrate (Figure 15):
Figure 15 - Values
To apply a Bound Constraint, you need to hover over value 1, click the green arrow icon to the right from the value name, do the same for value 2, and select the appropriate option from the three provided ones.
One-way Bound Pair
The underlying relationship type is many-to-one, represented by WHEN-THEN statements in the left column.
We would like to exclude combinations like Breed of Animal = Shiba Inu and Type of Animal = Cat from the generated scenarios.
We have two breeds for each of the first two types of animals. Therefore, we can set up four one-way bound pairs from Breed of Animal to Type of Animal (Figure 16):
Figure 16 - Breed
You can check the logic by reading the statement at the bottom of the dialog (Figure 16 - 1).
The order in which you set the pair matters.
In this case, we wouldn't be able to say WHEN Type of Animal = Cat THEN Breed of Animal = Siamese because that would prevent Cat + Persian from appearing, and would leave Persian without any possible pairing (Figure 18).
Figure 17 - Type
Mutually Bound Pair
The underlying relationship type is one-to-one, represented by ALWAYS-ALWAYS statements in the left column.
With Mutually Bound Constraints, the values are exclusive to each other. In this example (Figure 18), there is only one horse breed in the second parameter. So, with one constraint, we can specify that Horse should not be paired with Siamese, Persian, Shiba Inu, Golden Retriever, and Arabian should not be paired with Cat, Dog - i.e., Horse and Arabian must only be tested together.
Figure 18 - Breed
The final set of rules for this model (in the verbal form) would look like this (Figure 19):
Figure 19 - Final
In essence, Bound Constraints and Invalid Constraints perform similar tasks: they ensure that certain values only or never appear together - making all generated scenarios valid within the model scope. From a different point of view, Bound Constraints and Mutually Bound Constraints actually invalidate many values and Invalid Constraints bind many values.
Skip Constraints
Skip constraints are only available under the Advanced Mode (toggle in the top left of the Constraints screen).
Skip constraints allow for certain parameters to be excluded from Test cases when it is not appropriate for them to appear. It is a faster alternative to the Not applicable + Bound Pairs approach.
Creating Skip Constraints
For example, let’s use the model below for a flight booking system (Figure 20):
Figure 20 - Flight
In this model (Figure 20), only customers who are in First Class get in-flight dining. Coach and Business class customers do not get in-flight dining. Therefore, Test cases with Coach and Business class customers should have no value for Food Choice. Additionally, Coach customers do not even get Drink Choice.
Inside the model, we click Rules (Figure 21 - 1) and then Constraints (Figure 21 - 2) to access the Constraints screen (Figure 22) (the same way we did for Bound or Invalid constraints).
Figure 21 - Constraints
Click the toggle (Figure 22 - 1) to change to Advanced Mode – that is the only way to implement Skip Constraints.
Figure 22 - Advanced Constraints
Once in Advanced Mode, we use the following syntax for Skip constraints:
Parameter[Parameter Value] >> Parameter to be skipped
or
Parameter[Parameter Value] >> First Parameter to be skipped :: Last parameter to be skipped (The boundaries are inclusive)
So, for our example, the syntax is this (Figure 23):
Figure 23 - Skip Constraint Syntax
The ability to include multiple values in the same Constraint is an aspect of the Advanced Mode that is not limited to Skip constraints.
Now, we generate our Test suite by navigating to Scenarios (Figure 24 - 1).
Figure 24 - Scenarios
Any row that includes Ticket Type as Business will have no value for Food Choice (Figure 25). Any row that includes Ticket Type as Coach will have no value for Food Choice or Drink Choice (Figure 25).
Figure 25 - Skips based on Ticket Type
The final component of Skip Constraints is the skip-to-end capability. In this case, we can tell Xray Test Case Designer that for some parameter values, it must skip all parameters listed below (Figure 26) on the Parameters screen.
So, for our example, we could return to our Parameters screen and reorder the elements so that Food Choice and Drink Choice are the last two parameters (Figure 26):
Figure 26 - Parameters
Then, we use the syntax below to skip to the end:
Parameter[Parameter Value] >> First Parameter to be skipped::!!
For our example, that looks like this (Figure 27):
Figure 27 - Syntax
Lastly, you can do a quick review of available options and syntax by clicking the Usage button (Figure 28 - 1).
Figure 28 - Usage
Viewing a Constraint
Go to My Test Models (Figure 29 - 1) and enter a Test Model (Figure 29 - 2).
Figure 29 - Test model
You will enter the Parameters screen (Figure 30).
Click Rules (Figure 30 - 1) and then Constraints (Figure 30 - 2) to access the Constraints screen (Figure 31).
Figure 30 - Constraints
A list with all the simple bound and invalid Constraints will be displayed (Figure 31)
Figure 31 - Constraints
Editing a Constraint
It is not possible to edit a Constraint in the Standard View. In the Standard View, you are only allowed to delete and create a new Constraint.
Go to My Test Models (Figure 29 - 1) and enter a Test Model (Figure 29 - 2).
You will enter the Parameters screen (Figure 30).
Click Rules (Figure 30 - 1) and then Constraints (Figure 30 - 2) to access the Constraints screen (Figure 31).
Change view to bulk (Figure 32 - 1). You can edit the Constraint type by selecting the corresponding operator (Figure 32 - 2).
Figure 32 - Bulk Edit
Once you're finished, click the Save button (Figure 32 - 3).
Deleting a Constraint
Go to My Test Models (Figure 33 - 1) and enter a Test Model (Figure 33 - 2).
Figure 33 - Test models
You will enter the Parameters screen (Figure 34).
Click Rules (Figure 34 - 1) and then Constraints (Figure 34 - 2) to access the Constraints screen (Figure 35).
Figure 34 - Constraints
To delete a constraint, hover over the constraint and click the trash icon (Figure 35 - 1).
Figure 35 - Delete
Another option is to, in the Bulk view (Figure 36), delete the desired Constraint line (Figure 36 - 1).
Figure 36 - Bulk Delete