Page History
...
Table of Contents | ||
---|---|---|
|
Overview
Scripts are a line-by-line description descriptions containing the information about the system transactions that should be performed to validate the application or system under test. Test The test script should list out each step that should be taken with the expected results.
With The Xray Test Case Designer Scripts feature , create creates detailed, consistent execution instructions with conditional conditionally expected results for your manual testing efforts.
You can quickly transform optimized test data like this
...
… into customizable scripts. You can even add automatically generated Expected Results to your steps if you want to.
...
Automate
Automate scripts makes make it easy to write behavioral-driven development (BDD) automated acceptance tests in Gherkin format as data-driven scripts. These data-driven test scripts are based on a test model, so they are making them more maintainable than traditional BDD/Gherkin test scripts.
Automated test scripts are notoriously expensive to maintain, and BDD/Gherkin test scripts are not immune to this problem. Normally Typically changes to the system under test , or just changes to how the system will be tested means mean having to make changes to tens, hundreds, or (in some cases) thousands of feature files. With data-driven Automate test scripts though, that's not the case. Adding or removing parameter values, requirements, or constraints to the test model updates the data driving the automate Automate scripts, but there is no need to update the scripts themselves.
In addition, Automate Automated scripts makes make it easier to write correct Gherkin scripts through its syntax highlighting and autocomplete features.
How to create
...
Create or open a test model and
...
navigate to the "Scripts" menu. Once there, select the "
...
Automate" option in the menu.
...
You can create as many Gherkin/Robot Framework
...
files as you need for your test model, one per tab.
...
Click the "+" button in the tab bar to add another
...
file.
...
Click the pencil icon on a tab to rename a
...
file
...
and the
...
trash can icon to delete a
...
file.
Each feature file can include as many Scenario and Scenario Outline sections as you need. Variations are then automatically provided for you from your generated test cases. When you use a Scenario Outline the variations are provided in an Examples data table. When you use a Scenario, it acts as a template and a unique Scenario is automatically generated for each test case.
Edit your Gherkin feature in the editor in the panel on the left, and the generated Examples data tables and generated Scenario sections are shown in the preview on the right. You must generate test cases first before the preview becomes active.
To include parameter values from your test cases in a Scenario or Scenario Outline, simply enter the parameter name surrounded by angle brackets into a step. Automate will help you by autcompleting parameter names once you type <.
Feature: Use Automate
Scenario: including parameters in Gherkin
Given I select <State> from the state dropdown
And I input <Age> into the age input
When I push the "Go" button
Then I should see the completion indicator
From the script above, Automate will automatically generate a Scenario for every test case that has a unique combination of the included parameters, "State" and "Age", and will substitute parameter values from each test case into the text of the step. A generated Scenario might look like this:
# OUTPUT
Feature: Use Automate
Scenario: including parameters in Gherkin
Given I select North Carolina from the state dropdown
And I input 18 into the age input
When I push the "Go" button
Then I should see the completion indicator
# etc, one Scenario block for each unique test case
If you use Scenario Outline in place of Scenario in the same example, Automate will instead automatically generate an Examples table with one column for each included parameter, and one row for for every test case that has a unique combination of the included parameters. Here is an example of that output:
# OUTPUT
Feature: Use Automate
Scenario Outline: including parameters in Gherkin
Given I select <State> from the state dropdown
And I input <Age> into the age input
When I push the "Go" button
Then I should see the completion indicator
Examples:
| State | Age |
| North Carolina | 18 |
| Indiana | 28 |
| New Jersey | 42 |
# etc, one row for each unique test case
If a specific behavior doesn't apply to every test case, you can restrict a Scenario or Scenario Outline to only include the test cases that match a particular set of parameter values. The selector syntax utilizes curly braces { } and square brackets [ ] as such: {Parameter[Value 1]} -or- {Parameter[Value 1, Value 2, ...]} Automate will help you with the syntax by autcompleting once you type {. Here's an example which only includes test cases that have "North Carolina" or "Indiana" as the value of the "State" parameter:
Feature: Automate selectors
Scenario Outline: including parameters in Gherkin
Given I select {State[North Carolina, Indiana]} from the state dropdown
And I input <Age> into the age input
When I push the "Go" button
Then I should see the completion indicator
Which results in only test cases with "North Carolina" and "Indiana" in the Examples table:
# OUTPUT
Feature: Automate selectors
Scenario Outline: including parameters in Gherkin
Given I select <State> from the state dropdown
And I input <Age> into the age input
When I push the "Go" button
Then I should see the completion indicator
Examples:
| State | Age |
| North Carolina | 23 |
| Indiana | 18 |
| North Carolina | 48 |
| Indiana | 48 |
# etc, one row for each unique test case with "State" as "North Carolina" or "Indiana"
It is also possible to invert a restriction by adding an exclamation point just after the opening curly bracket, like so: {!Parameter[Value 1, Value 2]} This includes all the test cases in which "Parameter" is not "Value 1" or "Value 2".
If you find yourself needing to restrict the test cases of every behavior in a feature file, you can conveniently use the same selector syntax in the Background section of a feature file, like this:
Feature: global restrictions in the Background section
Background:
Given I select {State[New Jersey]} from the state dropdown
# Every Scenario/Outline block is now restricted to test cases
# in which the "State" parameter has the value "New Jersey"
In addition to the parameters defined in the test model, Automate provides a few "pseudo-parameters" that can be helpful in some situations:
Test NumberA purely sequential number for the row or generated Scenario.Test CaseThe index of the generated test case that corresponds to this Examples row or generated Scenario. This number may have gaps and so not be purely sequential.Expected OutcomeA comma-separated list of any Expected Outcomes that apply to the test case as defined with Requirements.
Here's an example using all of the pseudo-parameters:
# OUTPUT
Feature: Use Automate pseudo-parameters
Scenario Outline:
Given this is test <Test Number> for test case <Test Case>
When I select <State> from the state dropdown
And I input <Age> into the age input
And I push the "Go" button
Then the claim is <Expected Outcome>
Examples:
| Test Number | Test Case | State | Age | Expected Outcome |
| 1 | 3 | North Carolina | 18 | denied |
| 2 | 5 | Indiana | 28 | |
| 3 | 7 | New Jersey | 42 | accpeted |
# etc, one row for each unique test case
Bring back these tips again by click the Usage button on the right side of screen, above the Preview section.
Gherkin Feature File
The Hexawise Automate screen is a smart Gherkin feature file editor. It is aware of both the parameters, parameter values, & generated test cases of the current Hexawise model and the Gherkin test script syntax. It makes it easy to create a feature file with data-driven Scenario and Scenario Outline sections that are populated by the generated test cases of the Hexawise model.
Feature
Each Gherkin script has a Feature as its first keyword followed by : and a little bit of text to describe the feature, usually in the form of a user story which is demonstrated in the placeholder text. This section does not support parameterization of any kind.
Background
A Background section is useful when a set of initial setup steps is repeated for every Scenario and Scenario Outline in the feature file. The steps of the Background are run before each Scenario Outline and Scenario. Starting a new line in Background and further sections brings up a suggestion dialog with standard Gherkin keywords.
Keep in mind that restricting a parameter in the Background (with {}, as explained below) is valid syntax and is used to apply only the selected value to all Scenarios or Scenario Outlines for a given Feature.
Scenario
Gherkin Feature File
The Automate screen is an intelligent Gherkin feature file editor. It is aware of the parameters, values, & generated test cases of the current model and the Gherkin syntax. It makes it easy to create a feature file with data-driven Scenario and/or Scenario Outline sections populated by the model's generated test cases.
Feature
Each Gherkin script has a Feature as its first section keyword followed by ":" and a little bit of text to describe the scope, usually in the form of a user story which is demonstrated in the placeholder text. This section does not support parameterization of any kind.
Background
A Background section is useful when a set of initial setup steps is repeated for every Scenario and/or Scenario Outline in the feature file. The steps of the Background are run before each Scenario Outline and Scenario. Starting a new line in the Background and further sections brings up a suggestion dialog with standard Gherkin keywords.
Remember that restricting a parameter in the Background (with {}, as explained below) is valid syntax and applies only the selected value to all Scenarios or Scenario Outlines for a given Feature.
Scenario
Next in the feature Next in the feature file is the enumeration of different testing scenarios. One way to describe a test case is with a Scenario block. It contains sequential steps, expressed as plain English Given/When/Then statements, that describe describing the necessary actions for executing to execute the automated test script. If the Hexawise test model generates 28 test cases, then 28 Scenario sections would need to be written out in order to achieve the combinatorial coverage.
However, instead of writing out all 28 test cases individually, with Hexawise Automate, a single data-driven Scenario template that includes the <Parameter Name> syntax can generate all 28 needed Scenario sections in the exported feature file. You can check the number of associated test cases in the Preview panel.file. You can check the number of associated test cases in the Preview panel.
For example, writing the test step:
And they select <Color> for the car's color.
This means that for each generated Scenario section (one for each test case), the <Color> portion of the test step will be replaced with the value of the Color parameter in the generated test case:
And they select Deep Blue Metallic for the car's color.
Simply typing "<" Hexawise Automate provides a way to include the value of any test case parameter in a Scenario using the <Parameter Name> syntax. Simply typing < will populate a list of autocomplete suggestions related to all of the parameters in the model, also adding:
Test Number (sequential number within the Scenario),
Test Case (constant number of the row in the complete table on the Test Cases Scenarios screen)
Expected Outcome (from the Requirements screen, if applicable)
Wherever the <Parameter Name> appears in the test steps it will be replaced by the specified parameter's value for the test case being tested. This is conceptually very similar to how manual testing auto-scripts work in Hexawise.
For example, writing the test step:
...
Forced Interactions screen, if applicable)
Wherever the <Parameter Name> appears in the test steps, it will be replaced by the specified parameter's value from the associated row on the Scenarios screen. Automate feature files can contain as many data-driven Scenario sections as needed to automate all the testing ideas contained in the model.
Info |
---|
it is common to put dynamic elements into quotes (e.g., they select "<Color>" for the car's color) to make them more visually distinguishable). |
Scenario Outline
A Scenario Outline behaves almost exactly like a data-driven Scenario. When using a Scenario Outline, instead of having 28 Scenario sections in the export Means that for each generated Scenario section (one for each test case), the <Color> portion of the test step will be replaced with the value of the Color parameter in the generated test case:
And they select Deep Blue Metallic for the car's color
Writing a export has one data-driven Scenario with at least one <Parameter> reference for each of the inputs in the 2-way test set for this model would lead to a .feature file export containing 49 Scenario sections in the feature file, one for each of the 49 Hexawise generated test cases.Outline script block with a Gherkin Examples data table attached to it that contains 28 rows (one for each test case).
The Preview panel shows the first 10 rows of the Examples data table for each Scenario Outline. The order of parameters in the Examples is dictated by the script, not by the Parameters screen.
An A Hexawise Automate feature file can contain as many data-driven Scenario Outline sections as needed to automate all the testing ideas contained in the Hexawise test model.
Note: it is common to put dynamic elements into quotes (e.g., And they select "<Color>" for the car's color) to make them more visually distinguishable).
Scenario Outline
A Scenario Outline behaves almost exactly like a data-driven Scenario. When using a Scenario Outline, instead of having 49 Scenario sections in the export (one for each test case), the export has one data-driven Scenario Outline with a Gherkin Examples data table attached to it that contains 49 rows (one for each test case). The Preview panel shows the first 10 rows of the Examples data table for each Scenario Outline. The order of parameters in the Examples is dictated by the script, not by the Inputs tab of Hexawise.
A Hexawise Automate feature file can contain as many Scenario Outline sections as needed to automate all the testing ideas contained in the Hexawise test model and can include both Scenario and Scenario Outline sections in the same feature file.
...
model.
You can include both the Scenario and Scenario Outline sections in the same feature file. Where possible, using Scenario Outline is preferable to Scenario as it minimizes the number of items in test management and automation frameworks, which simplifies planning, execution, reporting, and maintenance.
Edit Automate scripts
To edit a script you need to make the necessary changes within the editor and save the changes by using the buttons for that effect or using the keyboard shortcuts:
Keyboard shortcuts | |
---|---|
Save the script in the active tab | Ctrl+S |
Save all the scripts in the test model | Ctrl+Shift+S |
Open the auto-complete dropdown | Ctrl+Space |
Undo / Redo | Ctrl+Z / Ctrl+Y |
To edit the script name, click the option right to the name on the tab:
Delete Automate scripts
To delete an automated script, select the tab and click the bin icon right after the name:
Manual Scripts
Remember Mad Libs?
Creating Auto-manual scripts in Xray Test Case Designer is similar to that. Instead of adding adjectives and nouns into pre-formed sentences, however, you’ll be more like the author of the Mad Libs sentences themselves. You need to:
- Create sentences containing execution instructions that will be common to most of the test scripts and
- Identify “spaces” to indicate where Xray Test Case Designer should “fill in the blanks” you’ve left with test data appropriate for each scenario.
How to create
UI Steps | ||||||
---|---|---|---|---|---|---|
| ||||||
Don’t forget to save each step before you add your next one! Thankfully, Xray Test Case Designer notifies you that there are unsaved edits under the last edited step that there are unsaved edits. Click on different test cases at the bottom half of your screen (preview section that mirrors the Scenarios screen) to see how your script steps will change. Finally, in the “Finish” section, you may want to add some instructions that will appear only once at the end of all of the scenario scripts. |
Incorporating “Parameterized expected results” into your plans
In the tests shown above, for example, we might want to include this Expected Result every time the necessary values appear together in a test case:
...
UI Steps | ||||||
---|---|---|---|---|---|---|
| ||||||
|
Info |
---|
Important Essential Usage Tips and things to know about the Expected Results feature1. This feature is a partial solution for straightforward Expected Results. It primarily exists so that you won’t have to manually type many , simple expected results manually. It is not designed to handle especially complex rules that you might have. 2. Be sure you understand the similarities & differences between Xray Test Case Designer Expected Results in the automated Manual Scripts screen and Expected Outcomes in the “Forced Interactions” featureone. There is a big, yet subtle, difference:
Suppose you want to define an Expected Result that requires 3 or more specific Values to appear in a single test script (, and you’re creating pairwise sets of tests ),. In that case, you should use the “Forced Interactions” feature or higher algorithm strength to guarantee the scenario is included in your suite. Then use the Manual Auto-Scripts feature to document the Expected Result for export. Xray Test Case Designer Automate can directly leverage that last column on from Forced Interactions directly as an internal variable. If you want to define an Expected Result that requires 2 or fewer specific Values to appear in a single test script (and you’re creating pairwise sets of tests), use the Manual Auto- Scripts feature without additional prep work. |
...