Page History
...
Parameters are embedded within the test specifications using the following notation: ${PARAMETER_NAME} for Steps definitions and Unstructured tests,and <PARAMETER_NAME> for Gherkin.
This notation is used to reference parameters within the test steps test definition. You can reference parameters on the Action, Data, Expected Result, and even on any test step custom fields that are in any text-based test step custom field, Unstructured definition, or Gherkin Scenarios.
Parameters are defined within datasets. However, it is possible to reference a parameter that is not yet defined (meaning it does not have a corresponding name within the dataset). In this case, the parameter will be highlighted in red.
...
Datasets
...
Currently, the test parameterization feature is only available for Manual Tests. Xray will support defining parameters and use datasets on BDD and generic test types as well.
Datasets
The parameters, along with The parameters, along with their values, are defined within a dataset. A dataset is a collection of data represented with a tabular view where every column of the table represents a particular variable (or parameter), and each row corresponds to a given record (or iteration) of the dataset.
...
| Section | |||||||
|---|---|---|---|---|---|---|---|
|
...
All iterations for a given test are executed within the context of the same test run. Each iteration can be expanded and the steps executed individually. The step parameters will be replaced by the corresponding iteration values. The steps affect the iteration status that, in turn, affects the overall test run status.
Please check the Execution page for more details.
...
Defining a parameterized test
...
Manual Steps
- Create or edit a given step using either the inline view within the test issue or the steps dialog.
- When specifying a step, to reference a parameter you have two options:
- Start typing ${. If there is a default dataset defined on the test, you should see a list of the available parameters. Choose the desired parameter using the cursor keys or mouse. The parameter will be placed with the text.
- Use the toolbar button ${. After pressing this button, and if there is a default dataset defined on the test, you should see a list of the available parameters. Choose the desired parameter using the cursor keys or mouse. The parameter will be placed on the cursor position.
Creating a dataset
In order to create or edit the default dataset (within a test):
1. Press the "Dataset" button. This will open the dataset dialog. Here, you can define a dataset by creating parameters and adding values to them.
2. Creating parameters:
2.1. Press the "Create parameter" button located on the dataset main toolbar or on the empty view. This will open a small dialog to specify the parameter attributes.
2.2. Specify the name. Parameter names must start with a letter or underscore and can only contain letters, numbers, a space between words, "_", "-" and a max of 64 characters.
2.3. Check the Combinatorial checkbox if you are creating a combinatorial parameter.
2.4. Choose the parameter type: Text or List. If the parameter type is a List you can:
a. create an ad hoc list just for this parameter. You need to specify the values for the list.
b. use a project predefined list.
2.5. Press "Save" to create the parameter. The parameter must be placed on the dataset.
3. Adding combinatorial parameter values:
Once you have at least one parameter, you can start filling their values and adding new iterations.
A placeholder is provided within each combinatorial parameter. In order to add new values to combinatorial parameters:
3.1. For text parameters, just type the value and press the check button next to the field.
3.2. For list parameters, select an option and press the check button next to the field.
4. Adding rows (filling the parameter values):
4.1. Once you have non-combinatorial parameters, an empty placeholder row will appear so that the parameters can be populated for the default iteration.
Editing parameter values is as simple as editing their corresponding cells. The values will be kept when the cell loses focus.
You can navigate between cells of the same row and also between rows using the keyboard: TAB (forward), SHIFT+TAB (backward).
4.2. To create new rows, you can press the "New" button below the table or navigate using the keyboard from the last row (a new row will be created automatically by navigating forward on the last cell of the last row).
5. Converting a seeding parameter to a combinatorial parameter:
It is possible to convert an existing parameter to a combinatorial parameter. This will remove the parameter column from the seeding parameters table and group all remaining rows automatically. A new combinatorial parameter will be created by grouping all the values.
5.1. Next to each column, there is an options button. Clicking this button will reveal a menu with options for Editing, Deleting, and Converting parameters.
5.2. Click on "Convert to combinatorial parameter."
6. Converting a combinatorial parameter to a seeding parameter:
It is also possible to convert a combinatorial parameter back into a seeding parameter.
6.1. Next to each combinatorial column, there is an options button. Clicking this button will reveal a menu with options for Editing, Deleting, and Converting parameters.
6.2. Click on the "Convert to non-combinatorial parameter".
7. Generating all combinations:
You don't need to generate all the combinations for a given dataset in order to execute all iterations. Xray will do this automatically for all of the combinatorial parameters. However, sometimes we don't need all the combinations. In order to do this, we can generate all combinations and remove some iterations afterward.
7.1. Next to each combinatorial column, there is an options button. Clicking this button will reveal a menu with options for Editing, Deleting, Converting parameters, and Generating all combinations.
7.2. Click on the "Generate all combinations." A confirmation dialog will appear. Confirming the changes will apply the cartesian product between the combinational parameter values and the seeding parameter rows.
8. After having performed all changes in the dataset, you need to press the "Save" button in order to persist the dataset into the database.
Importing a dataset from a CSV file
Besides defining a dataset by creating parameters and setting their values directly using the Xray UI, it is also possible to import an existing dataset from a CSV file.
In order to import a CSV file:
1. Open the dataset dialog.
2. Choose the "Import from CSV" option available on the Import dropdown button (located on the empty state view or on the dataset main button toolbar). A dialog will appear.
3. Select and fill the following options:
- File - find a local CSV file with the desired dataset
- CSV delimiter - defaults to "," but you can choose any other character
- File encoding - defaults to UTF-8
- Overwrite existing parameter values - if checked, this option will delete any values from existing parameters
- Create new parameters - If checked, it will create non-existing parameters automatically, based on the CSV column name. Otherwise, it will just append/update the values on the existing parameters.
4. Press "Import" to import the external dataset.
Generic or Unstructured definition
- Edit the definition using the inline view within the test issue.
- To reference a parameter, you have two options:
- Start typing ${. If a default dataset is defined on the test, you should see a list of the available parameters. Choose the desired parameter using the cursor keys or mouse. The parameter will be placed with the text.
- Use the toolbar button ${. After pressing this button, and if there is a default dataset defined on the test, you should see a list of the available parameters. Choose the desired parameter using the cursor keys or mouse. The parameter will be placed on the cursor position.
| Info |
|---|
To define parameters on unstructured tests, the Jira markup wiki language must be activated in the Generic Test Definition custom field. To enable the Rich Text Editor, you have to set the renderer of the definition custom field to Wiki Style Renderer. |
Gherkin Scenarios
- Create or edit a given step using either the inline view within the test issue or the steps dialog.
- When specifying a Scenario, to reference a parameter you should start typing <. If there is a default dataset defined on the test, you should see a list of the available parameters. Choose the desired parameter using the cursor keys or mouse. The parameter will be placed with the text.
Creating a dataset
In order to create or edit the default dataset (within a test):
1. Press the "Dataset" button. This will open the dataset dialog. Here, you can define a dataset by creating parameters and adding values to them.
2. Creating parameters:
2.1. Press the "Create parameter" button located on the dataset main toolbar or on the empty view. This will open a small dialog to specify the parameter attributes.
2.2. Specify the name. Parameter names must start with a letter or underscore and can only contain letters, numbers, a space between words, "_", "-" and a max of 64 characters.
2.3. Check the Combinatorial checkbox if you are creating a combinatorial parameter.
2.4. Choose the parameter type: Text or List. If the parameter type is a List you can:
a. create an ad hoc list just for this parameter. You need to specify the values for the list.
b. use a project predefined list.
2.5. Press "Save" to create the parameter. The parameter must be placed on the dataset.
3. Adding combinatorial parameter values:
Once you have at least one parameter, you can start filling their values and adding new iterations.
A placeholder is provided within each combinatorial parameter. In order to add new values to combinatorial parameters:
3.1. For text parameters, just type the value and press the check button next to the field.
3.2. For list parameters, select an option and press the check button next to the field.
4. Adding rows (filling the parameter values):
4.1. Once you have non-combinatorial parameters, an empty placeholder row will appear so that the parameters can be populated for the default iteration.
Editing parameter values is as simple as editing their corresponding cells. The values will be kept when the cell loses focus.
You can navigate between cells of the same row and also between rows using the keyboard: TAB (forward), SHIFT+TAB (backward).
4.2. To create new rows, you can press the "New" button below the table or navigate using the keyboard from the last row (a new row will be created automatically by navigating forward on the last cell of the last row).
5. Converting a seeding parameter to a combinatorial parameter:
It is possible to convert an existing parameter to a combinatorial parameter. This will remove the parameter column from the seeding parameters table and group all remaining rows automatically. A new combinatorial parameter will be created by grouping all the values.
5.1. Next to each column, there is an options button. Clicking this button will reveal a menu with options for Editing, Deleting, and Converting parameters.
5.2. Click on "Convert to combinatorial parameter."
6. Converting a combinatorial parameter to a seeding parameter:
It is also possible to convert a combinatorial parameter back into a seeding parameter.
6.1. Next to each combinatorial column, there is an options button. Clicking this button will reveal a menu with options for Editing, Deleting, and Converting parameters.
6.2. Click on the "Convert to non-combinatorial parameter".
7. Generating all combinations:
You don't need to generate all the combinations for a given dataset in order to execute all iterations. Xray will do this automatically for all of the combinatorial parameters. However, sometimes we don't need all the combinations. In order to do this, we can generate all combinations and remove some iterations afterward.
7.1. Next to each combinatorial column, there is an options button. Clicking this button will reveal a menu with options for Editing, Deleting, Converting parameters, and Generating all combinations.
7.2. Click on the "Generate all combinations." A confirmation dialog will appear. Confirming the changes will apply the cartesian product between the combinational parameter values and the seeding parameter rows.
8. Move left/right:
You can re-order any parameters in your dataset, by selecting the "Move left/right" option.
8.1. Open the options menu in either a Combinatorial or non-combinatorial (seeding) parameter, you will have a "Move left/right" option available. Notice that if the parameter is already place on the most outer left (right) position, the "Move left" (right) option is not available.
8.2. Select the move left/right option, This will swap the positions of the parameters.
9. After having performed all changes in the dataset, you need to press the "Save" button in order to persist the dataset into the database.
Importing a dataset from a CSV file
Besides defining a dataset by creating parameters and setting their values directly using the Xray UI, it is also possible to import an existing dataset from a CSV file.
In order to import a CSV file:
1. Open the dataset dialog.
2. Choose the "Import from CSV" option available on the Import dropdown button (located on the empty state view or on the dataset main button toolbar). A dialog will appear.
3. Select and fill the following options:
- File - find a local CSV file with the desired dataset
- CSV delimiter - defaults to "," but you can choose any other character
- File encoding - defaults to UTF-8
- Overwrite existing parameter values - if checked, this option will delete any values from existing parameters
- Create new parameters - If checked, it will create non-existing parameters automatically, based on the CSV column name. Otherwise, it will just append/update the values on the existing parameters.
4. Press "Import" to import the external dataset.
| Info | ||
|---|---|---|
| ||
You can also import combinational parameters. Any parameter with an asterisk suffix will be considered a combinational parameter. E.g., Quality*. In this case, the "Create new parameters" option must be enabled. |
Exporting a dataset to a CSV file
To export a dataset to a CSV file:
1. Open the Dataset dialog.
2. Click the options button next to the import button. Clicking this button will reveal a menu with options for Export to CSV and Delete.
3. Click the Export to CSV. A file should be downloaded.
| Info | ||
|---|---|---|
| ||
The delimiter used to generate the CSV file is a comma. The combinatorial parameters will be exported with an asterisk as a suffix | ||
| Info | ||
| ||
You can also import combinational parameters. Any parameter with an asterisk suffix will be considered a combinational parameter. E.g., Quality*. In this case, the "Create new parameters" option must be enabled. |
Overriding a dataset in a test plan or test execution
...