Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Parameters are defined within datasets. However, it is possible to reference a parameter that is not yet defined (meaning it does not have a corresponding name within the dataset). In this case, the parameter will be highlighted in red.

Image RemovedImage Added


Info

Currently, the test parameterization feature is only available for Manual Tests. Xray will support defining parameters and use datasets on BDD and generic test types as well.

...

Section
Column
width20px


Column
Let's consider the following example:

We create a test for checking if we can add books to a shopping cart in our online book store. We have parameters like Item, Price, Rating, In Stock, Condition, and Format. There are certain books we want to test (3 in this case). However, we would like to test all combinations of these books with the following parameters: Gift and Quantity

In this case, these will be the seeding parameters:

  • Item,
  • Price,
  • Rating,
  • In Stock,
  • Condition,
  • Format

Because we want to test these items with all the combinations of Gift and Quantity parameters, we can create these as Combinatorial parameters: 

  • Gift*
  • Quantity*

Combinatorial parameters are denoted with an asterisc (*) suffix. 


Image RemovedImage Added


Xray will generate all possible combinations upon execution automatically. 

Image RemovedImage Added


Dataset scopes

...

The test dataset is the default. If there is the need to override or change this dataset you can do this at the planning or execution phases.


Image RemovedImage Added


Dataset limits

...

The parameters will be unfolded on the execution screen, just like Test cases. For this, the dataset must have the same parameters, matched by name.

Image RemovedImage Added

Execution

All iterations for a given test are executed within the context of the same test run. Each iteration can be expanded and the steps executed individually. The step parameters will be replaced by the corresponding iteration values. The steps affect the iteration status that, in turn, affects the overall test run status.


Image RemovedImage Added


TODO: link to execution page

...

  1. Create or edit a given step using either the inline view within the test issue or the steps dialog.
  2. When specifying a step, to reference a parameter you have two options:
    1. Start typing ${. If there is a default dataset defined on the test, you should see a list of the available parameters. Choose the desired parameter using the cursor keys or mouse. The parameter will be placed with the text.
    2. Use the toolbar button ${. After pressing this button, and if there is a default dataset defined on the test, you should see a list of the available parameters. Choose the desired parameter using the cursor keys or mouse. The parameter will be placed on the cursor position.


Image RemovedImage Added

Creating a dataset

...

a. create an ad hoc list just for this parameter. You need to specify the values for the list.

Image RemovedImage Added

b. use a project predefined list.

Image RemovedImage Added

2.5. Press "Save" to create the parameter. The parameter must be placed on the dataset. 

Image RemovedImage Added


3. Adding combinatorial parameter values:

...

3.1. For text parameters, just type the value and press the check button next to the field.

Image RemovedImage Added

3.2. For list parameters, select an option and press the check button next to the field.

Image RemovedImage Added


4. Adding rows (filling the parameter values):

...

You can navigate between cells of the same row and also between rows using the keyboard: TAB (forward), SHIFT+TAB (backward).

Image RemovedImage Added

4.2. To create new rows, you can press the "New" button below the table or navigate using the keyboard from the last row (a new row will be created automatically by navigating forward on the last cell of the last row).

Image RemovedImage Added


5. Converting a seeding parameter to a combinatorial parameter:

...

5.1.  Next to each column, there is an options button. Clicking this button will reveal a menu with options for Editing, Deleting, and Converting parameters.

Image RemovedImage Added

5.2. Click on  "Convert to combinatorial parameter."

Image RemovedImage Added


6. Converting a combinatorial parameter to a seeding parameter:

...

6.1. Next to each combinatorial column, there is an options button. Clicking this button will reveal a menu with options for Editing, Deleting, and Converting parameters.

Image RemovedImage Added

6.2. Click on the "Convert to non-combinatorial parameter".

...

7.1. Next to each combinatorial column, there is an options button. Clicking this button will reveal a menu with options for Editing, Deleting, Converting parameters, and Generating all combinations.

Image RemovedImage Added

7.2. Click on the "Generate all combinations." A confirmation dialog will appear. Confirming the changes will apply the cartesian product between the combinational parameter values and the seeding parameter rows.

Image RemovedImage Added

8. After having performed all changes in the dataset, you need to press the "Save" button in order to persist the dataset into the database.

...

- Create new parameters - If checked, it will create non-existing parameters automatically, based on the CSV column name. Otherwise, it will just append/update the values on the existing parameters.

Image RemovedImage Added

4. Press "Import" to import the external dataset.

...

2.2. Open the dataset dialog by selecting the "Dataset" action from the test row options menu.

Image RemovedImage Added

3. The dataset dialog is opened.

3.1. If there is a dataset defined on a parent level, it is possible to Override the parent dataset and modify its values.

Image RemovedImage Added

3.2. Otherwise, you can start defining a new dataset at this level.

...