You can now define additional custom fields for Test Runs. These fields can be useful to add extra information to Test Runs usually only available during or after executing Tests.
Test Run custom fields can be configured by project and by Test Type. Therefore, these settings will not affect other projects within your Jira instance. For example, it is possible to have custom fields just for Manual Tests within a project.
Reporting
The Test Runs List report and gadget can already display Test Run Custom Field values for each Test Run.
Within a Test Execution issue, it is also possible to display Test Run Custom Field columns and to filter Test Runs by Test Run Custom Field values.
Ability to define variables in the Test specification (e.g. the steps) that can be replaced by values inherited from different sources (a calling test for example). In that scenario, the called test is used as a parameterized template (e.g. "login as the specified user").
title
Parameterized Tests and data driven testing
Panel
Test parameterization is a powerful practice that allows the same test to be executed multiple times with different parameters. Parameters are similar to input values (variables) that can change with each execution.
Parameterized tests in Xray are defined just like any other test with the addition of some parameter names within the specification using the following notation: ${PARAMETER_NAME}. This notation is used to reference parameters within thetest steps.
Preconditionissues can also be parameterized by including parameter names in the precondition specification.
The parameters, along with their values, are defined within adataset. A dataset is a collection of data represented with a tabular view where every column of the table represents a particular variable (orparameter), and each row corresponds to a given record (oriteration) of the dataset. The number of rows in the dataset determines the number of iterations to execute.
A dataset can be defined in the following entities/scopes:
Test (default dataset)
Test Plan - Test
Test Execution - Test (Test Run)
The closest dataset to the test run will be the one used to generate the iterations, effectively overriding any dataset defined in higher levels.
Image Added
All iterations for a given test are executed within the context of the same test run. Each iteration can be expanded, and the steps executed individually. The step parameters will be replaced by the corresponding iteration values. The steps affect the iteration status that, in turn, affects the overall test run status.
Ability to call an existing Test from the step of another Test and thus use tests as reusable building blocks for the composition of more complex testing scenarios.
The called test can be executed independently or be used as a step of a broader test (e.g. "login as the admin user").
Expand
title
Data-driven testing
Panel
borderColor
#EAECEE
borderWidth
1px
borderStyle
solid
Ability to perform data-driven testing (i.e. execute the same test against multiple parameters, for different combinations of data).
Data-driven testing will be implemented using the foundations of test parameterization.
Data may be defined at multiple levels including:
the test level (default values)
the caller test
the Test Plan
the Test Execution
Data might also be imported from external sources (like CSV files).
Wish list for future versions
...
Expand
title
Automated Steps Library
Panel
borderColor
#EAECEE
borderWidth
1px
borderStyle
solid
TheAutomated Steps Library will be a project-level steps library for BDD Tests/Preconditions, containing all the Gherkin steps referred by these issues.
Thus, it provides a facility to store and manage all steps, encouraging users to easily reuse steps across Tests/Preconditions with step autocompletion and discoverability when editing BDD scenarios and backgrounds with Xray.
Expand
title
Test Environment variables
Panel
borderColor
#EAECEE
borderWidth
1px
borderStyle
solid
This feature will allow test engineers to create or generate environments with variables such as Browser, Operating System, Database, etc. These environments (or configurations) can be managed and assigned to Test Plans and Test Plan folders. Test Executions can then be created automatically for each environment/configuration.
Xray will provide a report to track the progress of Test Runs by environment in the context of a specific Test Plan.
