Starting from v4.0, It is possible to define additional Test Run custom fields. These fields can be useful to add extra information to Test Runs, usually only available during or after executing Tests.
Test Run custom fields can be configured by project and by Test Type. Therefore, these settings will not affect other projects within your Jira instance. For example, it is possible to have custom fields just for Manual Tests within a project.
Reporting
The Test Runs List report provides an additional column that displays the Test Run Custom Field values for each Test Run.
It is also possible to include these fields on the Tests data-table within the Test Execution issue.
Test parameterization is a powerful practice that allows the same test to be executed multiple times with different parameters. Parameters are similar to input values (variables) that can change with each execution.
Parameterized tests in Xray are defined just like any other test with the addition of some parameter names within the specification using the following notation: ${PARAMETER_NAME}. This notation is used to reference parameters within thetest steps.
Preconditionissues can also be parameterized by including parameter names in the precondition specification.
The parameters, along with their values, are defined within adataset. A dataset is a collection of data represented with a tabular view where every column of the table represents a particular variable (orparameter), and each row corresponds to a given record (oriteration) of the dataset. The number of rows in the dataset determines the number of iterations to execute.
A dataset can be defined in the following entities/scopes:
Test (default dataset)
Test Plan - Test
Test Execution - Test (Test Run)
The closest dataset to the test run will be the one used to generate the iterations, effectively overriding any dataset defined in higher levels.
Image Added
All iterations for a given test are executed within the context of the same test run. Each iteration can be expanded, and the steps executed individually. The step parameters will be replaced by the corresponding iteration values. The steps affect the iteration status that, in turn, affects the overall test run status.
The Xray Connector app for Bamboo will be updated in order to support new functionality and APIs already available in Xray. The Xray Connector will also support Xray cloud APIs and connectivity.
Expand
title
Modular Tests
Panel
borderColor
#EAECEE
borderWidth
1px
borderStyle
solid
Ability to call an existing Test from the step of another Test and thus use tests as reusable building blocks for the composition of more complex testing scenarios.
The called test can be executed independently or be used as a step of a broader test (e.g. "login as the admin user").
Future Versions
Expand
title
Test ParameterizationUpdated Xray connector app for Bamboo
Panel
borderColor
#EAECEE
borderWidth
1px
borderStyle
solid
Ability to define variables in the Test specification (e.g. the steps) that can be replaced by values inherited from different sources (a calling test for example). In that scenario, the called test is used as a parameterized template (e.g. "login as the specified user").
Expand
The Xray Connector app for Bamboo will be updated in order to support new functionality and APIs already available in Xray. The Xray Connector will also support Xray cloud APIs and connectivity.
Expand
title
Support Jira Issue Archiving
title
Data-driven testing
Panel
borderColor
#EAECEE
borderWidth
1px
borderStyle
solid
Ability to perform data-driven testing (i.e. execute the same test against multiple parameters, for different combinations of data).
Data-driven testing will be implemented using the foundations of test parameterization.
Data may be defined at multiple levels including:
the test level (default values)
the caller test
the Test Plan
the Test Execution
Data might also be imported from external sources (like CSV files).
Future Versions
Jira Datacenter supports archiving of issues and projects since v8.1. By archiving issues, Jira will remove them from the indexes. The archived issues will still be accessible in read-only mode, however, they will not appear in Jira searches. This enhances Jira performance because Jira stores less data.
The goal of this feature is to support issue archiving so that Xray issues can be archived without some of the undesirable side effects (described here). A good use case for archiving Xray issues is to archive old Test Executions, especially if continuous integration processes are used. These can quickly create many Test Execution issues while importing new execution results into Xray on a daily basis. These issues are good candidates to be archived.
Expand
title
Test Environment variables
Expand
title
Support Jira Issue Archiving
Panel
borderColor
#EAECEE
borderWidth
1px
borderStyle
solid
This feature will allow test engineers to create or generate environments with variables such as Browser, Operating System, Database, etc. These environments (or configurations) can be managed and assigned to Test Plans and Test Plan folders. Test Executions can then be created automatically for each environment/configuration.
Xray will provide a report to track the progress of Test Runs by environment in the context of a specific Test Plan
Currently, the Test Plan is composed of a static list of Test cases. This means you must explicitly add the Tests to the Test Plan. If the Tests are all known and well defined when you start your Test Plan, this is ok. However, if you are working in an agile context where a Test Plan is created for a specific sprint, Tests will only be specified during the sprint and later added to the Test Plan. This process is not ideal because users might forget to add the Tests to the Test Plan.
Dynamic Test Plans can be defined with a JQL query which will be the source for Test cases. Considering the use case described above, we can define a JQL query that will get all Tests covering requirements of a specific sprint.
Expand
title
Dynamic Test Plans
Panel
borderColor
#EAECEE
borderWidth
1px
borderStyle
solid
Currently, the Test Plan is composed of a static list of Test cases. This means you must explicitly add the Tests to the Test Plan. If the Tests are all known and well defined when you start your Test Plan, this is ok. However, if you are working in an agile context where a Test Plan is created for a specific sprint, Tests will only be specified during the sprint and later added to the Test Plan. This process is not ideal because users might forget to add the Tests to the Test Plan.
Dynamic Test Plans can be defined with a JQL query which will be the source for Test cases. Considering the use case described above, we can define a JQL query that will get all Tests covering requirements of a specific sprint.
