Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Status
colourGreen
titlev5.0


Expand
titleParameterized Tests and data driven testing
Panel

Test parameterization is a powerful practice that allows the same test to be executed multiple times with different parameters. Parameters are similar to input values (variables) that can change with each execution.

Parameterized tests in Xray are defined just like any other test with the addition of some parameter names within the specification using the following notation: ${PARAMETER_NAME}. This notation is used to reference parameters within the test steps.

Precondition issues can also be parameterized by including parameter names in the precondition specification.

The parameters, along with their values, are defined within a dataset. A dataset is a collection of data represented with a tabular view where every column of the table represents a particular variable (or parameter), and each row corresponds to a given record (or iteration) of the dataset. The number of rows in the dataset determines the number of iterations to execute.

A dataset can be defined in the following entities/scopes:

  1. Test (default dataset)
  2. Test Plan - Test
  3. Test Execution - Test (Test Run)

The closest dataset to the test run will be the one used to generate the iterations, effectively overriding any dataset defined in higher levels.


Image Added


All iterations for a given test are executed within the context of the same test run. Each iteration can be expanded, and the steps executed individually. The step parameters will be replaced by the corresponding iteration values. The steps affect the iteration status that, in turn, affects the overall test run status.

Image Added


Learn more about parameterized tests here.


Status
colourGreen
titlev6.0


Expand
titleModular Tests
Panel
borderColor#EAECEE
borderWidth1px
borderStylesolid

Modular test design is a way of promoting test case reusability and composition across a large test repository. To design modular tests, you can create a test where some of the steps call steps call or include other test cases. This prevents testers from having to write the same steps over and over again in different high-level tests. Using a modular design approach, any test can become a building block of more extensive test scenarios. Still, they can also be executed individually if needed.

A called test can, in turn, also call other tests. You can compose a test scenario with up to five levels of depth.

Modular tests can also be parameterized. When calling a test, you can provide new parameter values according to the parent test data.

Upon executing, Xray will unfold all called test steps in the test run. This becomes transparent to testers as they only have to follow and execute the steps on the execution, even though test steps might come from different tests issues.

A common use case for modular tests is end-to-end testing. End-to-end tests often need to pass through the same area or component of the application before asserting the final result. With modular test design, you can reuse the tests for these common areas or components.



Learn more about modular tests here.

...

Expand
titleSupport Jira Issue Archiving
Panel
borderColor#EAECEE
borderWidth1px
borderStylesolid

Jira Datacenter supports archiving of issues and projects since v8.1. By archiving issues, Jira will remove them from the indexes. The archived issues will still be accessible in read-only mode, however, they will not appear in Jira searches. This enhances Jira performance because Jira stores less data.

The goal of this feature is to support issue archiving so that Xray issues can be archived without some of the undesirable side effects (described here). A good use case for archiving Xray issues is to archive old Test Executions, especially if continuous integration processes are used. These can quickly create many Test Execution issues while importing new execution results into Xray on a daily basis. These issues are good candidates to be archived.

Expand
titleCross-project reports
Panel
borderColor#EAECEE
borderWidth1px
borderStylesolid

Most of Xray's native reports are tied to the project level. We plan to add multi-project support for these reports by allowing a saved filer with issues from multiple projects to be used as the report data source.

Expand
titleSupport Jira Issue Archiving
Panel
borderColor#EAECEE
borderWidth1px
borderStylesolid

Jira Datacenter supports archiving of issues and projects since v8.1. By archiving issues, Jira will remove them from the indexes. The archived issues will still be accessible in read-only mode, however, they will not appear in Jira searches. This enhances Jira performance because Jira stores less data.

The goal of this feature is to support issue archiving so that Xray issues can be archived without some of the undesirable side effects (described here). A good use case for archiving Xray issues is to archive old Test Executions, especially if continuous integration processes are used. These can quickly create many Test Execution issues while importing new execution results into Xray on a daily basis. These issues are good candidates to be archived.

Future Versions

In the works


Expand
titleTest naturesCase Versioning
Panel
borderColor#EAECEE
borderWidth1px
borderStylesolid

This feature will cover a common scenario where manual Test cases evolve to automated Tests. In this case, the user can either change/select the Test type to see different definitions (refer to the configuration option: Delete test definition when changing Test type) or create separate Test issues to cover all the natures for the Test.

With Test natures, we plan to make it more explicit that multiple Test definitions can coexist within the same Test case. Users will also be able to set the current definition so that Test Runs are always created using the "current" definition.

Expand
titleDynamic Test Plans
Panel
borderColor#EAECEE
borderWidth1px
borderStylesolid

Currently, the Test Plan is composed of a static list of Test cases. This means you must explicitly add the Tests to the Test Plan. If the Tests are all known and well defined when you start your Test Plan, this is ok. However, if you are working in an agile context where a Test Plan is created for a specific sprint, Tests will only be specified during the sprint and later added to the Test Plan. This process is not ideal because users might forget to add the Tests to the Test Plan.

Dynamic Test Plans can be defined with a JQL query which will be the source for Test cases. Considering the use case described above, we can define a JQL query that will get all Tests covering requirements of a specific sprint.



Future Versions


Expand
titleTest Environment variables
Panel
borderColor#EAECEE
borderWidth1px
borderStylesolid

This feature will allow test engineers to create or generate environments with variables such as Browser, Operating System, Database, etc. These environments (or configurations) can be managed and assigned to Test Plans and Test Plan folders. Test Executions can then be created automatically for each environment/configuration.

Xray will provide a report to track the progress of Test Runs by environment in the context of a specific Test Plan.

...