Within a Requirement issue, a web panel named "Test Coverage" is provided by Xray. This panel allows users to cover an issue with Tests (or Test Sets) and also analyze the status of the requirement in terms of testing based on the latest execution results for different scopes.

This web panel will only be available if the issue type is mapped as a Requirement issue type in Xray global configuration. The project must also be enabled for Requirement Coverage.

In this panel, you can perform actions such as creating Tests automatically linked to the requirement and/or link existing Tests and Test Sets to the requirement. Sub-Test Execution issues can also be created directly from this location.

A data table is used to manage all the Tests and Test Stets that are associated with the issue. The data table can be filtered, sorted and configured by setting the visible columns. Using the data table is also possible to remove Tests (or Test Sets) individually or via bulk action.


What about defects?

Defect issues can also be covered by Test cases just like Requirements. In this case, the defect issue types (e.g. Bug) must also be mapped as a Requirements issue types in Xray global configuration.



How to cover a Requirement


In order to cover a requirement issue with Test cases, you need to create Jira issue links between the requirement and Test or Test Set issues. It is also possible to create new Tests directly from the Requirement issue.

Create Tests

This action allows you to access the create issue dialog, with some preset field values such as "Test" issue type and link to the requirement issue.

Associate Tests or Test Sets (using the Xray issue picker)

This option allows you to create Jira issue links between the Requirement and Test or Test Set issues thus covering the requirement with Test cases.

This operation can be performed using the Issue Picker Dialog which allows you to search, and select multiple issues to link with the Requirement. JQL searches are also possible. This dialog replaces the native Jira issue link dialog making it easy to associate Tests (or Test Sets) with requirement issues.


Associate Tests or Test Sets (using plain Jira issue Links)

Because the association between Requirement issues and Test (or Test Set) issues is done using Jira issue Links, you can just use the Jira issue linking actions as well.


Issue Link Types

Xray uses specific Jira issue link types for associating Requirements and Tests (or Test Sets). These issue link types are installed by Xray automatically. They are:

Testsoutbound: testsinbound: tested by
Defectoutbound: createdinbound: created by

These issue link types can be used explicitly by users to associate Requirements with Tests.

The Defect issue link type is used by Xray when defects are created upon executing tests. In this case, we have relations such as Defect1 "created by" Test1.

Whenever creating Tests from Requirements or Defects, Xray will always default to the "Tests" issue link type.

From the Requirement issue view screen

Step 1: Open the Requirement you wish to associate a Test with. In the given example, New Feature issue types are mapped as Requirements.

Step 2: Select More > Link. The Link screen will appear.

Step 3: On the This issue field, select the tested by option.

Step 4: In order to select the Tests or Test Sets to be associated, you can:

- Input the desired Test/Test Set Issue Key on the Issue field;
- Click on the Down Arrow on the Issue field and select the Test from its History Search list;
- Click on the search for an issue link to use the Find Jira issues browser;

Step 5: Click Link.

From Defect issues


From the Test issue view screen

Step 1: Open the Test you wish to associate a Requirement with.

Step 2: Select More > Link. The Link screen will appear.

Step 3: On the "This issue" field, select the "tests" issue link type.

Step 4: In order to select the Requirements to be associated, you can:

- Input the desired Requirement Issue Key on the Issue field;
- Click on the Down Arrow on the Issue field and select the Requirement from its History Search list;
- Click on the search for an issue link to use the Find Jira issues browser;

Step 5: Click Link.

From Defect issues


Tests data table

Once you have already Tests or Test Sets covering a requirement issue, you'll be able to see a data table directly below the Test Coverage analysis scope section. 

This data table contains all the Tests that are associated with the Requirement. Test Sets are automatically expanded. This means that all the Tests from Test Sets associated with the Requirement will be also displayed on the table direclty.

The rightmost column on this data table is the status column that displays the latest execution status of each Test, according to the analysis scope selected. 

The data table features the following topics.

Filtering

The tests presented as a result of the selected analysis scope can be filtered using a dynamic filter component.

The dynamic filter component has the following operating modes:

Basic 

The Basic mode has a set of default fields but you can also search and select other fields from the More option.

Advanced

The Advanced mode allows you to search based on a specific JQL.

Summary view

The summary view presents the field names and values applied in the active filter. When these exceed the available space "..." are added and all information is visible in the tooltip.


Sorting

The list can be sorted by the columns you can add or remove to and from the list using the Columns dropdown menu.

 

Sort preferences

The list sort criteria is kept in user preferences.

Configuring Columns

Columns can be quickly added to the list using the columns search, this allows to locate and select the desired field(s) in a practical manner.Columns can also be quickly removed from the list using the Restore Defaults link or by unselecting the undesired field(s).


Restore Defaults

The changes made to the selected columns are kept in user preferences if you wish to change the defaults those are set in Xray Administration > Default Column Layouts under the Test Coverage Columns field.


A linked test can be removed directly in the test coverage list using the following operations:

Inline remove

A confirmation message is always presented. When the selected test was added via a Test Set link, it cannot be removed separately, so by confirming the operation, the test set link to the requirement as well as the links to all the tests it includes will be removed.


Bulk selection for removal

A confirmation message is always presented. When within the selected tests, are tests that were added via a Test Set link(s), they cannot be removed separately, so by confirming the operation, the test set link(s) to the requirement as well as the links to all the tests they include will be removed. 

Analyzing Test Coverage

Xray makes it very easy to analyze the status of a Requirement in terms of testing. The status of a Requirement is clearly visible on the Test Coverage web panel.

The coverage status can be calculated for different analysis scopes.

The test coverage analysis scope is based on:

In combination with all or a specific Environment.

Once the analysis scope is changed, the status is automatically recalculated for the new scope. The status of the Tests within the data table will also be affected by the selected analysis scope. 

Analysis Scope preferences

The user analysis scope preferences are kept, per project and based on the last configuration.

Version

When the analysis scope is set per version it allows you to see the Requirement status for each project´s version. 


Test Plan

When the analysis scope is set per Test Plan it allows you to see the Requirement status for the selected Test Plan.


Environments

The statuses presented in the context of an analysis scope also depend on the Environments.

Therefore you can see both the requirement status and the test statuses based on the combination of scope and test environment you have selected.

Test Runs

The Test Runs column will present the link to the Test Run affecting a particular status of a Test. 

If only a given Test Run is responsible for the result (no environments are involved), then the link should take the user directly to the Test Run page otherwise it will present a dialog listing all the Test Runs affecting the status.

The user can then click on the Test Run to go to the execution page.


Coverage Analysis

For more details, please check the Coverage Analysis page. Here you can find all the areas where you can take advantage of the coverage analysis such as the entities (Requirements, Tests, Test Sets) and also reports.

Also, make sure to check the Understanding coverage and the calculation of Test and requirement statuses TTT article if you need to learn how is the Coverage status calculated in more detail.

Creating Test Execution / Sub-Test Executions

This action allows you to create a Test Execution (1) or a Sub-Test Execution (2) with some preset field values such as the links to all the tests currently covering the requirement issue. You can also create a Test Execution/Sub-Test Execution with a subgroup of the tests based on their status (3).

This action is useful if you create one Test execution per Requirement issue. Working in an agile context, this is a usual pattern. If you are using Scrum or Kanban boards in Jira, creating Test Executions / Sub-Test Executions allows the Tests to be just another work item for a given Requirement.