The Test Execution Report 

Purpose

This report enables you to evaluate some relevant metrics of Test Executions and compare them between different cycles.

Possible usage scenarios:

  • analyze both the progress of the Test Execution and the success rate (i.e., the % of Tests contributing to the requirement's OK status)
  • see the number of manual Tests vs. others in the Test Execution
  • see the overall execution status (i.e., the current status of the Test Runs)
  • see the number of opened/closed linked defects, in the context of the Test Execution

How to Use

This report is accessible either from the Xray Reports icon on the project left sidebar or from the standard Reports icon, which includes other kinds of reports besides Xray. 

At the top of the report you'll find two areas related with the report and with the data shown in the report.

  • A: Filter, for selecting the source data
  • B: visualization information and options

Defining Scope

Source Test Execution issues can be directly provided (within section A) using a saved filter or by specifying some Test Execution-related fields:

  • Saved filter: a saved filter containing Test Execution issues

  • Fix Version: the version assigned to the Test Plan
  • Assignee: the Test Execution issue assignee
  • Status: the Test Execution issue Status
  • Resolution: the Test Execution issue Resolution
  • Test Plan: the assigned Test Plan
  • Environment: the assigned Test Environment

Metrics

MetricNotes

total tests

Number of Tests in the Test Execution; it also corresponds to the number of test runs

number of tests by status

Number of Tests/Test Runs, per Test Run status

number of tests by test type

Number of Test Execution's Tests, per Test Type

progress

Overall progress; % of Test Runs with a final status

success rate

% of "successful" Tests (i.e., whose latest status contributes to the OK requirement status; the PASS Test Run status is an example of such a status)
Elapsed timeSum of the elapsed time of Test Runs that have been or are being executed
linked defects (open/closed)

Defects directly associated with the related Test Runs. Will count as unique Defects (i.e., if the same Defect issue is added multiple times in the same Test Run or os added multiple times within Test Runs of the same Test Execution, it will count as one)

Defects with Resolution will be counted in the Closed column; all others will be counted in the Open column.

Example

The report may be a bit wide for some screen resolutions so you will need to scroll horizontally to see the rest of the information.

Some values provide a link, so you can be redirected to the respective entities.

Export and Share

The report may be a bit wide for some screen resolutions so you will need to scroll horizontally to see the rest of the information.

Some values provide a link, so you can be redirected to the respective entities.


The report can be shared by copying and sending the URL located in the browser's address bar.

When opening the report the Filter will be automatically populated with the values provided in the URL.