The Test Runs Report

Purpose

This report enables you to see which tests were or are going to be executed, along with the corresponding result and key metrics.

Possible usage scenarios:

  • see all the runs of a given Test Execution
  • see all the runs of all Test Executions related with a Test Plan
  • see the number of opened/closed linked defects, in the context of a given run
  • see the elapsed time since the execution has started
  • see the executions performed by or assigned to a given user

How to Use

This report is accessible from either the Xray Reports icon on the project/global left sidebar or from the standard Reports icon, which includes other kinds of reports besides Xray.

Source Test Runs objects can be directly provided using a Test Execution or Sub-Test Execution-based saved filter or by specifying some Test and Test Execution fields.


Test Execution Filters

  • Project: the Test Executions project
  • Fix Version: the Test Execution Fix Version
  • Revision: the Test Execution Revision
  • Test Environments: the Test Environments of the Test Execution
  • Test Plan: the Test Plan from the Test Execution (when the Test Execution is created within the context of a Test Plan).
  • Contains: Test Execution Key or Test Execution Summary sub-string.

Test Filters

  • Test Priority: the Test Priority
  • Test Components: Components field of the Test
  • Contains: Test Key or Test Summary sub-string.

Test Run Filters

  • Assignee: the user assigned to the Test Run
  • Executed By: the user who executed the Test Run
  • Status: the Status of the TestRun

The test run finished date can also be filtered either by using a timeframe or in between fixed dates.

Defining Scope

Source Test Runs objects can be directly provided using a Test Execution or Sub-Test Execution-based saved filter (within section A) or by specifying some Test and Test Execution fields.

  • Fix Version: the Test Execution Fix Version
  • Revision: the Test Execution Revision
  • Test Environments: the Test Environments of the Test Execution
  • Test Plan: the Test Plan from the Test Execution (when the Test Execution is created within the context of a Test Plan).
  • Test Priority: the Test Priority
  • Test Components: Components field of the Test
  • TestRun Assignee: the user assigned to the Test Run
  • Executed By: the user who executed the Test Run
  • TestRun Status: the Status of the TestRun
  • Issue Key or Summary: Test Execution Key or Test Execution Summary sub-string.


Only Test Executions from the current project are taken into account.

For performance reasons, Xray limits the number of Test Runs shown to 10,000. If the number of filtered Test Runs is greater than this, then the user needs to review the applied filter in order to restrict it further.

Understanding the report columns 

An example of the report can be seen below.


Understanding each of the elements.

Metric

Notes

TestKey of the Test associated with the TestRun
Test SummarySummary of the Test associated with the Test Run
ComponentsComponents of the Test associated with the Test Run
Test PriorityPriority of the Test associated with the Test Run
Test ExecutionKey of the Test Execution associated with the Test Run
Test EnvironmentsTest Environments in which the Test has been executed
Test PlanIf the Test Execution has been created in the context of a Test Plan, this column will display the Test Plan key
Test TypeType of the Test Associated with the Test Run
TestRun StatusThe status (i.e., result) of the TestRun
Executed ByUser who performed the execution
Fix VersionFix Version associated with the Test Run
RevisionRevision of the Test Execution associated with the Test Run
Started AtDate of the TestRun start
Finished AtDate of the TestRun end
Elapsed TimeElapsed Time since the start of the TestRun. If the Status is not final, then the elapsed time will be the time between the start date and the current time.
Linked Defects

Defects directly associated with the Test Runs. Will count as unique Defects (I.e., if the same Defect issue is added multiple times in the same Test Run, it will count as one).

Defects with Resolution will be counted as Closed; all others will be counted as Open

Custom FieldsTest Run Custom Fields' values of the respective Test Run



Export and Share

The report can be exported to a CSV file, which will include all report rows (and not just the visible ones).

Click on Export and select To csv.