This report lists some details of the selected Test Executions in Xray, enabling them to be extracted in an Excel format. With this ability to extract the report you can use it for analysis of trends and current testing status, or process this information to generate some metrics, for example, or even share it with someone else who still needs access to Jira.
Possible usage scenarios:
The following table shows an example of the columns/rows you should expect.
This report can be generated from different places/contexts, including:
General information about all the existing places available to export from and how to perform it is available on the Exporting page. |
This report is applicable to:
The standard output format is .XLSX so you can open it in Microsoft Excel, Google Sheets, and other tools compatible with this format. From those tools, you can generate a .CSV file.
The template has a set of assumptions that you have to make sure your Jira/Xray environment complies with:
If any of these assumptions is not met, you need to update the template or the environment accordingly.
from the Issue Navigator/Search, search by the issueType (i.e., "Test Execution") from your project (e.g., "BOOK") and then use bulk export or Export→Document Generator
project = "BOOK" and issuetype="Test Execution" order BY created DESC |
project = "BOOK" and issuetype= "Test Execution" and fixVersion=1.2 order BY created DESC |
from the Issue Navigator/Search, search by Test Executions assigned to that Test Environment (e.g., "chrome") and then use bulk export or Export->Document Generator
project = "BOOK" and issuetype= "Test Execution" and testEnvironments = chrome order BY created DESC |
The report shows information about the Test Executions in a list form.
The report is composed of one sheet with the information on the "Test Executions".
By default, and to avoid overload/redundancy of information, only the "Test Executions" are rendered; all the other types will not be parsed.
This sheet will present a line per each Test Execution.
Column | Notes |
---|---|
Key | Issue key of the Test Execution |
Summary | Summary of the Test Execution |
Fix versions | Fix version(s) defined in the Test Execution |
Revision | Revision defined in the Test Execution |
Begin Date | Timestamp of when the Test Execution started, in the format "dd-MM-yyyy hh:mm:ss" |
End Date | Timestamp of when the Test Execution ended, in the format "dd-MM-yyyy hh:mm:ss" |
Test Environment | Test Environment(s) defined in the Test Execution |
Test Plan | Test Plan(s) linked to the Test Execution |
Defects | Defects linked to the Test Execution (at either the test run or the test step level) |
Elapsed Time | Sum of elapsed time of all associated test runs in HH:MM:SS format. Please note that it will output "00:00:00" when there are no tests associated with the execution (or your executions are really fast) and " " when there are tests in progress (i.e. "Executing" or "To Do" status). |
#Test Runs | Number of test runs that are part of the Test Execution. |
Passed | Number of runs in the passed status. |
Passed (%) | Percentage of runs in the passed status. |
Failed | Number of runs in the failed status. |
Failed (%) | Percentage of runs in the failed status. |
Executing | Number of runs in the executing status. |
Executing (%) | Percentage of runs in the executing status. |
To do | Number of runs in the to do status. |
To do (%) | Percentage of runs in the to do status. |
Aborted | Number of runs in the aborted status. |
Aborted (%) | Percentage of runs in the aborted status. |
The common customization actions are:
Let's say we have a "Description" field on the Test Execution issue that we want to show on the report.
We can copy the column "Summary" and adapt it.
As this report is column-based, if some columns are not relevant to you, you should be able to delete them. Make sure that no temporary variables are created in the cells of those columns that are used in other subsequent columns.
Let's say you want to see not only the Test Plan Key, but also the Test Plan Summary, and make it a hyperlink while you are at it.
You can apply similar changes to the Defects column by editing the following statements:
with the syntax like (the example is for line 6):
@{title=${TestRuns[n].ExecutionDefects[d].Key}|href=${BaseURL}/browse/${TestRuns[n].ExecutionDefects[d].Key}} | ${TestRuns[n].ExecutionDefects[d].Summary} | ${TestRuns[n].ExecutionDefects[d].Priority}
Keep in mind that only 1 hyperlink will be active per Excel cell (the one associated with the first line item). Also, ${Test Plan} value is treated as a single list, so if you have multiple test plans linked to the same execution and you want to separate the urls, you will need to further customize the code from the example above. On a related note, as you probably noticed, Test Plan and Test Environment are displayed slightly differently from Defects by default - comma separation, square brackets (the DocGen version has the brackets for empty Test Plan cells, doesn't have them for non-empty Test Environments).
You can further finetune the content and formatting via JavaScript, you can find more useful snippets in this tutorial for Document Generator.
Performance can be impacted by the information that is rendered and by how that information is collected/processed.
The number of Test Executions and Test Runs, depending on scenarios, can be considerably high, especially with CI/CD. As this report sums up quite a lot of information, please use it wisely.