Page History
...
This report lists some details of the selected Test Executions Defects in Xray, enabling them to be extracted in an Excel format. With this ability to extract the report you can use it for analysis of trends and current testing status, or process this information to generate some metrics, for example, or even share it with someone else who still needs access to Jira.
Possible usage scenarios:
- see all the Test Run Defect statuses & count for a given Test Executionversion
- see the linked Test Plans Requirements, Tests, and Defects
- track the Test Executions Defects that are taking the most time to get resolved
Output Example(s)
The following table shows template contains 4 tabs, the following tables show an example of the columns/rows you should expect.
How to use
This report can be generated from different places/contexts, including:
...
| Info | ||
|---|---|---|
| ||
| General information about all the existing places available to export from and how to perform it is available on the Exporting page. |
Source data
This report is applicable to:
In the store, you can find 2 subtypes for this template:
- "Bugsormix" - 1 or more Bug issue as well as a mix of Bug and 1 or more Test Execution issues
- "Testexecs" - 1 or more Test Execution issues
Output format
The standard output format is .XLSX so you can open it in Microsoft Excel, Google Sheets, and other tools compatible with this format. From those tools, you can generate a .CSV file.
...
- Issue types having the name: "Test Execution".", "Bug"
- Bug final status being "Done" or "Resolved"
- Finished Bugs having the "Resolved date" populated
If any of these assumptions is not met, you need to update the template or the environment accordingly.
...
| Code Block | ||
|---|---|---|
| ||
project = "BOOK" and issuetype="Test Execution" order BY created DESC |
Export
...
Bugs associated with a given fix version
- from the Issue Navigator/Search, search by the release (i.e., "fixVersion") of your project (e.g., "EWB") and then use bulk export or Export->Xporter
| Code Block | ||
|---|---|---|
| ||
project = "BOOK" and issuetype= "Test ExecutionBug" and fixVersion=1.2 order BY created DESC |
Export Test Executions and Bugs obtained
...
in a given
...
Environment
from the Issue Navigator/Search, search by Test Executions or Bugs assigned to that Test Environment (e.g., "chrome") and then use bulk export or Export->Xporter
...
| Code Block | ||
|---|---|---|
| ||
project = "BOOK" and issuetype= in ("Test Execution", Bug) and testEnvironmentsenvironment =IS chrome order BY created DESC |
Export Bug or Test Execution from its detailed view
- open the Bug/Test Execution issue and export it using this template
...
The report shows information about the Test Executions Defects in a list form as well as provides summary tables.
Layout
The report is composed of one sheet with the information on the "Test Executions".
By default, and to avoid overload/redundancy of information, only the "Test Executions" are rendered; all the other types will not be parsed.
"Test Executions" sheet
...
4 sheets - "ReqsImpactSummary", "DefectSummary", "DetailedDefectList", and "Input". By default, all tabs are rendered. The layout is the same regardless of the template subtype used.
"ReqsImpactSummary" sheet
This sheet presents a table listing requirements ("Story" issue type) impacted by defects from your exported issues.
| Column | Notes |
|---|---|
Story Key | Issue key of the Story |
| Summary | Summary of the Story |
| Priority | Priority of the Story |
| Component | Component of the Story |
| Total linked defects | Count of unique defect issues linked to a given requirement story |
"DefectSummary" sheet
| Column | Notes |
|---|---|
Total defects reported | Count of unique defects associated with all the items from the source JQL. Should match the number of content rows on the "DetailedDefectList" sheet. |
| Total defects resolved | Count of unique defects in "Done" or "Resolved" status |
| Total defects unresolved | Difference between the first 2 columns |
| Total found outside of testing | Count of defects from "DetailedDefectList" sheet that have "found outside of testing" value in the Source column (see the description of logic below). |
| Total found in testing | Difference between "Total defects reported" and "Total found outside of testing" |
"DetailedDefectList" sheet
| Column | Notes |
|---|---|
Key | Issue key of the Test Executiondefect |
| Summary | Summary of the defect |
Workflow Status | Workflow status of the defect |
| Affected Version | Affected Version(s) of the Test Executiondefect |
| Fix versionsversion | Fix versionVersion(s) defined in of the Test Executiondefect |
| Priority | Priority of the defect |
| Component | Component(s) of the defect |
| Environment | Environment of the defect |
| Reporter | Reporter of the defect |
| Assignee | Assignee of the defect |
| Linked Test Execution | Key for the Test Execution(s) linked to the defect (regardless of the link type) |
| Linked Test Key and Associated Latest Test Run Status | Key for the Test(s) linked to the defect (regardless of the link type) and the latest execution run status for the Test(s) |
| Linked Requirement Story Key and Summary | Key and Summary for the Story(ies) linked to the defect (regardless of the link type) |
| Linked Requirement Epic Key and Summary | Key and Summary for the Epic(s) linked to the defect (regardless of the link type) |
| Source | The value is "found in testing" if there is at least 1 linked Test, Test Execution, or Test Plan. The value is "found outside of testing" otherwise (e.g. if a bug is created directly from the story). |
| Date Created | "Created" date in the "dd-MM-yyyy HH:mm:ss" format |
| Time to fix | Difference between "Resolved" and "Created" dates |
"Input" sheet
| Column | Notes |
|---|---|
Key | Issue key of the item from the source JQL |
| Issue Type | Issue Type of the item from the source JQL |
| Summary | Summary of the item from the source JQL |
| Priority | Priority of the item from the source JQL |
| Component | Component of the item from the source JQL |
| Status | Workflow Status of the item from the source JQL |
| Assignee | Assignee of the item from the source JQL |
| Revision | Revision defined in the Test Execution |
| Begin Date | Timestamp of when the Test Execution started, in the format "dd-MM-yyyy hh:mm:ss" |
| End Date | Timestamp of when the Test Execution ended, in the format "dd-MM-yyyy hh:mm:ss" |
| Test Environment | Test Environment(s) defined in the Test Execution |
| Test Plan | Test Plan(s) linked to the Test Execution |
| Defects | Defects linked to the Test Execution (at either the test run or the test step level) |
| Elapsed Time | Sum of elapsed time of all associated test runs in HH:MM:SS format. Please note that it will output "00:00:00" when there are no tests associated with the execution (or your executions are really fast) and " " when there are tests in progress (i.e. "Executing" or "To Do" status). |
| #Test Runs | Number of test runs that are part of the Test Execution. |
| Passed | Number of runs in the passed status. |
| Passed (%) | Percentage of runs in the passed status. |
| Failed | Number of runs in the failed status. |
| Failed (%) | Percentage of runs in the failed status. |
| Executing | Number of runs in the executing status. |
| Executing (%) | Percentage of runs in the executing status. |
| To do | Number of runs in the to do status. |
| To do (%) | Percentage of runs in the to do status. |
| Aborted | Number of runs in the aborted status. |
| Aborted (%) | Percentage of runs in the aborted status. |
Customizing the report
The common customization actions are:
- adding/removing columns
- changing the level of detail in the columns
Exercise 1: add a field from the related Test Execution issue
Let's say we have a "Description" field on the Test Execution issue that we want to show on the report.
We can copy the column "Summary" and adapt it.
...
...
As this report is column-based, if some columns are not relevant to you, you should be able to delete them. Make sure that no temporary variables are created in the cells of those columns that are used in other subsequent columns.
Exercise 2: add more details to the Test Plan column
Let's say you want to see not only the Test Plan Key, but also the Test Plan Summary, and make it a hyperlink while you are at it.
- Navigate to the "Test Plan" column
- Edit the statement from ${Test Plan} to ${link:title=${Test Plan},href=${BaseURL}/browse/${Test Plan}}
You can apply similar changes to the Defects column by editing the following statements:
- ${TestRuns[n].ExecutionDefects[d].Key} (line 6)
- ${TestRuns[n].Iterations[it].TestSteps[r].Defects[dc].Key} (line 22)
...
Keep in mind that only 1 hyperlink will be active per Excel cell (the one associated with the first line item).
You can further finetune the content and formatting via JavaScript,you can find more useful snippets in this tutorial for Xporter and DocGen as well as Xporter documentation (Cloud, DC).
Exercise 1: extend the report to custom issue types
In the "bugsormix" subtype, find all instances of
#{if (%{'${IssueTypeName}'.equals('Bug')})}
and edit them to include more than 1 type with the OR (||) syntax like this
#{if (%{'${IssueTypeName}'.equals('Bug') || '${IssueTypeName}'.equals('Defect') || '${IssueTypeName}'.equals('Incident')})}
Similar changes would allow you to account for more statuses
Keep in mind that only 1 hyperlink will be active per Excel cell (the one associated with the first line item). Also, ${Test Plan} value is treated as a single list, so if you have multiple test plans linked to the same execution and you want to separate the urls, you will need to further customize the code from the example above. On a related note, as you probably noticed, Test Plan and Test Environment are displayed slightly differently from Defects by default - comma separation, square brackets (the DocGen version has the brackets for empty Test Plan cells, doesn't have them for non-empty Test Environments).
You can further finetune the content and formatting via JavaScript,you can find more useful snippets in this tutorial for Xporter and DocGen.
Performance
Performance can be impacted by the information that is rendered and by how that information is collected/processed (especially in the "bugsormix" subtype).
The number of Test Executions and Test RunsDefects and associated items, depending on scenarios, can be considerably high, especially with CI/CD. As this report sums up quite a lot of information, please use it wisely.
...