Purpose
This report enables you to extract the detailed specification of one or more Tests, so that you can generate a document that contains a snapshot of the latest specification. This can be used for revision/approval, or even to share it with someone else that hasn't access to Jira and wishes to pursue testing.
Possible usage scenarios:
- see the detailed specification of the Test(s), including the test steps of manual scripted test cases and the Gherkin of BDD test scenarios
- see all Preconditions required by the Test(s)
- see all the requirements covered by the given Test(s)
- see all Test Sets where the Test(s) are included in
- see all Test Plans where the Test(s) are included in
Output Example(s)
The following screenshots shows an example of the sections you should expect in this report.
How to use
This report can be generated from the Issue details screen, or from the Issue Search/navigator page.
Learn more
Source data
This report is applicable to:
- 1 or more Test issues
Output format
The standard output format is .DOCX, so you can open it in Microsoft Word, Google Docs, and other tools compatible with this format. You can also export it to .PDF.
Report assumptions
The template has a set of assumptions that you make sure that your Jira/Xray environment complies with:
- Issue types having these names
- "Test", "Test Set", "Test Plan"
- Tests use the "tests" issue link type to cover issues (e.g., Story)
If any of these assumptions is not met, you need to update the template accordingly.
Usage examples
Export the test specification of all Tests having the "Login" component, or by other criteria based on the Test issue fields
- open the Issue Search/navigator (go to Filters > Advanced issue search)
- search by component "Account", or by other criteria based on the Test issue's fields
-
- Select the "Xporter" action and export it using this template
Export the test specification for all Tests contained in a Test Repository folder
- select a folder in the Test Repository
- Select the "View Issues" action; you'll be redirected to the Issue Search/Navigator
- The generated query will include all Tests in all sub-folders; to just include the Tests on the selected folder, change the 'true' to 'false' on the last argument of the TestRepositoryFolder() JQL function
- On the Issue Search/Navigator, select the "Xporter" action and export it using this template
Export the test specification for all Tests contained in a Test Plan or in one of its folders (including sub-folders)
- select a folder in the Test Plan
- Select the "View Issues" action; you'll be redirected to the Issue Search/Navigator
- The generated query will include all Tests in all sub-folders; to just include the Tests on the selected folder, change the 'true' to 'false' on the last argument of the testPlanFolderTests() JQL function
- On the Issue Search/Navigator, select the "Xporter" action and export it using this template
Understanding the report
The report shows detailed information about the Tests, focused on their specification (e.g., manual steps, Gherkin, test charter).
Layout
The report is composed by several sections. Two major sections are available: Introduction and Test Executions details.
By default, and to avoid overload/redundancy of information, only the "Introduction" section will be rendered; you can change this behavior on the template (more info ahead).
"Introduction" section
This section is divided into 3 sub-sections:
- Document Overview
- Total tests
- Requirements covered by these Tests
Each of these sections is explained below.
Document Overview
Brief description of what you will find in this report and how it was generated.
Total tests
In this section, the total tests are grouped by their test type and a count is shown.
| Test Type | Number of Tests |
|---|---|
| Manual | Number of tests that are step/script based; usually these are known by "manual" (even though "manual" is a rough simplification) |
| Gherkin | Number of Gherkin based tests; usually these correspond to Cucumber test scenarios, with Test Type "Cucumber", but may also refer to other Gherkin based frameworks |
| Unstructured | Number of tests that are neither scripted nor Gherkin based; usually these correspond to unstructured tests; this includes the tests having Test Type "Generic", "Exploratory", etc. |
Please note
If you wish to count Tests by custom Test Types, you can do so. See more information ahead on how to customize this report.
The output will have the following information.
Requirements covered by these Tests
In this section we have an overview of all the requirements that are covered by the Tests in this report.
| Field | Description | Code |
|---|---|---|
| Key | Key of the requirement (in this case we are adding it as link) | @{title=${Links[j].Key}|href=${BaseURL}/browse/${Links[j].Key}} |
| Summary | Summary of the requirement | ${Links[j].Summary} |
| Version | Project version assigned to the requirement | ${Links[j].FixVersions} |
| Workflow Status | Workflow Status of the requirement | ${Links[j].Status} |
The requirements are listed in a table as follows.
Tests
This section will gather all the information related to each Test, focused on its specification.
It is composed with several sub-sections that will be filled with information if it is available or be filled with a message showing that no information is available.
Test Summary
This section have a table with information regarding each Test in this report (and will repeat these sections for each Test). The information is presented as a table with the following fields:
| Field | Description | Code |
|---|---|---|
| Test Type | Test Type | ${Test Type} |
| Test Repository Folder | Folder where Test is at in the Test Repository | ${TestRepositoryPath} |
| Label(s) | Labels assigned to the Test issue | ${Labels} |
| Component(s) | Components assigned to the Test issue | ${Components} |
| Priority | Priority assigned to the Test issue | ${Priority} |
The resulting table look like the one below.
Test Description
The description of the Test (${wiki:Description}).
Test Issue Attachments
This section only appears if you have any attachments associated to the Test.
| Key | Description |
|---|---|
| File Name | File Name of the attachment |
| Author | The Author of the attachment |
| File Size | File Size of the attachments in bytes |
This appears in the document in a table form. A thumbnail is shown for images.
Status of the last execution
This section shows the last execution status for the Test.
Requirements
In this section we have an overview of all the requirements that are covered by the Test.
| Field | Description | Code |
|---|---|---|
| Key | Key of the requirement (in this case we are adding it as link) | @{title=${Links[j].Key}|href=${BaseURL}/browse/${Links[j].Key}} |
| Summary | Summary of the requirement | ${Links[j].Summary} |
| Version | Project version assigned to the requirement | ${Links[j].FixVersions} |
| Workflow Status | Workflow Status of the requirement | ${Links[j].Status} |
The requirements are listed in a table as follows.
Preconditions
If the Test requires any Preconditions, they will be shown.
| Key | Description | Code |
|---|---|---|
| Key | Key of the Precondition in the form of a link | @{title=${PreConditions[n].Key}|href=${BaseURL}/browse/${PreConditions[n].Key}} |
| Summary | Summary of the Precondition | ${PreConditions[n].Summary} |
| Condition | Condition | ${wiki:PreConditions[n].Conditions} |
The Preconditions are listed in a table as follows.
Test Details
This section shows the Test details; for that we are considering the different possible Test Types we can have in Xray. For each type we will fetch different information.
| Type | Key | Description | Code | Output |
|---|---|---|---|---|
Unstructured (e.g., "Generic") | Test Type | Test Type field | ${Test Type} |
|
| Specification | Definition of the Generic test | ${Generic Test Definition} | ||
Gherkin (e.g., "Cucumber") | Test Type | Test Type field | ${Test Type} |
|
| Gherkin Specification | Gherkin specification of the Test | ${Cucumber Scenario} | ||
Structured (e.g., "Manual") | (Step) | Step Number | ${TestSteps[m].StepNumber} |
|
| Action | Action of the Test Step | ${TestSteps[m].Action} | ||
| Step Custom Fields | Step custom field(s) | ${TestSteps[m].<step_custom_field>} Note: Test Step Custom Fields are not rendered by default; they need to be specified explicitly | ||
| Data | Data of the Test Step | ${TestSteps[m].Data} | ||
| Expected Result | Expected Result of the Test Step | ${TestSteps[m].ExpectedResult} | ||
| Attachments | Attachment of the Test Step; images will be rendered | @{title=${TestSteps[m].Attachments[sa].Name}|href=${TestSteps[m].Attachments[sa].FileURL}} |
Test Sets
If the Test is part of Test Sets, they will be shown.
| Key | Description | Code |
|---|---|---|
| Key | Key of the Test Set in the form of a link | @{title=${JQLIssues[y].Key}|href=${BaseURL}/browse/${JQLIssues[y].Key}} |
| Summary | Summary of the Test Set | ${JQLIssues[y].Summary} |
| Version | Project version assigned to the Test Set | ${JQLIssues[y].FixVersions} |
The Test Sets are listed in a table as follows.
Test Plans
If the Test is part of Test Plans, they will be shown.
| Key | Description | Code |
|---|---|---|
| Key | Key of the Test Plan in the form of a link | @{title=${JQLIssues[x].Key}|href=${BaseURL}/browse/${JQLIssues[x].Key}} |
| Summary | Summary of the Test Plan | ${JQLIssues[x].Summary} |
| Version | Project version assigned to the Test Plan | ${JQLIssues[x].FixVersions} |
The Test Plans are listed in a table as follows.
Appendix A: Approval
This section is added for the cases where you need to have a signature validating the document.
Customizing the report
Adding or removing information to/from the report
To add additional information, usually we're thinking of adding:
- fields of the Preconditions, "requirements", Test Sets, or of the Test Plans associated with the Test
Exercise: add a field from the related "requirement" issue
Let's say we have a "Severity" field on the "requirement" (i.e., covered issue) that is linked to the Test, and that we want to show it on the report.
We can copy the column "Summary" from the "Requirements" section and adapt it.
- on the "Requirements" section,
- copy "Summary" (i.e., insert a column next to it and copy the values from the existing "Summary" column)
- change the contents to
- ${JQLIssues[z].Summary}
Exercise: show a Test step custom field on the Test details
Let's say we have a "StepField1" test step custom filed configured for our project and that it might be used in some test steps of some Tests.
- Go to the Test Details section
- Add the "Step Custom Fields" column to the table
- Add the name of the test step custom field along with the value in the content cell
- StepField1: ${wiki:TestSteps[n].StepField1}
Keep in mind that you cannot easily iterate through all such custom fields with 1 statement, you need to target each one individually by name.
Performance
Performance can be impacted by the information that is rendered and by how that information is collected/processed.
Some tips
- Remove rendering of images for Test Issue Attachments and for Test Step's attachments
- limit the number of input issues
Known limitations
- Modular tests are not fully supported: you will see a note that a test was called, but not its key/steps/etc.
- Data-driven tests are not supported.