Page History
...
The following screenshots shows an example of the sections you should expect in this report.
How to use
This report can be generated from the Issue details screen.
...
In this section we are extracting the Test Plan key in the header and show the Begin and End Date (formatted as demonstrated below), the Summary and the Description present in the Test Plan.
| Field | Description | Sample Code |
|---|---|---|
| Begin Date | Timestamp of the Begin Date field present in the Test Plan (with proper format) | ${dateformat("dd-MM-yyyy HH:mm:ss"):Begin Date} |
| End Date | Timestamp of the End Date field present in the Test Plan (with proper format) | ${dateformat("dd-MM-yyyy HH:mm:ss"):End Date} |
| Summary | Summary of the Test Plan | ${Summary} |
| Description | Description of the Test Plan (as this field accepts wiki markup we will use "wiki:" in the code to be interpreted by the document) | ${wiki:Description} |
The output will have the following information, notice that as the Description field support wiki markup we are using "wiki:" keyword so that it is correctly interpreted.
Requirements covered by the Tests in this Test Plan
...
In the Server version we have a query that fetches the Requirements linked with a Test: testRequirements('${Tests[n].Key}').
| Field | Description | Sample Code |
|---|---|---|
| Key | Key of the requirement (in this case we are adding it as link) | @{title=${JQLIssues[a].Key}|href=${BaseURL}/browse/${JQLIssues[a].Key}} |
| Summary | Summary of the requirement | ${JQLIssues[a].Summary} |
| Workflow Status | Workflow Status of the requirement | ${JQLIssues[a].Status} |
| Coverage Status | Requirement coverage status | ${JQLIssues[a].Requirement Status} |
The requirements are listed in a table with the informations explained above.
Overall Execution Status
...
To obtain this information we are using:
| Field | Description | Sample Code |
|---|---|---|
| TestsCount | The total number of Tests in this Test Plan | ${TestsCount} |
| #Tests | To extract the count of the overall execution status per each <status> (TO DO, EXECUTING, PASS, FAIL, ABORTED) | ${Overall Execution Status.<status>.Count} |
| Percentage | To extract the percentage of the overall execution status per each <status> (TO DO, EXECUTING, PASS, FAIL, ABORTED) | ${Overall Execution Status.<status>.Percentage} |
This will produce the following output:
Defects
In this section we are listing all the defects found that are associated with this Test Plan, we consider defects associated with TestRuns, defects in Test Steps or defects found during the iterations. We do not print duplicates.
| Field | Description | Sample Code |
|---|---|---|
| Key | Key of the Defect |
|
Summary | Summary of the Defect |
|
| Priority | Priority of the Defect |
|
The Defects appear in the document as a table with information regarding the defects found during the executions of the Test Plan.
Tests Summary
In this section we have a table with information regarding the Tests included in this Test Plan. You can find the following information about each Test:
| Field | Description | Sample Code |
|---|---|---|
| Key | Key of the Test in a link form | @{title=${Tests[n].Key}|href=${BaseURL}/browse/${Tests[n].Key}} |
| Summary | The Summary of the Test | ${Tests[n].Summary} |
| Issue Assignee | Full name of the assignee | ${fullname:Tests[n].Assignee} |
| Requirements | List of requirements covered by this Test (Check the template to see the extra cycle we need to list this information) | ${JQLIssues[a].Key} |
| #Test Executions | Number of Test Executions for each Test (Check the template to see the extra cycle we need to list this information) | ${jqlcount:issue in testTestExecutions('${Tests[n].Key}') and 'Test Plan' = ${Key}} |
| Latest Status | Latest Status of the execution | ${Tests[n].LatestStatus} |
This information is presented in a table as we can see below:
Some particularities to highlight a different behavior about the code needed to show the Tests Runs section:
...
This section have a table with information regarding each Test Execution in this Test Plan (and will repeat these sections for each Test Execution). The information is presented as a table with the following fields:
| Field | Description | Sample Code |
|---|---|---|
| Execution status | Execution Status of the Test Run | ${TestExecutions[n].TestRuns[a].Execution Status} |
| Assignee | Full Name of the Assignee of the Test Run | ${fullname:TestExecutions[n].TestRuns[a].Assignee} |
| Executed By | Full Name of the entity that has executed this Test Run | ${fullname:TestExecutions[n].TestRuns[a].Executed By} |
| Started On | Timestamp of the Started Date from the TestRun | ${dateformat('dd-MM-yyyy HH:mm:ss'):TestExecutions[n].TestRuns[a].Started On} |
| Finished On | Timestamp of the Finished Date from the TestRun | ${dateformat('dd-MM-yyyy HH:mm:ss'):TestExecutions[n].TestRuns[a].Finished On} |
| Versions | Fix Version field associated with the TestRun | ${TestExecutions[n].TestRuns[a].FixVersions} |
| Revision | Revision assigned to the TestRun | ${TestExecutions[n].TestRuns[a].Revision} |
All of these fields have code to handle empty fields. The resulting table look like the one below.
Execution Defects
If any Defects was found and associated globally with a TesRun it will appear here in the form of a table with the following fields:
| Key | Description | Sample Code |
|---|---|---|
| Key | Jira Key of the Defect in the form of a link | @{title=${TestExecutions[n].TestRuns[a].ExecutionDefects[d].Key}|href=${BaseURL}/browse/${TestExecutions[n].TestRuns[a].ExecutionDefects[d].Key}} |
| Summary | Summary of the Defect | ${TestExecutions[n].TestRuns[a].ExecutionDefects[d].Summary} |
| Priority | Priority associated with the defect | ${TestExecutions[n].TestRuns[a].ExecutionDefects[d].Priority} |
The table will be similar to the one below.
Execution Evidences
...
To obtain that information we have used the following code:
| Key | Description | Sample Code |
|---|---|---|
| File Name | The File Name of the Evidence attached to the Execution | @{title=${TestExecutions[n].TestRuns[a].ExecutionEvidences[d].Name}|href=${TestExecutions[n].TestRuns[a].ExecutionEvidences[d].FileURL}} |
| Author | Author of the Evidence | ${TestExecutions[n].TestRuns[a].ExecutionEvidences[d].Author} |
| File Size | File Size of the Evidence in bytes | ${TestExecutions[n].TestRuns[a].ExecutionEvidences[d].Size} |
| Evidence | The Evidence attached to the Execution | !{${TestExecutions[n].TestRuns[a].ExecutionEvidences[d].Evidence|maxwidth=100}} |
The table in case of an Evidence is of the type image will have the following aspect:
Comment
The comment associated to the TestRun (${wiki:TestExecutions[n].TestRuns[a].Comment}).
...
This section only appears if you have any attachments associated to the Test.
| Key | Description | Sample Code |
|---|---|---|
| File Name | File Name of the Attachment | @{title=${TestExecutions[n].TestRuns[a].Attachments[b].Name}|href=${TestExecutions[n].TestRuns[a].Attachments[b].FileURL}} |
| Author | The Author of the attachment | ${TestExecutions[n].TestRuns[a].Attachments[b].Author} |
| File Size | File Size of the attachments in bytes. | ${TestExecutions[n].TestRuns[a].Attachments[b].Size} |
This appears in the document in a table form:
Preconditions
This section only appear if you have a Precondition associated with the TestRun.
| Key | Description | Sample Code |
|---|---|---|
| Key | Key of the Precondition | @{title=${TestExecutions[n].TestRuns[a].PreConditions[c].Key}|href=${BaseURL}/browse/${TestExecutions[n].TestRuns[a].PreConditions[c].Key}} |
| Summary | Summary of the Precondition | ${TestExecutions[n].TestRuns[a].PreConditions[c].Summary} |
| Condition | Condition field present in the Precondition | ${wiki:TestExecutions[n].TestRuns[a].PreConditions[c].Conditions} |
A sub section will appear with the preconditions definitions.
Parameters
This section lists the existing parameters of the TestRun (we are iterating through the Parameters of the TestRun with: #{for m=TestExecutions[n].TestRuns[a].ParametersCount}).
| Key | Description | Sample Code |
|---|---|---|
| Name | Key of the parameter | ${TestExecutions[n].TestRuns[a].Parameters[m].Key} |
| Value | Value of the parameter | ${TestExecutions[n].TestRuns[a].Parameters[m].Value} |
It will list the Key and the Value of each parameter in a table.
Iterations
This section uses a sentence to show how many interactions we will go into more details in the next sections.
| Key | Description | Sample Code |
|---|---|---|
| Iterations | The iterations count of the Test Run | ${TestExecutions[n].TestRuns[a].IterationsCount} |
A sentence is added to the document with this information.
Iteration Overall Execution Status
To obtain the overall execution status of the iteration we use two variables:
| Key | Description | Sample Code |
|---|---|---|
| List of Statuses | Show the List of Statuses | ${TestExecutions[n].TestRuns[a].Iterations Overall Execution Status} |
TO DO EXECUTING PASS FAIL ABORTED | Overall Execution Status per Status | ${TestExecutions[n].TestRuns[a].Iterations Overall Execution Status.TO DO} ${TestExecutions[n].TestRuns[a].Iterations Overall Execution Status.EXECUTING} ${TestExecutions[n].TestRuns[a].Iterations Overall Execution Status.PASS} ${TestExecutions[n].TestRuns[a].Iterations Overall Execution Status.FAIL} ${TestExecutions[n].TestRuns[a].Iterations Overall Execution Status.ABORTED} |
The above code will produce the below table.
Test Run details
...
We extract that information using the following fields:
| Key | Description | Sample Code |
|---|---|---|
| Iteration Name | Name of the iteration | ${TestExecutions[n].TestRuns[a].Iterations[m].Name} |
| Status | Status of the iteration | ${TestExecutions[n].TestRuns[a].Iterations[m].Status} |
| Total Parameters | Total number of parameters | ${TestExecutions[n].TestRuns[a].Iterations[m].ParametersCount} |
| Parameters | Lists all parameters in the form of Key=Value | ${TestExecutions[n].TestRuns[a].Iterations[m].Parameters} |
This section will have the below appearance:
Iteration precondition definition
If a precondition is present we will use the following fields to extract that information:
| Key | Description | Sample Code |
|---|---|---|
| Key | Iteration precondition key | ${TestExecutions[n].TestRuns[a].Iterations[m].PreConditions[l].Key} |
| Definition | Iteration precondition definition | ${wiki:TestExecutions[n].TestRuns[a].Iterations[m].PreConditions[l].PreCondition.Definition} |
This will produce an entry like the one below:
Iteration parameters details
For that given Iteration we are listing the parameters used, that information is extracted with the following fields:
| Key | Description | Sample Code |
|---|---|---|
| Name | Parameter Key | ${TestExecutions[n].TestRuns[a].Iterations[m].Parameters[l].Key} |
| Value | Parameter Value | ${TestExecutions[n].TestRuns[a].Iterations[m].Parameters[l].Value} |
It generates a table of the following form:
Iteration Test Step Details
In this section we are listing the details of an iteration, we are listing each step present with details, the code we use for that purpose i present in the below table.
| Key | Description | Sample Code |
|---|---|---|
Step | The Step Number | ${TestExecutions[n].TestRuns[a].Iterations[m].TestSteps[r].StepNumber} |
| Action | Action defined in the Test Step | ${wiki:TestExecutions[n].TestRuns[a].Iterations[m].TestSteps[r].Action} |
| Data | Data defined in the Test Step | ${TestExecutions[n].TestRuns[a].Iterations[m].TestSteps[r].Data} |
| Expected Result | Expected Result defined in the Test Step | ${wiki:TestExecutions[n].TestRuns[a].Iterations[m].TestSteps[r].ExpectedResult} |
| Attachments | Attachments present in each Test Step (showing the FileURL and a screenshot in case of the Attachment being an image) | @{title=${TestExecutions[n].TestRuns[a].Iterations[m].TestSteps[r].Attachments[sa].Name}|href=${TestExecutions[n].TestRuns[a].Iterations[m].TestSteps[r].Attachments[sa].FileURL}} !{${TestExecutions[n].TestRuns[a].Iterations[m].TestSteps[r].Attachments[sa].Attachment|maxwidth=100}} |
| Comment | Comment | ${wiki:TestExecutions[n].TestRuns[a].Iterations[m].TestSteps[r].Comment} |
| Defects | Defects associated to this Iteration | @{title=${TestExecutions[n].TestRuns[a].Iterations[m].TestSteps[r].Defects[dc].Key}|href=${BaseURL}/browse/${TestExecutions[n].TestRuns[a].Iterations[m].TestSteps[r].Defects[dc].Key}} |
| Evidence | FileURL and screenshot (if it is an image) of the Evidence | @{title=${TestExecutions[n].TestRuns[a].Iterations[m].TestSteps[r].Evidences[e].Name}|href=${TestExecutions[n].TestRuns[a].Iterations[m].TestSteps[r].Evidences[e].FileURL}} by ${TestExecutions[n].TestRuns[a].Iterations[m].TestSteps[r].Evidences[e].Author} - ${TestExecutions[n].TestRuns[a].Iterations[m].TestSteps[r].Evidences[e].Size} !${TestExecutions[n].TestRuns[a].Iterations[m].TestSteps[r].Evidences[e].Evidence|maxwidth=100} |
| Status | Test Step Status | ${TestExecutions[n].TestRuns[a].Iterations[m].TestSteps[r].Status} |
The above information is gathered in a table like the one below:
Test Details
This section shows the Test details, for that we are considering the different possible Test we can have in Xray: Generic, Manual and Cucumber. For each type we will fetch different information.
It may seem similar with the Iteration Test Step Details section but in this section we will show the Test details (not instantiated in each Iteration like the previous section).
| Type | Key | Description | Sample Code | Output |
|---|---|---|---|---|
| Generic | Test Type | Test Type field | ${TestExecutions[n].TestRuns[a].TestType} |
| |||
| Specification | Definition of the Generic test | ${TestExecutions[n].TestRuns[a].Generic Test Definition} | |
| Cucumber | Test Type | Test Type field | ${TestExecutions[n].TestRuns[a].TestType} |
| |||
| Gherkin Specification | Gherkin specification of the Test | ${TestExecutions[n].TestRuns[a].Cucumber Scenario} #{for h=JQLIssuesCount|clause=key in testExampleResult('${TestExecutions[n].TestRuns[a].Key')} ${TestExecutions[n].TestRuns[a].TestStepsResults[h]} #{end} | |
| Manual | Step | Step Number | ${TestExecutions[n].TestRuns[a].TestSteps[r].StepNumber} |
| ||
| Action | Action of the Test Step | ${TestExecutions[n].TestRuns[a].TestSteps[r].Action} |
| Data | Data of the Test Step | ${TestExecutions[n].TestRuns[a].TestSteps[r].Data} |
| Expected Result | Expected Result of the Test Step | ${TestExecutions[n].TestRuns[a].TestSteps[r].ExpectedResult} |
| Attachment | Attachment of the Test Step | ${TestExecutions[n].TestRuns[a].TestSteps[r].Attachments[sa].FileURL} |
| Comment | Comment of the Test Step | ${wiki:TestExecutions[n].TestRuns[a].TestSteps[r].Comment} |
| Defects | Defects associated with the Test Step | ${TestExecutions[n].TestRuns[a].TestSteps[r].Defects[dc].Key} |
| Evidence | Evidence with the Test Step | ${TestExecutions[n].TestRuns[a].TestSteps[r].Evidences[e].FileURL}} !${TestExecutions[n].TestRuns[a].TestSteps[r].Evidences[e].Evidence|maxwidth=100} |
| Status | Status of the Test Step | ${TestExecutions[n].TestRuns[a].TestSteps[r].Status} |
Requirements linked with this test
For each Test we are listing the Requirements linked
| Key | Description | Sample Code |
|---|---|---|
| Requirement Key | Key of the Requirement | ${TestExecutions[n].TestRuns[a].Requirements[x].Key} |
| Requirement Summary | Summary of the Requirement | ${wiki:TestExecutions[n].TestRuns[a].Requirements[x].Summary} |
| Workflow Status | Workflow status of the Requirement | ${TestExecutions[n].TestRuns[a].Requirements[x].Status} |
This section present a table with that information like the one below:
Appendix A: Approval
This section is added for the cases where you need to have a signature validating the document.
Customizing the report
...
| Info | ||
|---|---|---|
| ||
${set(showTestRunDetails, 1)} |
| Variable/flag | Purpose | default | example(s) |
|---|---|---|---|
showTestRunDetails | render this section
| 0 | ${set(showTestRunDetails, 0)} |
| showTestRunEvidences | render this section
| 0 | ${set(showTestRunEvidences, 0)} |
| showTestRunAttachments | render this section
| 0 | ${set(showTestRunAttachments, 0)} |
| showTestRunIterations | render this section
| 0 | ${set(showTestRunIterations, 0)} |
| statusesToShowFirst | render Test Summary section whose reported status is one in this list first (delimited by comma); use an empty string '' to include all statuses
| '' (i.e., all statuses) | ${set(statusesToInclude, 'FAILED')} ${set(statusesToInclude, 'FAIL,EXECUTING')} ${set(statusesToInclude, '')} |
Adding or removing information to/from the report
...