Page History
...
Children Display | ||||
---|---|---|---|---|
|
Upgrading to version 2.1.0
Warning | ||
---|---|---|
| ||
|
Installation
The installation is made manually. For more information on how to install add-ons, please refer to how to install add-ons.
...
Format and specific endpoint | Field | Description | ||||
---|---|---|---|---|---|---|
Behave JSON multipart Cucumber JSON multipart NUnit XML multipart JUnit XML multipart Robot XML multipart TestNG XML multipart | Import to Same Test Execution | When this option is check, if you are importing multiple execution report files using a glob expression, the results will be imported to the same Test Execution | ||||
Test execution fields | An object (JSON) specifying the fields for the issue. You may specify the object either directly in the field or in the file path.
| |||||
NUnit XML JUnit XML Robot XML TestNG XML | Import to Same Test Execution | When this option is check, if you are importing multiple execution report files using a glob expression, the results will be imported to the same Test Execution | ||||
Project key | Key of the project where the Test Execution (if the Test Execution Key field wasn't provided) and the Tests (if they aren't created yet) are going to be created | |||||
Test execution key | Key of the Test Execution | |||||
Test plan key | Key of the Test Plan | |||||
Test environments | List of Test Environments separated by ";" | |||||
Revision | Source code's revision being target by the Test Execution | |||||
Fix version | The Fix Version associated with the test execution (it supports only one value) |
...
Cucumber
In a typical Cucumber Workflow, after having created a Cucumber project and the Cucumber tests specified in Jira, you may want to have a project that exports the features from Jira, executes the automated tests on a CI environment and then imports back its results.
For this scenario, the Jenkins project would be configured with a set of tasks responsible for:
- Pulling the Cucumber project
- Exporting Cucumber features from Jira to your Cucumber project
- Executing the tests in the CI environment
- Importing the execution results back to Jira
Exporting Cucumber features
To start the configuration, add the build step Xray: Cucumber Features Export Task.
After that, configure it.
In this example, we configured the task to extract the features from a set of issues (PROJ-78 and PROJ-79) to the folder that holds the Cucumber project.
Importing Cucumber features
To start the configuration, add the build step Xray: Cucumber Features Import Task.
After that, configure it.
In this example, we configured the task to import to the Project IF of the Xray instance all the .features and .zip files that are contained in \Cucumber directory and sub directories, which were modified in the last 3 hours.
Importing the execution results
To start the configuration, add the post-build action Xray: Results Import Task.
After that, configure it.
In this example, we configured the task to import the Cucumber JSON results back to Jira.
Once all configurations are done, click Save at the bottom of the page.
After running the job, the expected result is a new Test Execution issue created in the Jira instance.
Importing the execution results with user-defined field values
For Cucumber, Behave, JUnit, Nunit and Robot, Xray for Jenkins allows you to create new Test Executions and have control over newly-created Test Execution fields. You can send two files, the normal execution result file and a JSON file similar to the one Jira uses to create new issues. More details regarding how Jira creates new issues here.
For this scenario and example, the import task needs to be configured with the Cucumber JSON Multipart format. When selecting this option, you can additionally configure the Test Execution fields in one of two ways:
...
Xray: Build Enviroment Variables
Since version 2.2.0, the Xray plugin will now set some build enviroment variables according to the operation result after each of the Xray Steps mensioned above.
Build Enviroment Variable Name | Meaning and Value |
---|---|
XRAY_IS_REQUEST_SUCCESSFUL | Contains the string 'true' if all requests made by the step were sucesseful, or 'false' otherwise. |
XRAY_ISSUES_MODIFIED | All Issue keys that were modified and/or created by the step, seperated by ';' with no duplicated entries (E.g. 'CALC-100;CALC-101;CALC-102'). |
XRAY_RAW_RESPONSE | The unprocessed JSON response of all requests made by the step, seperated by ';'. |
XRAY_TEST_EXECS | All Test Execution Issue keys that were modified and/or created by the step, seperated by ';' with no duplicated entries (E.g. 'CALC-200;CALC-201;CALC-202'). Please note that in same cases, it will be not possible to determine the issue type of the Issue key returned in the request response and in that case, the key it will only be added to the XRAY_ISSUES_MODIFIED variable. |
XRAY_TEST | All Test Issue keys that were modified and/or created by the step, seperated by ';' with no duplicated entries (E.g. 'CALC-300;CALC-301;CALC-302'). Please note that in same cases, it will be not possible to determine the issue type of the Issue key returned in the request response and in that case, the key it will only be added to the XRAY_ISSUES_MODIFIED variable. |
Info | ||
---|---|---|
| ||
Due to Jenkins limitations, these variables will not be set on Pipeline projects. |
Examples
Cucumber
In a typical Cucumber Workflow, after having created a Cucumber project and the Cucumber tests specified in Jira, you may want to have a project that exports the features from Jira, executes the automated tests on a CI environment and then imports back its results.
For this scenario, the Jenkins project would be configured with a set of tasks responsible for:
- Pulling the Cucumber project
- Exporting Cucumber features from Jira to your Cucumber project
- Executing the tests in the CI environment
- Importing the execution results back to Jira
Exporting Cucumber features
To start the configuration, add the build step Xray: Cucumber Features Export Task.
After that, configure it.
In this example, we configured the task to extract the features from a set of issues (PROJ-78 and PROJ-79) to the folder that holds the Cucumber project.
Importing Cucumber features
To start the configuration, add the build step Xray: Cucumber Features Import Task.
After that, configure it.
In this example, we configured the task to import to the Project IF of the Xray instance all the .features and .zip files that are contained in \Cucumber directory and sub directories, which were modified in the last 3 hours.
Importing the execution results
To start the configuration, add the post-build action Xray: Results Import Task.
After that, configure it.
In this example, we configured the task to import the Cucumber JSON results back to Jira.
...
In this example, we configured the following object:
Code Block | ||
---|---|---|
| ||
{
"fields": {
"project": {
"key": "PROJ"
},
"summary": "Test Execution for Cucumber results (Generated by job: ${BUILD_TAG})",
"issuetype": {
"id": "10102"
}
}
} |
And configured the task to import the Cucumber JSON Multipart results back to Jira.
Once all configurations are done, click Save at the bottom of the page.
After running the jobthe job, the expected result is a new Test Execution issue created in the Jira instance, with the Test Execution fields as specified in the Jenkins build step configuration.
JUnit
Importing the execution results with user-defined field values
For Cucumber, Behave, JUnit, Nunit and RobotApart from supporting Cucumber natively, Xray enables you to take advantage of many other testing frameworks like JUnit. In this sense, Xray for Jenkins lets you import results in other formats besides Cucumber JSON.
If you want to import JUnit XML reports, a typical Job outline would be:
- Pulling the JUnit project
- Executing the tests in the CI environment
- Importing the execution results, including Tests, to JIRA
Importing the execution results
To start the configuration, add the post-build action Xray: Results Import Task.
After that, configure it.
In this example, we have a configuration where the JUnit XML format is chosen.
After running the plan, the expected result is a new Test Execution issue created in the JIRA instance.
You can also import multiple results using a glob expression, like in the following example
Pipeline projects support
Xray for Jenkins provides support for pipelines projects, allowing you to use Xray specific tasks.
Here is a simple example of a pipeline script using the Xray: Cucumber Features Export Task
allows you to create new Test Executions and have control over newly-created Test Execution fields. You can send two files, the normal execution result file and a JSON file similar to the one Jira uses to create new issues. More details regarding how Jira creates new issues here.
For this scenario and example, the import task needs to be configured with the Cucumber JSON Multipart format. When selecting this option, you can additionally configure the Test Execution fields in one of two ways:
- Insert the relative path to the JSON file containing the information, or
- Insert the JSON content directly in the field.
In this example, we configured the following object:
Code Block | ||||
---|---|---|---|---|
| ||||
{
"fields": {
"project": | ||||
Code Block | ||||
| ||||
pipeline { agent any stages { stage('Export Cucumber') { "key": "PROJ" }, steps {"summary": "Test Execution for Cucumber results (Generated by job: ${BUILD_TAG})", "issuetype": { step([$class: 'XrayExportBuilder', filePath: '\\features', issues: 'IF-1', serverInstance: '2ffc3a3e-9e2f-4279-abcd-e9301fe47bed']) } "id": "10102" } } } |
Info | ||
---|---|---|
| ||
For Pipeline specific documentation, you may want to give a look at: |
Examples
JUnit
This is a declarative example, for JUnit based tests.
And configured the task to import the Cucumber JSON Multipart results back to Jira.
Once all configurations are done, click Save at the bottom of the page.
After running the job, the expected result is a new Test Execution issue created in the Jira instance, with the Test Execution fields as specified in the Jenkins build step configuration.
JUnit
Apart from supporting Cucumber natively, Xray enables you to take advantage of many other testing frameworks like JUnit. In this sense, Xray for Jenkins lets you import results in other formats besides Cucumber JSON.
If you want to import JUnit XML reports, a typical Job outline would be:
- Pulling the JUnit project
- Executing the tests in the CI environment
- Importing the execution results, including Tests, to JIRA
Importing the execution results
To start the configuration, add the post-build action Xray: Results Import Task.
After that, configure it.
In this example, we have a configuration where the JUnit XML format is chosen.
After running the plan, the expected result is a new Test Execution issue created in the JIRA instance.
You can also import multiple results using a glob expression, like in the following example
Pipeline projects support
Xray for Jenkins provides support for pipelines projects, allowing you to use Xray specific tasks.
Here is a simple example of a pipeline script using the Xray: Cucumber Features Export Task
Code Block | ||||
---|---|---|---|---|
| ||||
pipeline {
agent any
stages {
stage('Export Cucumber') {
steps {
step([$class: 'XrayExportBuilder', filePath: '\\features', issues: 'IF-1', serverInstance: '2ffc3a3e-9e2f-4279-abcd-e9301fe47bed'])
}
}
}
} |
Info | ||
---|---|---|
| ||
For Pipeline specific documentation, you may want to give a look at: |
Examples
JUnit
This is a declarative example, for JUnit based tests.
Code Block | ||||
---|---|---|---|---|
| ||||
pipeline {
agent any
stages {
stage('Compile'){
steps {
checkout([$class: 'GitSCM', branches: [[name: '*/master']], doGenerateSubmoduleConfigurations: false, extensions: [[$class: 'SparseCheckoutPaths', sparseCheckoutPaths: [[path: 'java-junit-calc/']]]], submoduleCfg: [], userRemoteConfigs: [[credentialsId: 'a3285253-a867-4ea7-a843-da349fd36490', url: 'ssh://git@localhost/home/git/repos/automation-samples.git']]])
sh "mvn clean compile -f java-junit-calc/pom.xml"
}
}
stage('Test'){
steps{
sh "mvn test -f java-junit-calc/pom.xml"
}
}
stage('Import results to Xray') {
steps {
step([$class: 'XrayImportBuilder', endpointName: '/junit', fixVersion: 'v3.0', importFilePath: 'java-junit-calc/target/surefire-reports/*.xml', importToSameExecution: 'true', projectKey: 'CALC', serverInstance: '552d0cb6-6f8d-48ba-bbad-50e94f39b722'])
}
}
}
} |
JUnit multipart
This is a declarative example, for JUnit based tests using the multipart variant/endpoint which allows customization over the Test Execution issue fields.
By changing the value of the endpointName variable, you can easily adapt it for other automation frameworks (e.g. Robot framework, TestNG, NUnit).
Code Block | ||||
---|---|---|---|---|
| ||||
pipeline {
agent any
stages {
stage('Compile'){
steps {
checkout([$class: 'GitSCM', branches: [[name: '*/master']], doGenerateSubmoduleConfigurations: false, extensions: [[$class: 'SparseCheckoutPaths', sparseCheckoutPaths: [[path: 'java-junit-calc/']]]], submoduleCfg: [], userRemoteConfigs: [[credentialsId: 'a3285253-a867-4ea7-a843-da349fd36490', url: 'ssh://git@localhost/home/git/repos/automation-samples.git']]])
sh "mvn clean compile -f java-junit-calc/pom.xml"
}
}
stage('Test'){
steps{
sh "mvn test -f java-junit-calc/pom.xml"
}
}
stage('Import results to Xray') {
steps { | ||||
Code Block | ||||
| ||||
pipeline { agent any stages { stage('Compile'){ step([$class: 'XrayImportBuilder', endpointName: '/junit/multipart', importFilePath: 'java-junit-calc/target/surefire-reports/TEST-com.xpand.java.CalcTest.xml', steps importInfo: '''{ checkout([$class: 'GitSCM', branches: [[name: '*/master']], doGenerateSubmoduleConfigurations: false, extensions: [[$class: 'SparseCheckoutPaths', sparseCheckoutPaths: [[path: 'java-junit-calc/']]]], submoduleCfg: [], userRemoteConfigs: [[credentialsId: 'a3285253-a867-4ea7-a843-da349fd36490', url: 'ssh://git@localhost/home/git/repos/automation-samples.git']]])"fields": { "project": { sh "key"mvn clean compile -f java-junit-calc/pom.xml: "CALC" }, } "summary": "Test Execution for java junit ${BUILD_NUMBER}", stage('Test'){ "issuetype": steps{ sh "mvn test -f java-junit-calc/pom.xml" id": "9" }, } "customfield_11807": [ stage('Import results to Xray') { "CALC-1200" steps { ] step([$class: 'XrayImportBuilder', endpointName: '/junit', fixVersion: 'v3.0', importFilePath: 'java-junit-calc/target/surefire-reports/*.xml', importToSameExecution: 'true', projectKey: 'CALC } }''', inputInfoSwitcher: 'fileContent', serverInstance: '552d0cb6-6f8d-48ba-bbad-50e94f39b722']) } } } } |
Cucumber ("standard" workflow)
...