Overview

Xray enables easy integration with Jenkins through the "Xray for JIRA Jenkins Plugin", providing the means for successful Continuous Integration by allowing users to report automated testing results.

...

The installation is made manually. For more information on how to install add-ons, please refer to to how to install add-ons.

Manual Installation

...

Xray for Jenkins is configured in the global settings configuration page  Manage Jenkins >  > Configure System > Xray > Xray for Jira configuration.

Jira servers

The Jira servers configuration defines connections with Jira instances. 

To add a new Jira instance connection, you need to specify some properties:

1. Configuration alias
2. Hosting:   Hosting (instance type) in this case Server/Data Center.
3. Server Address: The address of the Jira Server where Xray is running
4. Credentials:
   
   1. Use the Jenkins Credentials PluginCredentials Plugin to set the username/password (if you are using a Server/Data Center instance).
   2. Make sure that the user you are using have the following permissions in the projects where you want to import the results and import/export feature files: View, Edit, Create.

...

This build step will export the Cucumber Tests (i.e., Scenario/Scenario Outlines) in .feature or bundled in a .zip file. The rules for exporting are defined defined here.

It invokes Xray's Export Cucumber Tests REST API endpoint (see more information here).

...

Some fields need to be configured in order to export the Cucumber Tests. As input, you can either specify issue keys (see the endpoint documention documention here) or the ID of the saved filter in Jira.

...

...

For those formats, the file path needs to be relative to the workspace.

Configuration Configuration

...

...

Examples

Cucumber

In a typical typical Cucumber Workflow, after having created a Cucumber project and the Cucumber tests specified in Jira, you may want to have a project that exports the features from Jira, executes the automated tests on a CI environment and environment and then imports back its results.

...

In this example, we configured the task to extract the the features from a set of issues (PROJ-78 and PROJ-79) to the folder that holds the Cucumber project.

...

To start the configuration, add the build step Xray Xray: Cucumber Features Import Task.

...

To start the configuration, add the post-build action Xray Xray: Results Import Task.

After that, configure it.

In this example,  we we configured the task to import the Cucumber JSON results back to Jira.

...

Once all configurations are done, click Save at the bottom of the page.

After running the jobthe job, the expected result is a new Test Execution issue created in the Jira instance.

...

For Cucumber, Behave, JUnit, Nunit and Robot, Xray for Jenkins allows you to create new Test Executions and have control over newly-created Test Execution fields. You can send two files, the normal execution result file and a JSON file similar to the one Jira one Jira uses to create new issues. More details regarding how Jira creates new issues here. 

For this scenario and example, the import task needs to be configured with the Cucumber JSON Multipart format. When selecting this option, you can additionally configure the Test Execution fields in in one of two ways:

• Insert the relative path to the JSON file containing the information, or
• Insert the JSON content directly in the field.
  
  

...

And configured the task to import the the Cucumber JSON Multipart results results back to Jira.

Once all configurations are done, click Save at the bottom of the page.

...

Apart from supporting Cucumber natively, Xray enables you enables you to take advantage of many other testing frameworks like JUnit. In this sense, Xray for Jenkins lets you import results in other formats besides Cucumber JSON.

...

To start the configuration, add the post-build action Xray Xray: Results Import  Task.

After that, configure it.

...

Here is a simple example of a pipeline script using the Xray: Cucumber Features Export Task Task

pipeline {
    agent any
    stages {
        stage('Export Cucumber') {
            steps {
               step([$class: 'XrayExportBuilder', filePath: '\\features', issues: 'IF-1', serverInstance: '2ffc3a3e-9e2f-4279-abcd-e9301fe47bed'])
            }
        }
    }
}

...

This is a declarative example, for JUnit based tests using the multipart variant/endpoint which allows customization over the Test Execution issue fields.

By changing the value of the the endpointName variable, you can easily adapt it for other automation frameworks (e.g. Robot framework, TestNG, NUnit).  

pipeline {
    agent any
    stages {
        stage('Compile'){
            steps {
                checkout([$class: 'GitSCM', branches: [[name: '*/master']], doGenerateSubmoduleConfigurations: false, extensions: [[$class: 'SparseCheckoutPaths', sparseCheckoutPaths: [[path: 'java-junit-calc/']]]], submoduleCfg: [], userRemoteConfigs: [[credentialsId: 'a3285253-a867-4ea7-a843-da349fd36490', url: 'ssh://git@localhost/home/git/repos/automation-samples.git']]])
                sh "mvn clean compile -f java-junit-calc/pom.xml"
            }
        }
        
        stage('Test'){
            steps{
                sh "mvn test -f java-junit-calc/pom.xml"
            }
        }
        
        stage('Import results to Xray') {
             steps {    
                step([$class: 'XrayImportBuilder', endpointName: '/junit/multipart', importFilePath: 'java-junit-calc/target/surefire-reports/TEST-com.xpand.java.CalcTest.xml', importInfo: '''{
           "fields": {
              "project": {
                 "key": "CALC"
              },
              "summary": "Test Execution for java junit ${BUILD_NUMBER}",
              "issuetype": {
                 "id": "9"
              },
              "customfield_11807": [
                 "CALC-1200"
              ]
           }
        }''', inputInfoSwitcher: 'fileContent', serverInstance: '552d0cb6-6f8d-48ba-bbad-50e94f39b722'])
            }
        }

    }
}

...

This is a declarative example, for Cucumber tests using the "VCS/Git based" workflow (see see Testing in BDD with Gherkin based frameworks (e.g. Cucumber)).

...

This is the simplest way to generate your step script, and we strongly recommend the use of this snippet due to the complexity of some task related parameters. 

Troubleshooting

The build process is failing with status code 403

When you check the log, it has the following:

By default, when you successively uccessively try to log into Jira with the wrong credentials, the Jira instance will prompt you to provide a CAPTCHA the next time you try to log in. It is not possible to provide this information via the build process, so it will fail with status code 403 Forbidden.

...