Image Modified

...

Overview

In this example, we are configuring a Remote Job Trigger for Jenkins that executes Playwright tests and sends the execution results back to Xray.

Prerequisites

Configure a new RJT for Jenkins in Xray

This example requires configuration on both sides (Xray and Jenkins) to take advantage of the combination of both tools.

The jenkinsfile will configure a multi-step pipeline that extracts the Playwright test code, executes it and ships the execution results back to Xray.

Configure Jenkins using a jenkinsfile

We use a jenkinsfile to configure the pipeline in Jenkins.

pipeline {
  parameters {
    string(name: 'projectKey', defaultValue: '')
    string(name: 'testExecKey', defaultValue: '')
  }
  agent {
    docker { 
      image 'mcr.microsoft.com/playwright:v1.27.0-focal'
    } 
  }
  stages {
    stage('install playwright') {
      steps {
        sh '''
          npm i -D @playwright/test
          npx playwright install
        '''
      }
    }
    stage('test') {
      steps {
        catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') {
          sh '''
            PLAYWRIGHT_JUNIT_OUTPUT_NAME=xray-report.xml npx playwright test
          '''
        }
      }
    }
    stage('Import results to Xray') {
      steps {
        step([$class: 'XrayImportBuilder', endpointName: '/junit', importFilePath: 'xray-report.xml', importToSameExecution: 'true', projectKey: params.projectKey, testExecKey: params.testExecKey, fixVersion: '1.2', revision: '131', serverInstance: '10be58cc-2776-49a7-be60-b615dc99f4c0'])
      }
    }
    stage('Extract Variable from log'){
      steps {
        script {
          def logContent = Jenkins.getInstance().getItemByFullName(env.JOB_NAME).getBuildByNumber(Integer.parseInt(env.BUILD_NUMBER)).logFile.text
          env.testExecs = (logContent =~ /XRAY_TEST_EXECS:.*/).findAll().first()
          echo testExecs
         }
      }
    }
  }
  post 
  {
    always {
      junit '*.xml'
    }
  }
}

On the above jenkinsfile we are defining two parameters that will be passed when the build is invoked.

...
parameters {
    string(name: 'projectKey', defaultValue: '')
    string(name: 'testExecKey', defaultValue: '')
  }
...

The received parameters can be used in the remaining steps of the pipeline. To define what are the parameters, we have added a parameters section with the name of the parameter and a possible default value.

Within the step to run the tests, we added a function to ensure that the next step is performed even if the actual step fails. This is important to ensure that the results are sent to Xray.

...
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') {
...

Using the catchError method in case of an error, we are forcing the step to continue the Pipeline execution from the statement following the catchError step.

The behavior of the step when an exception is thrown can be configured to print a message, set a build result other than failure, change the stage result, or ignore certain kinds of exceptions that are used to interrupt the build.

Once the jenkinsfile is uploaded in Jenkins it is ready to perform the tasks defined in the stages and steps.

We are going to dive into the last part of the jenkinsfile, where we send the results back to Xray. 

Configure a Remote Jobs Trigger in Xray for Jenkins

To use the configuration of

...

an RJT in Xray, you need to access

...

the Project Settings area and, under Xray Settings, click on the Remote Jobs Trigger option.

Image Modified

This will open the Remote Jobs Trigger configuration page where we can configure remote jobs for Jenkins, Bamboo, GitLab, GitHub and Azure DevOps. You can activate and deactivate the configurations by switching the toggles next to each.

Image Modified

After you have activated the configuration that you are interested, in our case the Jenkins Configuration, click the New Jenkins Configuration to configure your new job. This opens a new form that must filled with proper information.

Image Modified

Fill the Configuration Name with meaningful information that allows you to know the job's purpose just by reading the name.

The Job Name is the name of the project/job in Jenkins, make sure that they match.

Next we have the API Token, this can be defined in the project configuration in Jenkins. Access your project page in Jenkins and scroll down to the Build Triggers area.

Image Modified

The Authentication Token defined in Jenkins must be the same defined in Xray API Token of the Remote Jobs Trigger configuration, in our case 'MyToken'.

In the Username field, define the name of the Jenkins user with proper permissions to invoke the Jenkins Job. Make sure it has the right permissions in Jenkins or the job will return an authentication error when used.

Fill the API URL with the base URL of your Jenkins instance and in the Password field put the password used by the user identified by the Username above.

Finally we have reached the last configuration fields available: Parameters (Optional), that is a list of key/value pairs that you have defined in

...

your Jenkins pipeline and want to pass from Xray.

You can define static ones or you can use dynamic filled fields available in Xray (more info here). In our case we want to pass the Project key and the Test Execution key so that we can use them when shipping the execution results back to Xray.

For that we use the following options available from Xray:

• ${PROJECT_KEY} that will be filled with the key of the project from where the job is called.
• ${ISSUE_KEY} that will be filled with the issue key, in our case the Test Execution key, from where the job was started from.

The configuration with all the fields filled will look like this:

Image Modified

Once done click the Edit Configuration button and activate the configuration by switching the Status toggle to on:

Image Modified

...

Trigger

Once the configuration is done and active, you can navigate to, e.g., the Test Execution issue, from where you want to trigger the job. You will see that a new button is available to trigger the job you configured earlier.

Image Modified

When you choose to call the Remote Jobs Trigger job a new confirmation window will appear with the status of the invocation, either success or failure.

Image Modified

...

Send results back to Xray

In the above example, we have defined a job in another tool and invoked it from Xray. This job will execute the Playwright tests and generate a JUnit XML report.

Now, we need to have those results back into Xray for full visibility. The following section will show two ways to have those results shipped back into Xray.

pipeline {
  parameters {
    string(name: 'projectKey', defaultValue: '')
    string(name: 'testExecKey', defaultValue: '')
  }
...
stage('Import results to Xray') {
      steps {
        step([$class: 'XrayImportBuilder', endpointName: '/junit', importFilePath: 'xray-report.xml', importToSameExecution: 'true', projectKey: params.projectKey, testExecKey: params.testExecKey, fixVersion: '1.2', revision: '131', serverInstance: '10be58cc-2776-49a7-be60-b615dc99f4c0'])
      }
    }
...

...

Tips

• Make sure you define parameters in the jenkinsfile to be able to pass some.
• When defining parameters, ensure the parameters' names match in both tools.
• The Authentication Token from the Jenkins pipeline must match the API Token from the Xray side.
• Confirm that the user you are using has the correct permissions in both tools.
• If we're accessing the Jira cloud instance over HTTPS (which is the standard), then the Jenkins URL needs to be also HTTPS so Xray Cloud can trigger Jenkins APIs
• Configure your Jenkins instance to handle CORS properly. Go to Manage Jenkins > Plugins and add the CORS Filter plugin. Configure it as shown in the next screenshot, so that it accepts requests from your Xray cloud instance considering its region (e.g., https://eu.xray.cloud.getxray.app) and also the URL of your Jira cloud instance. Accepts GET, PUT, and POST requests. Include the "Authorization" header on the "Access-Control-Allow-Headers".
  • Image Added
•  You can use the browser developer tools to check if the remote job trigger from Xray is done successfully or not, and debug any error that might occur.

...

References

• Triggering Remote Jobs
• Jenkins pipeline integration
• Integration with Jenkins
• Playwright

...

.toc-btf {
    position: fixed;
}

...