Overview

In this example, we are configuring a Remote Job Trigger for Jenkins that executes Playwright tests and sends the execution results back to Xray.

Prerequisites

Configure a new RJT for GitHub in Xray

This example requires configuration on both sides (Xray and GitHub) to take advantage of the combination of both tools.

The workflow will configure a multi-step pipeline that extracts the Playwright test code, executes it, and ships the execution results back to Xray.

Configure GitHub using a workflow definition

The workflow definition we have used to define this pipeline is below.

name: Node.js CI

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]
  workflow_dispatch:
    inputs:
      testExecKey:
        description: 'Test Exec ID from Xray'
        default: ''
        required: true
        type: string
      projectKey:
        description: 'Project ID from Xray'
        default: ''
        required: true
        type: string
jobs:
  build:
    runs-on: ubuntu-latest

    strategy:
      matrix:
        node-version: [15.x]
        # See supported Node.js release schedule at https://nodejs.org/en/about/releases/

    steps:
    - uses: actions/checkout@v2
    - name: Output Inputs
      run: echo "${{ toJSON(github.event.inputs) }}"
    - name: Use Node.js ${{ matrix.node-version }}
      uses: actions/setup-node@v1
      with:
        node-version: ${{ matrix.node-version }}
    - run: |
        npm install
        npm i -D @playwright/test
        npx playwright install 
    - name: Run tests
      continue-on-error: true
      run: |  
        PLAYWRIGHT_JUNIT_OUTPUT_NAME=xray-report.xml npx playwright test
    - name: Output results file
      run: |
        cat xray-report.xml
    - name: "Import results to Xray"
      uses: mikepenz/xray-action@v2.3.0
      with:
         username: ${{ secrets.JIRA_USERNAME }}
         password: ${{ secrets.JIRA_PASSWORD }}
         xrayCloud: "false"
         xrayBaseUrl: ${{ secrets.JIRA_BASEURL }}
         testFormat: "junit"
         testPaths: "**/xray-report.xml"
         testExecKey: "${{ inputs.testExecKey }}"
         projectKey: "${{ inputs.projectKey }}"
         testEnvironments: "NODE_${{ matrix.node-version }}"

In the above workflow, we are defining two parameters that will be passed when the build is invoked and setting the workflow type as a workflow_dispatch.

...
workflow_dispatch:
    inputs:
      testExecKey:
        description: 'Test Exec ID from Xray'
        default: ''
        required: true
        type: string
      projectKey:
        description: 'Project ID from Xray'
        default: ''
        required: true
        type: string
...

The parameters received can be used in the remaining steps of the pipeline. We have added an inputs section with the parameter's name and some properties to define the parameters.

Within the step to run the tests, we added a function to ensure that the next step is performed even if the actual step fails. This is important to ensure that the results are sent to Xray.

...
- name: Run tests
      continue-on-error: true
      run: |  
        PLAYWRIGHT_JUNIT_OUTPUT_NAME=xray-report.xml npx playwright test
...

Using the continue-on-error:true method in case of an error, we are forcing the step to continue the Pipeline execution from the statement following the continue-on-error definition.

Once the workflow is defined in GitHub, it can perform the tasks defined in the stages and steps.

We will dive into the last part of the GitHub workflow, where we send the results back to Xray, in another section. 

Configure a Remote Jobs Trigger in Xray for GitHub

To use the configuration of a RJT in Xray, access the Project Settings area and click on the Apps → Xray Settings → Remote Jobs Trigger option.

This will open the Remote Jobs Trigger configuration page where we can configure remote jobs for Jenkins, Bamboo, GitLab, GitHub and Azure DevOps. You can activate and deactivate the configurations by switching the toggles next to each configuration entry.

After activating the configuration you are interested in, in our case, the GitHub Configuration, click the New GitHub Configuration to configure your new job. This opens a new form that must be filled with proper information.

Fill the Configuration Name with meaningful information that allows you to know the job's purpose just by reading the name.

Next we have the Workflow Id, this is the name of the workflow that you have defined in GitHub. Access your project page in GitHub and click on the Actions tab. On the left side you will have the list of available workflows, click over the one you want to use and copy the name of it.

Fill the API URL with the endpoint of GitHub (api.github.com).

The Repository Name is the repository's name from GitHub, without the organization.

In the Branch Name insert the branch name to use (e. g. 'main').

In the Username field define the user name with proper permissions to invoke the GitHub workflow. Make sure it has the right permissions in GitHub or the job will return an authentication error when used.

The Authentication Token can be found in your GH profile settings, accessing Settings → Developers Settings → Personal access tokens → Tokens. If you do not have any you can create one to be used here.

Finally we have reached the last configuration fields available:

• Parameters (Optional), that is a list of key/value pairs that you have defined in you GitHub pipeline and want to pass from Xray.

You can define static ones or you can use dynamic filled fields available in Xray (more info here). In our case we want to pass the Project key and the Test Execution key so that we can use them when shipping the execution results back to Xray.

For that we use the following options available from Xray:

• ${PROJECT_KEY} that will be filled with the key of the project from where the job is called.
• ${ISSUE_KEY} that will be filled with the issue key, in our case the Test Execution key, from where the job was started from.

The configuration with all the fields filled will look like this:

Once done click the Save button and activate the configuration by switching the Status toggle to on:

Trigger

Once the configuration is done and active, you can navigate to the Test Execution from where you want to trigger the job; there, you will see that a new button is available to trigger the job you just have configured.

When you choose to call the Remote Jobs Trigger job a new confirmation window will appear with the status of the invocation, either success or failure.

Send results back to Xray

In the above example we have defined a job in another tool and invoked it from Xray. This job will execute the Playwright tests and generate a JUnit report.

Now we need to have those results back into Xray for full visibility. The following section will show you two ways you can use to have those results shipped back into Xray.

...
workflow_dispatch:
    inputs:
      testExecKey:
        description: 'Test Exec ID from Xray'
        default: ''
        required: true
        type: string
      projectKey:
        description: 'Project ID from Xray'
        default: ''
        required: true
        type: string
...
- name: "Import results to Xray"
      uses: mikepenz/xray-action@v2.3.0
      with:
         username: "${{ secrets.XRAYCLOUD_CLIENT_ID }}"
         password: "${{ secrets.XRAYCLOUD_CLIENT_SECRET }}"
         xrayCloud: "true"
         testPaths: "**/xray-report.xml"
         testExecKey: "${{ inputs.testExecKey }}"
         projectKey: "${{ inputs.projectKey }}"
         testEnvironments: "NODE_${{ matrix.node-version }}"
...

Tips

• Make sure you define workflow parameters if you want to pass some.
• When defining parameters make sure that the parameters names match in both tools.
• The Authentication Token from GitHub must have the correct permissions and accesses.
• Confirm that the user that you are using have the correct permissions in both tools.

References

• Triggering Remote Jobs
• Integration with GitHub
• Xray-action
• Playwright