Bitbucket is a well-known CI/CD tool available on-premises and as SaaS. In this document we detail the integration with Bitbucket Cloud, namely using Pipelines.

Xray does not provide a specific plugin for Bitbucket; however, it is easy to set up Bitbucket in order to integrate it with Xray Cloud.

Since Xray provides a full REST API, you may interact with Xray, for submitting results for example.

Submitting test automation results to  Xray

JUnit example

In this scenario, we want to get visibility of the automated test results from some tests implemented in Java, using the JUnit framework. 

This recipe could also be applied for other frameworks such as NUnit or Robot (if supported).

We need to set up a Git repository containing the code along with the configuration for Bitbucket build process.

The tests are implemented in a JUnit class as follows.

package com.xpand.java;

import org.junit.After;
import org.junit.Before;
import org.junit.Test;

import static org.hamcrest.CoreMatchers.is;
import static org.junit.Assert.assertThat;

public class CalcTest {

    @Before
    public void setUp() throws Exception {

    }

    @After
    public void tearDown() throws Exception {

    }

	@Test
    public void CanAddNumbers()
    {
        assertThat(Calculator.Add(1, 1), is(2));
        assertThat(Calculator.Add(-1, 1), is(0));
    }


    @Test
    public void CanSubtract()
    {
        assertThat(Calculator.Subtract(1, 1), is(0));
        assertThat(Calculator.Subtract(-1, -1), is(0));
        assertThat(Calculator.Subtract(100, 5), is(95));
    }


    @Test
    public void CanMultiply()
    {
        assertThat(Calculator.Multiply(1, 1), is(1));
        assertThat(Calculator.Multiply(-1, -1), is(1));
        assertThat(Calculator.Multiply(100, 5), is(500));
    }


    public void CanDivide()
    {
        assertThat(Calculator.Divide(1, 1), is(1));
        assertThat(Calculator.Divide(-1, -1), is(1));
        assertThat(Calculator.Divide(100, 5), is(20));
    }


    @Test
    public void CanDoStuff()
    {
        assertThat(true, is(true));
    }


}

The Bitbucket Pipelines configuration file bitbucket-pipelines.yml contains the definition of the build steps, including running the automated tests and submitting the results.

# Use Maven 3.5 and JDK8
image: maven:3.5-jdk-8


pipelines:
  default:
    - step:
        caches:
          - maven
        script:
          - |
              echo "building my amazing repo..."
              mvn test
              export token=$(curl -H "Content-Type: application/json" -X POST --data "{ \"client_id\": \"$client_id\",\"client_secret\": \"$client_secret\" }" https://xray.cloud.getxray.app/api/v1/authenticate| tr -d '"')
              curl -H "Content-Type: text/xml" -H "Authorization: Bearer $token" --data @target/surefire-reports/TEST-com.xpand.java.CalcTest.xml  "https://xray.cloud.getxray.app/api/v1/import/execution/junit?projectKey=CALC"
              echo "done"

In order to submit those results, we'll just need to invoke the REST API (as detailed in Import Execution Results - REST).

However, we do not want to have the Xray API credentials hardcoded in Bitbucket Pipelines configuration file. Therefore, we'll use some environment variables defined in the Repository variables (within Pipeline configuration section), including:

• client_id: the client_id associated with the API key created in the Xray cloud instance 
• client_secret: the client_secret associated with the API key created in the Xray cloud instance

In bitbucket-pipelines.yml a "step" must be included that will use "curl" in order to first obtain a token and then finally submit the results to the REST API, using that token.

export token=$(curl -H "Content-Type: application/json" -X POST --data "{ \"client_id\": \"$client_id\",\"client_secret\": \"$client_secret\" }" https://xray.cloud.getxray.app/api/v1/authenticate| tr -d '"')

curl -H "Content-Type: text/xml" -H "Authorization: Bearer $token" --data @target/surefire-reports/TEST-com.xpand.java.CalcTest.xml  "https://xray.cloud.getxray.app/api/v1/import/execution/junit?projectKey=SP"

We're using "curl" utility that comes in Unix based OS'es but you can easily use another tool to make the HTTP request; however, "curl" is provided in the container used by Bitbucket.

Cucumber example

In this scenario, we are managing the specification of Cucumber Scenarios/Scenario Outline(s) based tests in Jira, using Xray, as detailed in the "standard workflow" mentioned in Testing in BDD with Gherkin based frameworks (e.g. Cucumber).

Then we need to extract this specification from Jira (i.e. generate related Cucumber .feature files), and run it in Bitbucket against the code that actually implements each step that is part of those scenarios.

Finally, we can then submit the results back to JIRA and they'll be reflected on the related entities. 

The Bitbucket Pipelines configuration file bitbucket-pipelines.yml contains the definition of the build steps, including extracting the cucumber specification from Xray, running the automated tests, and submitting back the results.

# This is a sample build configuration for Ruby.
# Check our guides at https://confluence.atlassian.com/x/8r-5Mw for more examples.
# Only use spaces to indent your .yml configuration.
# -----
# You can specify a custom docker image from Docker Hub as your build environment.
image: ruby:2.5

pipelines:
  default:
    - step:
        caches:
          - bundler
        script: # Modify the commands below to build your repository.
          - |
              apt-get update -qq
              apt-get install unzip
              gem install cucumber
              gem install rspec-expectations
              export token=$(curl -H "Content-Type: application/json" -X POST --data "{ \"client_id\": \"$client_id\",\"client_secret\": \"$client_secret\" }" https://xray.cloud.getxray.app/api/v1/authenticate| tr -d '"')
              curl -H "Content-Type: application/json" --output features/features.zip -X GET -H "Authorization: Bearer ${token}"  "https://xray.cloud.getxray.app/api/v2/export/cucumber?keys=$cucumber_keys"
              rm -f features/*.feature
              unzip -o features/features.zip -d features/
              cucumber -x -f json -o data.json
              curl -H "Content-Type: application/json" -X POST -H "Authorization: Bearer ${token}" --data @data.json https://xray.cloud.getxray.app/api/v1/import/execution/cucumber
              echo "done"

definitions:
  caches:
    bundler: ./vendor

In this example, we're using a variable cucumber_keys defined in the Repository variables (within Pipeline configuration section) in Bitbucket. This variable contains one or more keys of the issues that will be used as source data for generating the Cucumber .feature files; it can be the key(s) of Test Plan(s), Test Execution(s), Test(s), requirement(s). For more info, please see: Exporting Cucumber Tests - REST.

Triggering automation/builds from within Xray / Jira

Please have a look at Taking advantage of Jira Cloud built-in automation capabilities to know how to implement this.

Related articles

• Test automation of Jira Cloud workflows using Bitbucket pipelines and Xray Test Management