Overview

If you aim to interact with Xray data, directly or indirectly, there are several APIs you can use for different purposes.

But first, you need to answer:

1. Am I using Xray on Jira Server/Datacenter or Xray on Jira Cloud? This is crucial as these are different products, using different technologies, providing slightly different APIs.
2. What do I aim to do? This is important in order to select the proper API to invoke; the format of the API call may be different even if there are similar API endpoints for the other deployment type.
3. Which authentication mechanism do I want to use on the API calls? This is mostly dependent on the previous answers.

This page helps you answer these questions.

In the end, you'll find all relevant references with links to the proper documentation.

Xray Server/DC vs Xray Cloud

Although similar, Xray for Jira server/Data Center(DC) and Xray for Jira Cloud are different products, essentially because they are built on top of different infrastructures and application capabilities. Jira Server/Datacenter and Jira Cloud are also distinct products, with different roadmaps, built using distinct technologies, and providing different APIs. This has a consequence that apps for Jira Server/DC and for Jira Cloud are essentially totally different from an architecture standpoint, but eventually also from a feature perspective.

The Xray Cloud product closely integrates with Jira Cloud; each one uses its own infrastructure, the first managed by Xray and the latter managed by Atlassian.

Xray server/DC works on top of (i.e., deployed into) Jira server/DC, reusing the Jira server/DC infrastructure.

Now that you know the Jira "flavor" (i.e., deployment type) being used and the corresponding Xray product variant (whether Xray Cloud or Xray server/DC), let's see which APIs are available in that context, how they work and for that, and what they can be used for.

Available APIs

There are several APIs that users can take advantage of.

APIs can either be provided by Xray or by Jira itself. 

Most users will use the REST API provided by Xray to import test automation results during CI/CD. Please have in mind that, in this case, Xray Cloud's REST API is a different API than the Xray Server/DC REST API.

The Jira REST API is only used in case you want to perform operations on Jira data, such as standard CRUD operations (Create, Read, Update, Delete) on issues of, for example, search for issues using JQL. Jira Cloud and Jira Server/DC have similar (not equal) REST APIs.

APIs available in Xray Cloud context

Xray Cloud provides a REST API and also a, more advanced, GraphQL API.

Besides, there is also Jira's REST API in case you aim to interact with Jira itself and, for example, do CRUD operations on issues (except for testing data).

API available in Xray server/DC context

Xray Server/DC only provides a REST API, used to import test automation results and also to obtain and manage relations between Xray entities.

Besides, there is also Jira's REST API in case you aim to interact with Jira itself and, for example, do CRUD operations on issues (except for testing data).

Use cases

Importing test automation results to Xray using Xray JSON format

Importing automation results into Xray is a very common use case.

There are a bunch of test report formats supported (e.g., JUnit XML, Xray JSON, etc), which may be slightly different for Xray Cloud and for Xray server/DC. Depending on the report format and whether you're using Xray Cloud or server/DC, different capabilities will be available (server/DC, cloud).

In this case, we'll use the Xray JSON format as means to encapsulate the test results.

At a high level, the flow can be depicted in the following diagram.

In this case, we're assuming:

• Test issue(s) already exist in Xray (they could have been created as usual by a user or by some automation process)
• An automated test equivalent exists in the external tool/code, for each Test

And we want to...

• report test automation result back to the existent Test issue(s) 

The following request shows how to import results for two tests, identified by their issue keys. The first one failed while the second passed.

Both tests were performed against version 1.0 of the SUT, using the Chrome browser; since it is relevant for us to analyze the results later on from each browser perspective, we map the browser name to a Test Environment.

On the failed test, we leave a comment on the Test Run and also attached a screenshot as evidence.

Xray Server/DC

curl -H "Content-Type: application/json" -X POST -u jira_username:jira_password --data @payload.json https://jiraserver.example.com/rest/raven/2.0/import/execution

{
    "info": {
        "summary": "Execution of automated tests for release",
        "description": "This execution was automatically created when importing execution results from an external source",
        "project": "BOOK",
        "version": "1.0",
        "revision": "123",
        "startDate": "2021-07-14T11:47:35+01:00",
        "finishDate": "2021-07-14T11:53:00+01:00",
        "testEnvironments": [
            "chrome"
        ]
    },
    "tests": [
        {
            "testKey": "BOOK-429",
        	"start": "2021-07-14T11:47:35+01:00",
        	"finish": "2021-07-14T11:53:00+01:00",
            "comment": "invalid user",
            "status": "FAIL",
            "evidences" : [
              {
               "data": "/9j/4AAQSkZJRgABAQAAAQABAAD/2wCEAAkG...."
               "filename": "screenshot.jpg",
               "contentType": "image/jpeg"
               }
            ]
        },
        {
            "testKey": "BOOK-458",
        	"start": "2021-07-14T11:47:35+01:00",
        	"finish": "2021-07-14T11:53:00+01:00",
            "status": "PASS"
        }
    ]
}

After importing results, a Test Execution would be created containing the results for the two tests; the details of the execution screen show the data of the Test Run, including its status, comment, timings, among others.

Xray Cloud

The first thing to do would be to obtain a token, using the Client Id and Client Secret from the corresponding API key on Xray. 

This would require performing an HTTP request to the authentication REST API endpoint from Xray Cloud.

Then a second request could be made to upload the test automation results; in the following example, the Xray JSON is stored on a payload.json file.

token=$(curl -H "Content-Type: application/json" -X POST --data '{ "client_id": "32A27E69B0AC4E539C14016437000000","client_secret": "d62f81eb9ed859e11e54356dd8a00e4a5f0d0c2a2b52340776f6c7d6d7000000" }'  https://xray.cloud.getxray.app/api/v2/authenticate)

curl -H "Content-Type: application/json" -X POST -H "Authorization: Bearer $token"  --data @"payload.json" "https://xray.cloud.getxray.app/api/v2/import/execution

{
    "info": {
        "summary": "Execution of automated tests for release",
        "description": "This execution was automatically created when importing execution results from an external source",
        "project": "BOOK",
        "version": "1.0",
        "revision": "123",
        "startDate": "2021-07-14T11:47:35+01:00",
        "finishDate": "2021-07-14T11:53:00+01:00",
        "testEnvironments": [
            "chrome"
        ]
    },
    "tests": [
        {
            "testKey": "BOOK-70",
        	"start": "2021-07-14T11:47:35+01:00",
        	"finish": "2021-07-14T11:53:00+01:00",
            "comment": "invalid user",
            "status": "FAILED",
            "evidences" : [
              {
               "data": "/9j/4AAQSkZJRgABAQAAAQABAAD/2wCEAAkG...."
               "filename": "screenshot.jpg",
               "contentType": "image/jpeg"
                        }
                    ]
        },
        {
            "testKey": "BOOK-71",
        	"start": "2021-07-14T11:47:35+01:00",
        	"finish": "2021-07-14T11:53:00+01:00",
            "status": "PASSED"
        }
    ]
}

After importing results, a Test Execution would be created containing the results for the two tests; the details of the execution screen show the data of the Test Run, including its status, comment, timings, among others.

Importing test automation results to Xray using other formats (non-Xray JSON)

Besides the Xray JSON format as described in the previous scenario, Xray also can import and process test automation results from reports stored in different formats (e.g., JUnit XML, NUnit XML, Cucumber JSON, etc).

This will make use of Xray Cloud REST API or Xray server/DC REST API.

In this case, what's important to have in mind is that:

• the supported test automation report formats may be different between Xray Cloud and Xray Server/DC
• the authentication mechanism is different in Xray Cloud and on Xray Server/DC
• HTTP POST request for the REST API call is slightly different for Xray Cloud and Xray Server/DC

Therefore, make sure you are using the proper documentation of the REST API, either Xray Cloud REST API or the Xray server/DC REST API one.

There's an open-source repository on GitHub with some code samples, in different languages, that you can check out 

Note: the import of Cucumber or Behave JSON reports follows a different flow than the standard one. More info on that here.

References

• Xray server/DC and Jira server/DC
  • Xray server/DC main documentation site
  • APIs
    • Jira server/DC REST API
    • Xray server/DC REST API
• Xray Cloud and Jira Cloud
  • Xray Cloud main documentation site
  • API's
    • Jira Cloud REST API 
    • Xray Cloud REST API
    • Xray Cloud GraphQL API
• Xray Code Snippets (open-source project with code samples, using different languages)
• Xray Postman collections (Postman collections showcasing usage of APIs both for Xray server/DC and Xray Cloud, including REST and GraphQL examples)
• Other useful resources
  • RestMan (a Chrome extension very useful to try out REST API call)
  • Postman (API client with good support for REST APIs)
  • Insomnia (API client that provides good support for GraphQL)