• Xray Cloud vs Xray server/datacenter
  • Am I using Jira server/datancer or Jira Cloud?
• Xray APIs
  • REST API
    • v1 vs v2
  • GraphQL
• Authentication
  • Jira server/datacenter
    • basic authentication (username + password)
    • Personal Access tokens
  • Jira Cloud
    • API keys (i.e. Client Id + Client Secret pair)

Overview

If you aim to interact with Xray data, there are a bunch of APIs you can use for different purpose.

But first you need to answer:

1. Am I using Xray on Jira server/datacenter or Xray on Jira Cloud? This is crucial as these are different products, using different technologies, providing slightly different APIs.
2. What do I aim to do? This is important to select the proper API to invoke; the format of the API call may be different even if there is a similar API endpoint for the other deployment type.
3. Which authentication mechanism do I want to use on the API calls? This is mostly dependent on the previous answers.2

This page will help you out answer these questions.

Xray server/DC vs Xray Cloud

Although similar, Xray for Jira server/Data Center(DC) and Xray for Jira Cloud are different products, essentially because they are built on top of different infrastructures and application capabilities. Jira server/datacenter and Jira Cloud are distinct products, with different roadmaps, built using distinct technologies, and providing different APIs. This has a consequence that apps for Jira server/DC and for Jira Cloud are essentially totally different from an architecture standpoint, but eventually also from a feature perspective.

Jira server/DC provides certain customizations capabilities, which may not be available on Jira Cloud.

From an integration standpoint, there will always be changes if a 3rd party wishes to integrate with both Xray server/DC and Xray Cloud. However, these can be subtle, depending on the flow to be implemented.

Available APIs

Xray Cloud 

Xray server/DC

Use cases

Importing test automation results to Xray

Importing automation results into Xray is a very common use case.

At high-level, the flow can be depicted in the following diagram.

There are some initial steps that need to be performed on the tool prior to importing results. The creation of the Test issue in Xray, could be done at any moment prior to importing results(manually or via the API).

The recommended way for importing results is by using the Xray JSON format and one of the respective endpoints (server/DC, cloud), which is quite flexible and very straightforward if you wish just have overall visibility of the test result, for example. 

There are also other well-known formats, each one providing different capabilities (server/DC, cloud).

Examples

Scenario1: report test automation results to existing Tests in Xray

In this case we're assuming:

• Test issue(s) already exist in Xray (they could have been created as usual by a user or by some automation process)
• An automated test equivalent exist in the external tool, for each Test

And we want to...

• report test automation result back to the existent Test issue(s) 

The following request shows how to import results for two tests, identified by their issue keys. The first one failed while the second passed.

Both tests were performed against version 1.0 of the SUT, using the Chrome browser; since it is relevant for us to analyze the results later on from each browser perspective, we map the browser name to a Test Environment.

On the failed test, we leave a comment on the Test Run and also attached a screenshot as evidence.

Xray server/DC

curl -H "Content-Type: application/json" -X POST -u jira_username:jira_password --data @payload.json https://jiraserver.example.com/rest/raven/2.0/import/execution

{
    "info": {
        "summary": "Execution of automated tests for release",
        "description": "This execution was automatically created when importing execution results from an external source",
        "project": "BOOK",
        "version": "1.0",
        "revision": "123",
        "startDate": "2021-07-14T11:47:35+01:00",
        "finishDate": "2021-07-14T11:53:00+01:00",
        "testEnvironments": [
            "chrome"
        ]
    },
    "tests": [
        {
            "testKey": "BOOK-429",
        	"start": "2021-07-14T11:47:35+01:00",
        	"finish": "2021-07-14T11:53:00+01:00",
            "comment": "invalid user",
            "status": "FAIL",
            "evidences" : [
              {
               "data": "/9j/4AAQSkZJRgABAQAAAQABAAD/2wCEAAkG...."
               "filename": "screenshot.jpg",
               "contentType": "image/jpeg"
               }
            ]
        },
        {
            "testKey": "BOOK-458",
        	"start": "2021-07-14T11:47:35+01:00",
        	"finish": "2021-07-14T11:53:00+01:00",
            "status": "PASS"
        }
    ]
}

After importing results, a Test Execution would be created containing the results for the two tests; the details of the execution screen show the data of the Test Run, including its status, comment, timings, among other.

Xray Cloud

token=$(curl -H "Content-Type: application/json" -X POST --data '{ "client_id": "32A27E69B0AC4E539C14016437000000","client_secret": "d62f81eb9ed859e11e54356dd8a00e4a5f0d0c2a2b52340776f6c7d6d7000000" }'  https://xray.cloud.getxray.app/api/v2/authenticate)

curl -H "Content-Type: application/json" -X POST -H "Authorization: Bearer $token"  --data @"payload.json" "https://xray.cloud.getxray.app/api/v2/import/execution

{
    "info": {
        "summary": "Execution of automated tests for release",
        "description": "This execution was automatically created when importing execution results from an external source",
        "project": "BOOK",
        "version": "1.0",
        "revision": "123",
        "startDate": "2021-07-14T11:47:35+01:00",
        "finishDate": "2021-07-14T11:53:00+01:00",
        "testEnvironments": [
            "chrome"
        ]
    },
    "tests": [
        {
            "testKey": "BOOK-70",
        	"start": "2021-07-14T11:47:35+01:00",
        	"finish": "2021-07-14T11:53:00+01:00",
            "comment": "invalid user",
            "status": "FAILED",
            "evidences" : [
              {
               "data": "/9j/4AAQSkZJRgABAQAAAQABAAD/2wCEAAkG...."
               "filename": "screenshot.jpg",
               "contentType": "image/jpeg"
                        }
                    ]
        },
        {
            "testKey": "BOOK-71",
        	"start": "2021-07-14T11:47:35+01:00",
        	"finish": "2021-07-14T11:53:00+01:00",
            "status": "PASSED"
        }
    ]
}

After importing results, a Test Execution would be created containing the results for the two tests; the details of the execution screen show the data of the Test Run, including its status, comment, timings, among other.

References

• Xray server/DC and Jira server/DC
  • Xray server/DC main documentation site
  • APIs
    • Jira server/DC REST API
    • Xray server/DC REST API
• Xray Cloud and Jira Cloud
  • Xray Cloud main documentation site
  • API's
    • Jira Cloud REST API 
    • Xray Cloud REST API
    • Xray Cloud GraphQL API
• Xray Code Snippets (open-source project with code samples, using different languages)
• Xray Postman collections (Postman collections showcasing usage of APIs both for Xray server/DC and Xray Cloud, including REST and GraphQL examples)
• Other useful resources
  • RestMan (a Chrome extension very useful to try out REST API call)
  • Postman (API client with good support for REST APIs)
  • Insomnia (API client that provides good support for GraphQL)