Overview

If you aim to interact with Xray data, there are a bunch of APIs you can use for different purposes.

But first you need to answer:

  1. Am I using Xray on Jira server/datacenter or Xray on Jira Cloud? This is crucial as these are different products, using different technologies, providing slightly different APIs.
  2. What do I aim to do? This is important to select the proper API to invoke; the format of the API call may be different even if there is a similar API endpoint for the other deployment type.
  3. Which authentication mechanism do I want to use on the API calls? This is mostly dependent on the previous answers.2


This page will help you out answer these questions.

Xray server/DC vs Xray Cloud

Although similar, Xray for Jira server/Data Center(DC) and Xray for Jira Cloud are different products, essentially because they are built on top of different infrastructures and application capabilities. Jira server/datacenter and Jira Cloud are distinct products, with different roadmaps, built using distinct technologies, and providing different APIs. This has a consequence that apps for Jira server/DC and for Jira Cloud are essentially totally different from an architecture standpoint, but eventually also from a feature perspective.

The Xray Cloud product closely integrates with Jira Cloud; each one uses their own infrastructure, the first managed by Xray and the later managed by Atlassian.

Xray server/DC works on top of (i.e., deployed into) Jira server/DC, reusing the Jira server/DC infrastructure.



So, you only know that you're using Jira or Xray but you don't know exactly which "flavour" (i.e., deployment type) you're using.

First, you can ask your Jira administrator. Besides, you can easily check for yourself.


Jira Cloud and Xray for Jira Cloud

If you see a top menu like the following one, having on the top-left corner to switch between Atlassian apps, followed by "Your work", "Filters", ... "Apps", then you are using a Jira Cloud instance.

If you click on a project that is using Xray, then the project left side menu will show you a "Testing Board" shortcut (and no Xray Reports, Xray Test Repository, Xray Test Plan Board, or Automated Steps Library). This telld you are using Xray on a Jira Cloud instance (i.e., "Xray Cloud").


Jira server/datacenter and Xray for Jira server/datacenter

If you see a top menu like the following one, having on the top-left corner an icon just to go some Jira page (depending on your profile settings),  followed by "Dashboard", "Projects", "Issues", ...,  and then "Tests", then you are using a Jira server/datacenter instance. The "Tests" top menu shortcut only appears if you have Xray (for Jira server/datacenter) installed.

If you click on a project that is using Xray, then the project left side menu will show you some options from Xray. Xray Reports, Xray Test Repository, Xray Test Plan Board, and Automated Steps Library are great tips that tell you are using Xray on a Jira server/datacenter instance.


Available APIs

There are several APIs that users can take advantage of.

Most users will the REST API provide by Xray to import test automation results during CI/CD. Please have in mind that, in this case, Xray Cloud's REST API is a different API than the Xray server/DC' REST API.

Xray Cloud 


API provided bytype of APIversionsURL syntaxauthenticationpurposenotes
1XrayREST APIv1, v2

https://xray.cloud.getxray.app/api/v1/<resource_name>

https://xray.cloud.getxray.app/api/v2/<resource_name>


managed by Xray Cloud

  • API key (i.e., pair of Client Id + Client Secret created by Jira administrator)
  • authentication
  • importing test results
  • importing tests
  • import/exporting Cucumber scenarios
  • backups
This is the most used API, as it used for importing test automation results.
2XrayGraphQLv2https://xray.cloud.getxray.app/api/v2/graphql

managed by Xray Cloud

  • API key (i.e., pair of Client Id + Client Secret created by Jira administrator)


Note: to make GraphQL requests, an initial REST API call needs to be made to the authentication endpoint

  • CRUD operations for Xray entities, with access to all data, including creation of tests
  • obtain Xray entities and other entities related to them, and manage these associations
  • export test results

This API cannot be used to import test automation results.

This API is more advanced and usually only required whenever implementing some custom scenarios.

3JiraREST APIv3

https://<site_url>/rest/api/3/<resource_name>


Note: <site_url> can be something like <xxx>.atlassian.net or it can be your own domain, in case you have it defined for your Jira cloud instance.

managed by Jira Cloud

  • basic authentication, OAuth 2.0
  • standard CRUD operations on Jira entities, including Jira issues (which includes Xray issue based entities)

This API does not provide ways of accessing/modifying internal Xray data, such as test steps, for example.

Xray server/DC



API provided bytype of APIversionsURL syntaxauthenticationpurposenotes
1XrayREST APIv1.0, v2.0

<jira_base_url>/rest/raven/1.0/api/<resource_name>

<jira_base_url>/rest/raven/2.0/api/<resource_name>

managed by Jira server/DC

  • importing test results
  • import/exporting Cucumber scenarios
This is the most used API, as it used for importing test automation results.
2JiraREST APIlatest<jira_base_url>/rest/api/latest/<resource_name>

managed by Jira server/DC

  • standard CRUD operations on Jira entities, including Jira issues (which includes Xray issue based entities)
This API does not provide ways of accessing/modifying internal Xray data, such as test steps, for example.
  • Xray server/DC doesn't provide a GraphQL API
  • Xray server/DC REST API is different from Xray Cloud REST API; some endpoints may be similar, like for importing test automation results, but there will be differences
  • Xray server/DC doesn't provide API Keys (pair client id + client secret), as it happens with Xray Cloud. Please check instead Personal Access Tokens (since v8.14, for Jira DC only)

The REST API v2.0 for Xray server/DC is an extension to the original v1.0 of the REST API. Therefore, all existing endpoints on v1.0 also exist on v2.0 (even if you don't see that in the docs); you just need to replace the "1.0" by "2.0" on the URL.

Example:

  <jira_base_url>/rest/raven/1.0/api/test/{key}/testexecutions   becomes   <jira_base_url>/rest/raven/2.0/api/test/{key}/testexecutions

Use cases

Importing test automation results to Xray

Importing automation results into Xray is a very common use case.

At high-level, the flow can be depicted in the following diagram.


There are some initial steps that need to be performed on the tool prior to importing results. The creation of the Test issue in Xray, could be done at any moment prior to importing results(manually or via the API).

The recommended way for importing results is by using the Xray JSON format and one of the respective endpoints (server/DC, cloud), which is quite flexible and very straightforward if you wish just have overall visibility of the test result, for example. 

There are also other well-known formats, each one providing different capabilities (server/DC, cloud).


Xray JSON format has the same syntax but to the distinct nature of Jira server/DC and Jira Cloud (and also Xray), there are some nuances.



Xray Server/DCXray CloudNotes
values for "status" attribute, for each "test" and "step" elements"PASS", "FAIL", "TODO", "EXECUTING""PASSED", "FAILED", "TO DO", "EXECUTING"
"user" attributeusername of Jira user

name of Jira user.

Usually, you don't need to specify the user as by default it is assigned to the user related to the given credentials. 

With Xray Cloud, you can use the "userId" attribute instead and in that case you need to use the hash of the user (you can see it as the ending part of the URL of the user profile)


Examples

Scenario1: report test automation results to existing Tests in Xray

In this case we're assuming:

And we want to...


The following request shows how to import results for two tests, identified by their issue keys. The first one failed while the second passed.

Both tests were performed against version 1.0 of the SUT, using the Chrome browser; since it is relevant for us to analyze the results later on from each browser perspective, we map the browser name to a Test Environment.

On the failed test, we leave a comment on the Test Run and also attached a screenshot as evidence.


Xray server/DC


curl -H "Content-Type: application/json" -X POST -u jira_username:jira_password --data @payload.json https://jiraserver.example.com/rest/raven/2.0/import/execution
{
    "info": {
        "summary": "Execution of automated tests for release",
        "description": "This execution was automatically created when importing execution results from an external source",
        "project": "BOOK",
        "version": "1.0",
        "revision": "123",
        "startDate": "2021-07-14T11:47:35+01:00",
        "finishDate": "2021-07-14T11:53:00+01:00",
        "testEnvironments": [
            "chrome"
        ]
    },
    "tests": [
        {
            "testKey": "BOOK-429",
        	"start": "2021-07-14T11:47:35+01:00",
        	"finish": "2021-07-14T11:53:00+01:00",
            "comment": "invalid user",
            "status": "FAIL",
            "evidences" : [
              {
               "data": "/9j/4AAQSkZJRgABAQAAAQABAAD/2wCEAAkG...."
               "filename": "screenshot.jpg",
               "contentType": "image/jpeg"
               }
            ]
        },
        {
            "testKey": "BOOK-458",
        	"start": "2021-07-14T11:47:35+01:00",
        	"finish": "2021-07-14T11:53:00+01:00",
            "status": "PASS"
        }
    ]
}


After importing results, a Test Execution would be created containing the results for the two tests; the details of the execution screen show the data of the Test Run, including its status, comment, timings, among other.


Xray Cloud


token=$(curl -H "Content-Type: application/json" -X POST --data '{ "client_id": "32A27E69B0AC4E539C14016437000000","client_secret": "d62f81eb9ed859e11e54356dd8a00e4a5f0d0c2a2b52340776f6c7d6d7000000" }'  https://xray.cloud.getxray.app/api/v2/authenticate)

curl -H "Content-Type: application/json" -X POST -H "Authorization: Bearer $token"  --data @"payload.json" "https://xray.cloud.getxray.app/api/v2/import/execution
{
    "info": {
        "summary": "Execution of automated tests for release",
        "description": "This execution was automatically created when importing execution results from an external source",
        "project": "BOOK",
        "version": "1.0",
        "revision": "123",
        "startDate": "2021-07-14T11:47:35+01:00",
        "finishDate": "2021-07-14T11:53:00+01:00",
        "testEnvironments": [
            "chrome"
        ]
    },
    "tests": [
        {
            "testKey": "BOOK-70",
        	"start": "2021-07-14T11:47:35+01:00",
        	"finish": "2021-07-14T11:53:00+01:00",
            "comment": "invalid user",
            "status": "FAILED",
            "evidences" : [
              {
               "data": "/9j/4AAQSkZJRgABAQAAAQABAAD/2wCEAAkG...."
               "filename": "screenshot.jpg",
               "contentType": "image/jpeg"
                        }
                    ]
        },
        {
            "testKey": "BOOK-71",
        	"start": "2021-07-14T11:47:35+01:00",
        	"finish": "2021-07-14T11:53:00+01:00",
            "status": "PASSED"
        }
    ]
}


After importing results, a Test Execution would be created containing the results for the two tests; the details of the execution screen show the data of the Test Run, including its status, comment, timings, among other.


References