Overview

This document provides general guidance on how integration with Xray can be achieved.

Not all possible usage scenarios are covered in this document, as 3rd party tools have different capabilities and different usage usage patterns.

Please have in mind that Xray is a product that is continuously evolving, so you should look at the available capabilities periodically.

Another important aspects is that Xray Server/Data Center(DC) and Xray Cloud are distinct products, with different capabilities (additional info ahead). Specific information for each product will be highlighted using the notation (server/DC, cloud), with respective links.

If your goal is to support both Xray on Jira server/DC and Jira Cloud, you'll have to make two different implementations; the differences may be subtle.

Some questions to answer before implementing the integration:

What are the use cases for the integration and corresponding flows? Where will entities will be created? Can we easily navigate/jump between tools, to the right context?
What information do I want to show in Xray? How valuable is it from the Xray user perspective? Can we even make it more valuable?

Background

Xray concepts

In Xray, testing concepts are implemented either as issues or as internal entities.

Not all concepts are, or need to be, used by teams. Some concepts may address similar purposes (e.g. Test Set and Test Repository) but have distinct capabilities.

Many of Xray core entities are standard Jira issue types. Therefore they can be managed as any other Jira issue and standard Jira features are also available for them; more info here.

Issue Type	Description/Purpose
Test	An abstraction of a test idea, usually a test case. It may be composed of multiple steps, actions and expected results or it may be a gherkin scenario or even just have an abstract definition (e.g. a charter). Usually a Test is linked to a requirement/user story.
Pre-Condition	Initial conditions that must be assured before the execution of the Test's own steps. A Pre-Condition may be used by multiple Tests.
Test Set	A group of tests, organized in some logical way. A flat list of tests that doesn't contain results.
Test Plan	A "formal" plan of the tests intended to be executed for a given version or sprint. Used for grouping multiple Test Executions and presenting a consolidated overview of them; tracks the results of some Tests in some version/sprint of the System Under Test (SUT).
Test Execution	An assignable, "schedulable" task to execute one or more tests for a given version/revision along with its results. A Test Execution can be linked to a Test Plan, and if so results will be visible on that Test Plan. A Test Execution may have a Version (i.e. the Fix Version of the underlying Jira project being targeted by testing) Revision (i.e. an open text field used for identifying the code revision, or the build number) Test Environment (i.e. where was it executed?) Test Plan (i.e. is this part of a plan?)
Sub-Test Execution	A Sub-Test Execution has the same functionalities as the Test Execution issue type. The difference between them is that the Sub-Test Execution is a sub-Task and can be created in a requirement's context. Creating a Test Execution as a sub-task from within the requirement issue screen will provide the user the ability to track executions in the Agile board. A Sub-Test Execution has also the same attributes as a Test Execution.

A Test is an abstraction of a test idea or experiment. Exactly what it is, depends on its type (which ultimately depends on its kind).

Xray cloud provides management capabilities (i.e. creating new test types of a specific kind) for test types; Xray server/DC doesn't.

Xray server/DC and Xray cloud provide 3 test types by default: Manual, Cucumber and Generic. In Xray Cloud, these types are managed and each one belongs to a specific kind: structured, gherkin, and unstructured.

These default test types and their names exist for legacy reasons but to clarify their intended usage please see following table.

Test type

Description

Manual

(i.e., structured/step-based)

A test composed of well-defined steps, having actions and expected results.

Usually, executed manually. Sometimes some automation tools use also these tests when they want to "automate" existing tests or have information similar to the one that exists for standard step-based test cases.

Some teams refer to traditional test cases as "manual tests", and thus the reason for its original name. However, "Manual" is not totally accurate and may be a bit misleading.

Cucumber

(i.e., Gherkin)

A testing scenario described in natural language using Gherkin (i.e. Given/When/That). These can be Cucumber, SpecFlow, Behave, or similar.

Usually executed by test automation; can be manually executed (few teams do that).

"Generic"

(i.e., unstructured)

A test that has no structure; it has just an open definition field.

This is the simplest abstraction of a test.

These tests are used in two distinct use cases:

for exploratory testing, where the definition can be used to specify the charter/goal, risks, etc
for test automation, to abstract some automated tests implemented elsewhere by some test automation tool/framework. In that case, the generic field has a unique identifier for the test (e.g. classname+method) so the results are always reported against the same Test entity

Automated test?

In Xray, there is no clear identification that says whether a certain Test is manual/automated or whether a related run for that Test was performed using automation or was executed manually instead.

Even if there is the "Manual" test type, as mentioned earlier these tests may be used as an abstraction of some test automation script composed of steps.

An external tool doesn't need to explicitly identify that a Test is automated; if needed, a label may eventually be added in the Test issue for that purpose or even a specific Test Type (Xray Cloud only).

Some entities/concepts are internal to Xray are therefore not Jira issue types. The Test Run is an example of one.

Other Entities	Description/Purpose
Test Run	an instance of a Test in the context of some Test Execution; each time a Test is run, a Test Run is created for that Test. contains a copy of the original Test specification along with the recorded results it contains information about the version, "revision" and the Test Environment
Test Repository	an alternate approach to Test Sets; a per project hierarchical organization of Test cases using folders.
Test Plan Board	a per Test Plan hierarchical organization of Test cases, at the planning phase, using folders; each Test Plan has implicitly one Board used for grouping, organizing the tests in the context of the Test Plan and to easily track the results of certain Tests grouped in some folder; used also for easily changing the ranking of the Tests, to create Test Executions for them afterwards.
Test Environment	An attribute of a Test Execution that identifies where testing was performed. Depending on your context, a test environment may represent: a testing stage (e.g. "development", "staging", "preproduction", "production") browser (e.g. "edge", "chrome", "firefox") or a device model or device operating system (e.g. "android", "ios") More info here.

Question

Do you need to use all these concepts?

No. Most times you will use Tests, Test Executions and the related Test Runs, but it all depends on the use case.

Should we create the entity in Xray from our tool or should that entity be created and managed in Xray?

It depends on the scenario.

Tests are usually created in Xray and linked to requirements/user stories; however, Tests may also be provisioned in Xray at the moment of importing test automation results, for example.

We have to consider the use case for end users.

Xray server/DC vs Xray Cloud

Although similar, Xray for Jira server/Data Center(DC) and Xray for Jira Cloud are different products, essentially because they are built on top of different infrastructures and application capabilities. Jira server/datacenter and Jira Cloud are distinct products, with different roadmaps, built using distinct technologies, and providing different APIs. This has a consequence that apps for Jira server/DC and for Jira Cloud are essentially totally different from an architecture standpoint, but eventually also from a feature perspective.

Jira server/DC provides certain customizations capabilities, which may not be available on Jira Cloud.

From an integration standpoint, there will always be changes if a 3rd party wishes to integrate with both Xray server/DC and Xray Cloud. However, these can be subtle, depending on the flow to be implemented.

Preparing a environment with Xray to test the integration

To test out the integration with Xray, first of all you'll need access to a Jira server/DC instance or a Jira Cloud instance. The instance needs to have the respective Xray product installed and configured, which also requires having an Xray "enabled" project to play around.

What do you mean with "enable" Xray in a project?

Xray doesn't provide a "enable" kind of action for projects. You should read it as "configured" instead.

Xray is a flexible tool, so you can "enable" it in different ways. Most times, this ultimately is materialized as adding the Xray issue types to the project and enable coverage on the project.

For Jira server/DC:

you can use a public sandbox environment to start immediately; note that this environment is shared with other users and is reset weekly, on Sundays. This environment already has Jira and Xray installed and it contain a sample project called BOOK, with some existing test data. You'll need to create an account in this instance.
- https://sandbox.getxray.app
to install and configure Xray on your local Jira environment ,
- see Installing instructions. A trial version is fully functional for a 1 month period which may be extended
- see Quick Setup instructions, to create a Xray enabled project; you can create a standard project and then add Xray issue types and enable coverage for the project

For Jira Cloud:

to install and configure Xray on your local Jira environment ,
- see Installing instructions
- see Quick Setup instructions, to create an Xray enabled project. You can create a Scrum project, for example, and then follow the previous instructions.
  - in Jira Cloud there are company and team managed projects, which are quite different. Xray supports both but please make sure you follow the right setup instructions

Integration possibilities

Xray server/DC

The most common way of integrating with Xray server/DC is by using the Xray REST API.

	Usage scenarios	Notes
Jira REST API	create, update and search for Jira issues, including Test, Precondition, Test Set, Test Execution, Test Plan issues	Creating and modifying issues using Jira's REST API is limited to typical fields; usually, Xray internal data cannot be modified this way.
Xray REST API	import test automation results manage associations between Xray entities (e.g. Tests that belong to a Test Plan) export test runs get/modify details on Test Runs import/export Cucumber scenarios from/to .feature files	The most common and preferred to integrate with Xray.
Custom field on Xray issues		In Jira server/DC, custom fields may have custom logic; this usually requires the development of a custom Jira app/plugin.
Web panel on Xray issues		In Jira server/DC and Jira cloud it's possible to add web sections/panels on the screens of issues (i.e., view issue screen). This requires the development of a custom Jira app/plugin.

Jira's REST API and Xray REST API

Xray uses Jira issues for implementing Test, Pre-Condition, Test Set, Test Execution, and Test Plan entities. Therefore, these entities may be created or updated using Jira's native REST API, taking into consideration Xray's custom fields.

The Xray REST API provides additional endpoints specifically made for dealing with test entities/fields and importing execution results from automated processes.

Authentication

Jira 8.14 introduced the concept of Personal Access Tokens, which Xray takes advantage of. These tokens can be created in the user's profile section in Jira and have an expiration date; they can also be revoked at any moment (more information here).

To use them in Jira's and in Xray REST API calls, we need to use the HTTP header "Authorization" with the "Bearer <token>" value.

example of a API request with "curl" using Personal Access Tokens

curl -H "Accept: application/json" -H "Authorization: Bearer MDkzNjIyMTEwNjQzOqb+ApTGm3af+z3eGdcyDm7xHv6R" https://jiraserver.example.com/rest/raven/1.0/api/test/CALC-1880/step

You may also use HTTP basic authentication with some user credentials.

example of a API request with "curl" using basic authentication

curl -H "Accept: application/json" -u jira_username:jira_password https://jiraserver.example.com/rest/raven/1.0/api/test/CALC-1880/step

Xray Cloud

Xray Cloud provides a REST API to handle more heavy and time-consuming operations such as importing executions results, generating cucumber feature files and importing Tests.

Besides, there's a GraphQL API which allows users to perform CRUD operations directly on Xray entities. GraphQL gives clients the power to get exactly what they need and nothing more, in one call.

Both APIs use the same authentication mechanism as shown ahead.

Authentication

The authentication process uses a Client Id and a Client Secret, both of which are based on an API Key created for a specific user in Xray Global Settings: API Keys global settings.

Thus, the first step for you is to obtain a token based on the Client Id and Client Secret of your assigned API Key. You can then use any GraphQL or REST API end points.

Tokens are valid for 1 day.

example of authentication requests

# authentication request with client id and client secret passed inline
curl -H "Content-Type: application/json" -X POST --data '{ "client_id": "32A27E69B0AC4E539C14016437000000","client_secret": "d62f81eb9ed859e11e54356dd8a00e4a5f0d0c2a2b52340776f6c7d6d7000000" }'  https://xray.cloud.getxray.app/api/v2/authenticate

# or, if you have the client id/secret stored in a file (e.g. cloud_auth.json) having the content syntax as { "client_id": "xxx","client_secret": "xxx" }
curl -H "Content-Type: application/json" -X POST --data @"cloud_auth.json" https://xray.cloud.getxray.app/api/v2/authenticate

Typical use cases

Importing test automation results to Xray

Importing automation results into Xray is a very common use case.

At high-level, the flow can be depicted in the following diagram.

There are some initial steps that need to be performed on the tool prior to importing results. The creation of the Test issue in Xray, could be done at any moment prior to importing results(manually or via the API).

The recommended way for importing results is by using the Xray JSON format and one of the respective endpoints (server/DC, cloud), which is quite flexible and very straightforward if you wish just have overall visibility of the test result, for example.

There are also other well-known formats, each one providing different capabilities (server/DC, cloud).

Capability

Supported?

Main features

Import results for existing Tests

(based on the Test issue key)

Yes. All test types supported (e.g. Manual, Cucumber, Generic).

Global attributes:

overall status (e.g. PASS, FAIL, TODO); please see the notes ahead on the differences between server/DC vs cloud
start and finish timestamps
comment
evidence (i.e. attachments)

Create Tests from results

(create Tests whenever importing results)

Not yet (very soon).

Automatic linking to requirements

(create links to requirements while importing results)

No.

This linkage needs to be done by another process, usually by users in Jira. It can also be automated though, using Jira's REST API for creating issue links.

Detailed Results

Manual Tests

(i.e. structured/step-based)

Yes.

For each step, it's possible to report:

status (e.g. PASS, FAIL); please see the notes ahead on the differences between server/DC vs cloud
comment (this is a text field
evidence (i.e., attachments)
actual result (this is a text field)

Cucumber Tests

(i.e. gherkin)

Yes, however the Xray JSON format is not the best fit for this purpose; the native Cucumber JSON is recommended instead.

With Xray JSON it's possible to report:

status for each row of Scenario Outline Examples

Generic Tests

(i.e. unstructured)

Yes.

Attributes:

name
log (i.e., output)
duration (in microseconds)

Other alternatives

Another approach, since many tools provide it by default, is to import results using JUnit XML, which can also be used to auto-provision Tests issues in Xray, unless they already exist. However, this format lacks the ability to provide additional information that Xray can provide. So, even though it may work, the data is superficial and using the Xray JSON format is preferred.

Before implementing the integration, review the following questions to help guide your implementation approach.

Question	Possible answers	Tip
What kind of information do we need to see in Xray for each test result?	just the overall result (i.e. pass, fail) overall result, duration and/or logs overall result and automated step results, duration and/or logs the previous but also with screenshots during each automated step	Each Test Run has always these attributes: status (i.e. overall result) start and finish timestamps comment evidence (e.g. screenshots) Depending on the type of Test (i.e. Manual, Cucumber, Generic) additional fields may be available. Generic Usually, automated tests are abstracted using Generic (i.e., unstructured) Tests. For these, we may provide multiple inner results and also log/output for each item. These inner results are usually steps in the test automation code, where assertions are performed. Evidence (e.g. screenshots) at each step are not yet possible using the Xray JSON format.
Do we need to create the Test entities in Xray or will be reporting test results against existing Test issues?	No, we're just reporting results for existing Tests. Yes, we need to create Test issues in Xray and report results for it. Each Test will be created once, as the external tool will store a reference to the Test initially created and reuse that afterwards.	Usually, tools report results against existing Test issues in Xray.
Can we report test automation results against an existing "manual" test case?	Yes	The Xray JSON format allows you to specify results for "Manual" (i.e., structured) tests, including for each step.
Do I need to know the type of test whenever importing the results?	Yes, because the format for reporting results is different from test type to test type, due to their kind (i.e., whether they are composed of steps or not, or if they are gherkin)
Should we create defects (e.g.m Bug issues) automatically in Jira?		Usually no, as defects are mostly a consequence of analysis and a human decision.

Xray JSON format has the same syntax but to the distinct nature of Jira server/DC and Jira Cloud (and also Xray), there are some nuances.

Xray Server/DC

Xray Cloud

Notes

values for "status" attribute, for each "test" and "step" elements

"PASS", "FAIL", "TODO", "EXECUTING"

"PASSED", "FAILED", "TO DO", "EXECUTING"

"user" attribute

username of Jira user

name of Jira user.

Usually, you don't need to specify the user as by default it is assigned to the user related to the given credentials.

With Xray Cloud, you can use the "userId" attribute instead and in that case you need to use the hash of the user (you can see it as the ending part of the URL of the user profile)

Xray provides coverage analysis capabilities (server/DC, cloud) which the team uses to understand how a certain epic/requirement/story, or a bunch of those, are in some context.

Sometimes a team member wants to assess requirement status, on a given version, so they can have a quick insight about quality aspects of that story and decide whether it's ready to be delivered. Even though coverage is a heuristic, it's quite helpful to understand how a requirement or a whole release is.

Therefore, associating results to a Jira project version/release, is essential as it affects the state (i.e., the coverage) of requirements for that release.

Additionally, people may want to analyze coverage for a specific browser, mobile device, or testing stage (depending on what matters most to them). In this case, results need to be tagged somehow; this is done by setting the Test Environment attribute on the related Test Execution.

Another useful perspective for analysis is the Test Plan. Associating Test Executions to a Test Plan allows the team to always see aggregated results at the Test Plan level. Grouping results in one place helps teams analyze the status of various requirements/epics/stories and test coverage within a specific Test Plan. When combined with Test Environments, Test Plans gives a holistic view across Test Environments.

Associating results to a Test Plan, and eventually to a Test Environment, is recommended as it will facilitate analysis on Xray side. Test Plan usage in Xray is very common, so we should be able to link results to Test Plans whenever importing them.

Examples

Scenario1: report test automation results to existing Tests in Xray

In this case we're assuming:

Test issue(s) already exist in Xray (they could have been created as usual by a user or by some automation process)
An automated test equivalent exist in the external tool, for each Test

And we want to...

report test automation result back to the existent Test issue(s)

The following request shows how to import results for two tests, identified by their issue keys. The first one failed while the second passed.

Both tests were performed against version 1.0 of the SUT, using the Chrome browser; since it is relevant for us to analyze the results later on from each browser perspective, we map the browser name to a Test Environment.

On the failed test, we leave a comment on the Test Run and also attached a screenshot as evidence.

Xray server/DC

example of a API request with "curl" using basic authentication

curl -H "Content-Type: application/json" -X POST -u jira_username:jira_password --data @payload.json https://jiraserver.example.com/rest/raven/2.0/import/execution

JSON body payload

{
    "info": {
        "summary": "Execution of automated tests for release",
        "description": "This execution was automatically created when importing execution results from an external source",
        "project": "BOOK",
        "version": "1.0",
        "revision": "123",
        "startDate": "2021-07-14T11:47:35+01:00",
        "finishDate": "2021-07-14T11:53:00+01:00",
        "testEnvironments": [
            "chrome"
        ]
    },
    "tests": [
        {
            "testKey": "BOOK-429",
        	"start": "2021-07-14T11:47:35+01:00",
        	"finish": "2021-07-14T11:53:00+01:00",
            "comment": "invalid user",
            "status": "FAIL",
            "evidences" : [
              {
               "data": "/9j/4AAQSkZJRgABAQAAAQABAAD/2wCEAAkG...."
               "filename": "screenshot.jpg",
               "contentType": "image/jpeg"
               }
            ]
        },
        {
            "testKey": "BOOK-458",
        	"start": "2021-07-14T11:47:35+01:00",
        	"finish": "2021-07-14T11:53:00+01:00",
            "status": "PASS"
        }
    ]
}

After importing results, a Test Execution would be created containing the results for the two tests; the details of the execution screen show the data of the Test Run, including its status, comment, timings, among other.

Xray Cloud

example of a API request with "curl" using basic authentication

token=$(curl -H "Content-Type: application/json" -X POST --data '{ "client_id": "32A27E69B0AC4E539C14016437000000","client_secret": "d62f81eb9ed859e11e54356dd8a00e4a5f0d0c2a2b52340776f6c7d6d7000000" }'  https://xray.cloud.getxray.app/api/v2/authenticate)

curl -H "Content-Type: application/json" -X POST -H "Authorization: Bearer $token"  --data @"payload.json" "https://xray.cloud.getxray.app/api/v2/import/execution

JSON body payload

{
    "info": {
        "summary": "Execution of automated tests for release",
        "description": "This execution was automatically created when importing execution results from an external source",
        "project": "BOOK",
        "version": "1.0",
        "revision": "123",
        "startDate": "2021-07-14T11:47:35+01:00",
        "finishDate": "2021-07-14T11:53:00+01:00",
        "testEnvironments": [
            "chrome"
        ]
    },
    "tests": [
        {
            "testKey": "BOOK-70",
        	"start": "2021-07-14T11:47:35+01:00",
        	"finish": "2021-07-14T11:53:00+01:00",
            "comment": "invalid user",
            "status": "FAILED",
            "evidences" : [
              {
               "data": "/9j/4AAQSkZJRgABAQAAAQABAAD/2wCEAAkG...."
               "filename": "screenshot.jpg",
               "contentType": "image/jpeg"
                        }
                    ]
        },
        {
            "testKey": "BOOK-71",
        	"start": "2021-07-14T11:47:35+01:00",
        	"finish": "2021-07-14T11:53:00+01:00",
            "status": "PASSED"
        }
    ]
}

After importing results, a Test Execution would be created containing the results for the two tests; the details of the execution screen show the data of the Test Run, including its status, comment, timings, among other.

Scenario2: report test automation results to existing Tests in Xray, with additional step-like detail

This use case is very similar to the previous one; the main difference is that we aim to provide more detailed information on the results for each test, including having visibility of each "automated step/action/assertion".

In this case we're assuming:

Test issue(s) already exist in Xray (they could have been created as usual by a user or by some automation process)
An automated test equivalent exist in the external tool, for each Test

And we want to...

report test automation result back to the existent Test issue(s), where we want to see the step-level details (i.e. results for the assertions performed by the test automation script)

The following request shows how to import results for two tests, identified by their issue keys. The first one failed while the second passed.

Both tests were perfomed against version 1.0 of the SUT, using the Chrome browser; since it is relevant for us to analyze the results later on from each browser perspective, we map the browser name to a Test Environment.

On the failed test, we leave a comment on the Test Run and also attached a screenshot as evidence. On the Test Run we can see the results for all the steps/assertions performed by the automated test.

Xray server/DC

example of a API request with "curl" using basic authentication

curl -H "Content-Type: application/json" -X POST -u jira_username:jira_password --data @data.json https://jiraserver.example.com/rest/raven/2.0/import/execution

JSON body payload

{
    "info": {
        "summary": "Execution of automated tests for release",
        "description": "This execution was automatically created when importing execution results from an external source",
        "project": "BOOK",
        "version": "1.0",
        "revision": "1.0.42134",
        "startDate": "2021-07-14T11:47:35+01:00",
        "finishDate": "2021-07-14T11:53:00+01:00",
        "testEnvironments": [
            "chrome"
        ]
    },
    "tests": [
        {
        "testKey": "BOOK-429",
        "start": "2021-07-14T11:47:35+01:00",
        "finish": "2021-07-14T11:53:00+01:00",
        "comment": "failed execution",
        "status": "FAIL",
        "results": [
              {
                "name": "open homepage",
                "log": "all good so far",
                "duration": 2,
                "status": "PASS"
              },
              {
                "name": "enter username and password",
                "log": "",
                "duration": 3,
                "status": "PASS"
              },
              {
                "name": "login",
                "log": "invalid credentials",
                "duration": 123400,
                "status": "FAIL"
              }   
           ],
		"evidences" : [
              {
               "data": "/9j/4AAQSkZJRgABAQAAAQABAAD/2wCEAAkG...."
               "filename": "screenshot.jpg",
               "contentType": "image/jpeg"
              }
         ]
       },
      
        {
            "testKey": "BOOK-458",
        	"start": "2021-07-14T11:47:35+01:00",
        	"finish": "2021-07-14T11:53:00+01:00",
            "status": "PASS",
        	"results": [
              {
                "name": "open login page",
                "log": "page opened",
                "duration": 2,
                "status": "PASS"
              },
              {
                "name": "enter valid username and password, and login",
                "log": "",
                "duration": 3,
                "status": "PASS"
              }
             ]
        }
    ]
}

After importing results, a Test Execution would be created containing the results for the two tests; the details of the execution screen show the data of the Test Run, including its status, comment, timings, among other.

Xray Cloud

example of a API request with "curl" using basic authentication

token=$(curl -H "Content-Type: application/json" -X POST --data '{ "client_id": "32A27E69B0AC4E539C14016437000000","client_secret": "d62f81eb9ed859e11e54356dd8a00e4a5f0d0c2a2b52340776f6c7d6d7000000" }'  https://xray.cloud.getxray.app/api/v2/authenticate)

curl -H "Content-Type: application/json" -X POST -H "Authorization: Bearer $token"  --data @"payload.json" "https://xray.cloud.getxray.app/api/v2/import/execution

JSON body payload

{
    "info": {
        "summary": "Execution of automated tests for release",
        "description": "This execution was automatically created when importing execution results from an external source",
        "project": "BOOK",
        "version": "1.0",
        "revision": "1.0.42134",
        "startDate": "2021-07-14T11:47:35+01:00",
        "finishDate": "2021-07-14T11:53:00+01:00",
        "testEnvironments": [
            "chrome"
        ]
    },
    "tests": [
        {
        "testKey": "BOOK-429",
        "start": "2021-07-14T11:47:35+01:00",
        "finish": "2021-07-14T11:53:00+01:00",
        "comment": "failed execution",
        "status": "FAILED",
        "results": [
              {
                "name": "open homepage",
                "log": "all good so far",
                "duration": 2,
                "status": "PASSED"
              },
              {
                "name": "enter username and password",
                "log": "",
                "duration": 3,
                "status": "PASSED"
              },
              {
                "name": "login",
                "log": "invalid credentials",
                "duration": 123400,
                "status": "FAILED"
              }   
           ],
		"evidences" : [
              {
               "data": "/9j/4AAQSkZJRgABAQAAAQABAAD/2wCEAAkG...."
               "filename": "screenshot.jpg",
               "contentType": "image/jpeg"
              }
         ]
       },
      
        {
            "testKey": "BOOK-458",
        	"start": "2021-07-14T11:47:35+01:00",
        	"finish": "2021-07-14T11:53:00+01:00",
            "status": "PASSED",
        	"results": [
              {
                "name": "open login page",
                "log": "page opened",
                "duration": 2,
                "status": "PASSED"
              },
              {
                "name": "enter valid username and password, and login",
                "log": "",
                "duration": 3,
                "status": "PASSED"
              }
             ]
        }
    ]
}

After importing results, a Test Execution would be created containing the results for the two tests; the details of the execution screen show the data of the Test Run, including its status, comment, timings, among other.

Tips

Use wiki markup to not only to format text but also to provide additional information or access to it

The following fields support Jira wiki markup, so you can use formatting in several fields (Jira related and Xray specific):

Description field of Test Execution
Overall comment on Test Run
Actual Result and Comment fields on the steps of Test Run (only for Test Runs of step-based Tests, such as "Manual" Tests)

There can be some subtle differences between Jira server/DC and Jira Cloud, so we advise you to try it out on both deployment types.

What can it be used for?

Some ideas:

On the Description field,
- provide a link to the external tool, where users can see the results for that batch execution as a whole
- provide some table, or some bullets, with some additional info about the context of the batch execution and that thus apply to all results (i.e. Test Runs)
- embed a image (e.g. a chart); image must be stored in some accessible URL; it's also possible to atach a image on the Test Execution issue but that would require a separate, Jira specific, call after importing the results
On the overall comment of each Test Run,
- provide a link to the external tool, where users can see detailed results for this specific Test Run
On the Actual Result, or on the Comment fields of each step (only for Test Runs related to step-based/structured Tests), highlight some information using bold

Xray Cloud

Xray server/DC

markup content for the "description" field

Performance testing results, for more information visit [XPTO Web|http://www.example.com] \n!https://docs.getxray.app/plugins/servlet/rw-resources/sites/1/images/1.png!\n\nHere are the global statistics of the test :\n\n* User load:45\n* Number of Requests/s : 51.034195\n* Average Request duration (ms): 89.76852\n

Xray JSON body payload (cloud)

{
    "info": {
        "summary": "Execution of automated tests for release",
        "description": "Performance testing results, for more information visit [XPTO Web|http://www.example.com] \n!https://i.postimg.cc/rs4kL7q5/image.png!\n\nHere are the global statistics of the test :\n\n* User load:45\n* Number of Requests/s : 51.034195\n* Average Request duration (ms): 89.76852\n",
        "project": "BOOK",
        "version": "1.0",
        "revision": "1234",
        "startDate": "2021-07-14T11:47:35+01:00",
        "finishDate": "2021-07-14T11:53:00+01:00",
        "testEnvironments": [
            "chrome"
        ]
    },
    "tests": [
        {
        "testKey": "BOOK-70",
        "start": "2021-07-14T11:47:35+01:00",
        "finish": "2021-07-14T11:53:00+01:00",
        "comment": "...",
        "status": "FAILED"
       }
    ]
}

Xray Cloud

Xray server/DC

markup content for the Test Run comment field (i.e. "comment" field of each "test" element on Xray JSON)

This execution *failed*. \n\n||*Col1*||*Col2*||\n|row1_col1|row1_col2|\n|row2_col2|row2_col2|\n\nMore details [here|http://www.example.com].

Xray JSON body payload (cloud)

{
    "info": {
        "summary": "Execution of automated tests for release",
        "description": "test automation results",
        "project": "BOOK",
        "version": "1.0",
        "revision": "1234",
        "startDate": "2021-07-14T11:47:35+01:00",
        "finishDate": "2021-07-14T11:53:00+01:00",
        "testEnvironments": [
            "chrome"
        ]
    },
    "tests": [
        {
        "testKey": "BOOK-70",
        "start": "2021-07-14T11:47:35+01:00",
        "finish": "2021-07-14T11:53:00+01:00",
        "comment": "This execution *failed*. \n\n||*Col1*||*Col2*||\n|row1_col1|row1_col2|\n|row2_col2|row2_col2|\n\nMore details [here|http://www.example.com].",
        "status": "FAILED"
       }
    ]
}

Attach files (e.g. screenshots) as evidence

During the execution of automated test scripts, some evidence may be produced. Usually these can be screenshots, a video recording of the whole session, page source, or log/dump files.

Even though we should not mix up Xray with a CI/CD tool, these files can be helpful for providing an immediate analysis capability, right on Xray.

What can it be used for?

Some ideas:

On the overall evidence of each Test Run, no matter the type of Test,
- to attach a video recording of the session
- or to attach screenshots that were taken by the test script; screenshot names could be prefixed or suffixed with a number to help out identify them better (e.g. 1_screenshot.jpg, screenshot_1.jpg). The latter can be helpful for generic/unstructured Test Runs, where steps as such are not available
On the evidence of each step (only for Test Runs related to step-based/structured Tests), attach

Are there any constrains on evidence/attachment size?

On Xray server/DC,

similarly to what happens with attachments on Jira issues, Test Run evidence are stored as attachments on the parent Test Execution issue; this means that the global setting for Jira attachments also apply to Test Run related attachments. Attachments are stored in a local folder in the Jira server/DC instance

On Xray Cloud,

standard Jira issue attachments are stored in Jira Cloud's infrastructure; there are some constraints on the size limit and total space for attachments. Xray data, including evidence/attachments added at the Test Run level, are stored in Xray Cloud infrastructure; in this case, please see Xray' storage documentation.

No matter the deployment type, we advise not to upload very large files (up to a few tens of MBs).

Xray Cloud

Xray server/DC

Xray JSON body payload (cloud)

{
    "info": {
        "summary": "Execution of automated tests for release",
        "description": "test automation results",
        "project": "BOOK",
        "version": "1.0",
        "revision": "1.0.42134",
        "startDate": "2021-07-14T11:47:35+01:00",
        "finishDate": "2021-07-14T11:53:00+01:00",
        "testEnvironments": [
            "chrome"
        ]
    },
    "tests": [
        {
        "testKey": "BOOK-70",
        "start": "2021-07-14T11:47:35+01:00",
        "finish": "2021-07-14T11:53:00+01:00",
        "comment": "failed execution",
        "status": "FAILED",
		"evidences" : [
              {
               "data": "/9j/4AAQSkZJRgABAQAAAQABAAD/2wCEAAkGBxMSEhUSEhIVFRUWGBgYFhgWFhcWFxYYFxgXFhUVFRcaHiggGB0lHRUVITEhJSkuLi4uFx8zODMtNygtLi4BCgoKDg0OGxAQGyslHSUyLS0tKy0tLS0tLS0rLS0tLS0tLS0rLS0tLS0tLS0tLS0tLS0tLS0tLS0rLS0tLS0tLf/AABEIAQwAvAMBIgACEQEDEQH/xAAcAAABBAMBAAAAAAAAAAAAAAAGAAQFBwECAwj/xABVEAACAQIDAwYHCgkLAgUFAAABAgMAEQQSIQUGMQcTIkFRkTJUYXGBktEUFhcjUpOhscHSFSQzQlNygrLCNENEYmNzoqOz0+FVgyU1dKTwZJTDxOL/xAAZAQEBAQEBAQAAAAAAAAAAAAAAAgEDBAX/xAAwEQACAgEEAAUCAwkBAAAAAAAAAQIREgMhMVEEExQiQXHwMoGhM0JSYoKRscHhBf/aAAwDAQACEQMRAD8AvGlSpUAqVKsUBitqVYoDNKlSoBUqVKgFWtZrNAKlSpUAqVKlQCpVhjWFNAKtqVYoDNKlSoBUqVKgGO2dpDDxGQjMbhUW9szscqrfqFzqbGwBOtqoverla2nBiZIUaBQhA0iPWoa12Y3te19L24DhVr8oJ6GE/wDVpf5ubj262rzpyjpbaWI86HvjSqrYz5Jh+WHa5/n4x5oY/tFcm5W9rn+lD0QxD+GgelWGhoeVba/jh+ai+7WrcqW1j/TG9SP7tBtKgC48pu1vHpPVj+7Wh5SdrePS9yfdoUrtg5FV1Z0DqDqpJAOhtcjW17Hy2oAjPKLtU/0+bvX2VzPKBtTx/Eev/wAVA45EEjBGDLpqoYLcgFgobXKGLAX4gCuFAEg3+2p4/iPXrPv/ANqeP4j16GqVAEvv/wBqeP4j16x7/tqeP4j16HKVAEfv+2p4/iPXp/sve/aUmr7SxCJqqtnB6Yym1uNrHVuC5lJNReE2CFKmdgL3BS9irdLJzjC+VWC3B679mtYx+1TGxSImwXIRmzILKFVgFOUtk6LDVcyki+lYCfl5QcdE0gbEYnNeMLG7sCAFu7Fh/WtwOoJ0Gloo8oe1Lk+75terMLDzaUMAUq0BQOUXavj83+H2Vv8ACTtXx6XuT7tClKgCwcpm1h/TpPVj+7XfD8qe1lYE41iLi94oSLX10yUGVvAAWUEXBIFu25tQHrndDbT4mH44IJkID5L5WBGZJFB1AI6jwKsLkAEztA3J+lsVigDoIcIPTfE0dUYQMcoI/F4m+TiIT3tl/irz3yrpbaL+VIz9BH2V6F5Rv5Cx+TLhz/7iIfbVBcrUN9oqL2zxx69l3db/AEVv7pnyA9KvTMHJnstFVfcivYWLO0hZj1k9LiaxiOTzZIGZsEgA7DJ9jVj23Mc0lbPM9Zr0qu4uyPEU/wAf3q295OyfEI+5vvVGcezn6jT7PNFKvS53K2T4hH3H71Y95WyfEY+4/ep5kezPUafZ5ptSr0ody9leIx9x+9SG5uyvEY/V/wD6p5kOx6nS7PNdKvSnvQ2X4jF6g9tL3o7L8Qi9Qe2nmQ7M9Vpdnm1IyxCqLkmwA4kngKnYdmQp0jKb2JVrLZHjILKbEh2W1ytrMpuLm1XvFuxs1SGXAxAg3BEa3BHWNa7YjYWAcWfBxsBbQopGl7cT1Zj31nmQ7HqtLs88bU2qWJWM2W1iwzKzC3SS97mPMWsra2sOq1RVelvevs3xCH5qOs+9jZviEPzUdPNh2Z6vS7PNFKvTHvY2b/0+H5qOs+9rZv8A0+D5mKnmw7N9VpdnmalXpr3t7N/6dB8zFWY919nNwwGGFv7GP2VqnFukzY+I05Ok9zzJTrZUeaeFe2WMd7qK9KHdLZ/iOG+Zj9lUntnZkUO3OYhXLGuIiKr1LcJIQPJcmrO1l7bhp8djG8sK+qjH+OjGhHk8192N/wDUhfVghP8AFRfRhA9ygJfZ8/kCN6siN9lUHyyr+NRP2xfUxI/er0DvwL7PxfkgkPqqW+yqC5Yx08M3bE31Rn7a1fhMfJ6BLXse0U22n+TPo+sVvg3zRRt2op71BrXaP5NvR9YqdT8D+hw8R+yl9H/gFN7tqSQ+5ViljjMs6Ic6ls6kgFEAU2N2GulNFwm2LC+Iwt7SX6JtmIIht8XwBsTWm+E7riMFkTEOE5yQrDfK+QKwSQAHNfKQB5azJvhiApb8F4onmw9gr3uXC83+T8IA5j5BXlS2VI8UU8FSX6ElsPD49ZCcXLC8fNqFEYsecsudj0BpfPbXgRU0a5NMRHnIscuYjsNrkUCbptiJMOuEaLELHMkjNimxAleNszKEVggBNkUg30zeSoq9zljnctlQfkVrahRtnS4AiWH3RjWk5uFkeWyxqq2M3gnW6C/aWNY3xnkXE4Tm1nfJzshWKTKrlQrKkigEvcpYec1mNsladuk/tBXasWoWfY0uNUYh5sThGkiCGFXPxdnvm4DpEDjbgakdi46QiVJIJEWDoI7kkzhQRzguo45QeJ8KscSXClz9fv5Ji1K1CcOFk2iiYkyYjB9CSPmlc9eZecbhci9xppYVpNh5NnFZg+Jxhk5uARs5sll1l69SU1NvzqYfHyb5S4vfr/vAX1m1CW9+KkTFYUxriXCLNIUhLBJCq5lSQAHNcpYDy1vLseXGr7oOIxWEMsSrzKsy80Q4bNbSzkLY6cGIrMNrYWlsm3SYVgUH7C29JJgsZiWnV8jTc24jIEYVAVBGUFrE9hvUtsjaTskyvBLGICUVpL3nCAjnVuBocoN9eND2zjImxZCRiC7ZtGkvNrIE0e2mgvwqox/0dIQq0+0ONobcmj2SmJ90oJWynnTH0TmckWjy9ai3CjTZ18upudLntNtTQTvDG4wOBiHP5meAMUktIOgcxke3SFzrwvRzhOB89ddOs1+Z20q8yNfzHc1Q20OnvC//AKj92P8A4q+CaoXZx5zeCQ9mJxH+ESL9letn0UXnycr8TiG+ViZD3JGn8NFlC3Jv/I83yp8Qe6Z1/hopqWWiK3qjzYLFL8rDzDvjYV595WxeLBP/AGbDvSM/ZXo3akeaGVe1HHepFeduUxb4HBMeNlH+Wfu1S4ZL5Rd2xHzYaA9sMR70Wu2P/Jt5qYbnsWwGDY9eGgPfEtSU6ZlK9opJXFo5aquDS6ZXW3MZDHtPDyTNEqwYdnLMZTIufOl1VeiQSQNQTrU2d78Fe3uhb5o1tlk8KUFox4PWAalJdgRydKSCB2KhSzorEqDcKSVuRfW1anduG9/c2HvcNfm0vmXRWvl4i5seqvJg6Vpngwbik4vZETtDbUM2BxM0MisixyqWIkChgnBrANbpC+X0Utx4VTAYcIEAKFhkz5emzPdec6Vul11NwbGREMaRRKhuSiqFU343ULY9VdEwRUAAKANABoAOwACpcJVSTIenPHFRfJzvQjtDK+2cODzZMUDOL87zgzc4pIt8XbUcdfooz9yN/V7z7K4fg0Z+cyx57WzW6VuzNa9vJWKE18Ew09SN+18Mzemu1I2aGVUALtG4UMSAWKkAEjUC5p77lbtXvPspe527V7z7Kny59ELR1E7xYG7C2thsBh1w08iRSQi8qIJXVDI5ZbNZib5x1njS21l2g0Iw3NzLh8SDPn5xObKEXyeDmYDN2iiefYcTlmeGBi1sxZAxa2gzErrbqrrh9mLHfm1jTMczZVy5mPFjYany1eErunZ2cJ3movL9AVxiq+2YL82TFh2YXMnOAtzikgA5MtmHEX+iiy9N58AquJcsQkPRMmW75eOXNa5Gg0vXaCJzfNlFrdo0IuL1L05utiJ6WpJL2vZUNtrSxLBKZzaLI3OHpeAQQ3g9LgerWoLGRbNTZscchtgny5Py12zMZV4DnNSCdaKZcFmBVwjKdCGFwfIQRrWkmykKhGjiKLbKpUFVsLDKpFhYGtWnNfDNhp6i5T5+/wAyG23FgjLhIcR+UD5sMvxnhR5dejpp0fC0ohwx09NcZcArMHZYyy3ysRdlvxyki49FdoksLGumjCSkrR20ITU02nwzrVD7odLbcrf2mJbvcj7avgVQnJy2bacz/wBWZr+eRfbXqZ9FF+cnSW2fF5Wmb1p5W+2iSoLcVbbPw3ljVtdPC6XD01PVJZq63BHaLV5u36Q/gnBHsKA+pIPsr0nXnnfpAdlAD+amIPkyyun21UeGS+UEW5PKbs+PBYeHEStDJFGsTAxyMDkAUMGRSLEAcbUTQ8oWy21GOi/azr+8orzFeless2j1Iu+uzjwx+G9Mqj666JvZgDwx2FP/AH4/vV5XvWDW2Zier12/hDwxeHPmmjP8VbjasB4TxHzSJ7a8mWpWpkMT1n7vi/Sx+uvtrPuyP9InrD215LtWLUyMxPWZxK/LX1hWDiV+WvrCvJtqzYdlMhierzik+WvrCtGxsY/nE9ZfbXlOwpWpkMT1Q+OhPhSx+l19tcvwthk/n4R55EH1mvLlqVqZDE9QtvJgxxxeHHnmj+9XI73YDx3DfPR+2vMdqVZZuJ6YbfXZw442D0OD9VNpOULZi8cYnoWRvqU15wp2NnSZGcqVUKGu2gYMVAy9vhDyDrtSxiX5i+UvZ6xs0cjyMFLKoikTN1DpuoUAnS9+Omp0qseSofHYhz+bA3eSD/DUbjp0w6PFljZTYRxkl2XUM5kObo3IGgIBIBXS95Hk3kCrjn4Wh9A0kNtfNRchqkejdz48uAwg7MPDf5tb1L3prsiLJBCvyY0HcoFO6woVQWP3dwcmZpMJA5YnMWiQkk8STbWp2mc3CgBeTcTZjccBB6Ey/Vamc3JlsluODUfqySr9T1L7U29HAQrK7Ei/RA085YimJ3yj/QTf5X36UyckRj8k2yT/AEdx5ppftauMnJBss8EmHmmb7b1MjfGP9BP/AJP+5UnsnbcWIzBcystsyuLML6qesEGx1BI0I6jWbi0wJk5F9nHg2JHmkU/WlcDyI4HqnxXrRf7dGeK3ohQkZZGt1gLY200zMD6a4e/CP9DL/l/fraYyQIHkQwfVicT3xfcrU8h+E8axHdH92jMb3Rfopf8AL+/Uph9rQvEZg1kUMWJBBXLfNmHVaxrNxaZW3wH4Xxqfuj9lY+A/C+NT+rH7KNpd7Yh/NTH0Rr9DOCPSK09+MX6Gb/K/3K2mMl2BvwHYXxqfuj9lZ+A/C+NYjuj+7RrHvbCTrHKvlsh+hXJ+ipObasSxiUtdDlykAnNmICgAcb3FZuLK4HIhhPGsR/l/drdeRHBdeIxPfEP4KMn3shH5kncv3q5+/CD5En+D71KYyXYKJyJ4AcZsUf24x9UdOE5GtmjicQfPKPsWizC7yQyGwDjzrp6bXp5tHakUCh5CdeAVWdj2kKoJNuvsoVFZbIDk5IdljjHKfPM+vdan0nJvs1gFaBiAbqOdlGW9tFysLDQG3C+tPG31w3yZj/2yPrtWBvrhvkTD/t3+o1mR29LrfwP+zOA5OtmXLHCKxJuS7yOSe0lmN6f4XczZ6AhcHCAePQGvn7fTW2zt58NO/NoXDcQHjdAfMxFifJe9Ta8DRM5zhKDqSr6kinAeatqwvCs1RIqZz+DTiaXKL2J1tpTJcQHBsDofJ5eFvNQArtFlXFqzKGCxuxB4GyObfRWYoIoky3Byc6S4VXbVVZSA2hsrLp2g1J7U2Cs7BucZCBbQKQfPcX+mmXvQ/t++MX+hhVqSo5OLsYbUw7ANzUaBLyc7lVGynnGyi5F1GTIRa3Gmu6g/HJv7iD9+eppd0B1zd0Y9tSOyNhRwFnBLu9gWa3grfKoAAsBmY9vSNZKSaEYuwQwqjn4SbWvEdeH5p1p3jYocigGwZzISqhiOcXMiWuNAuQceN6lcVukGYss7qCdFKxsq+RTlDW85Nq4+89vGB80PsetUkY4MHcREgPxbMwtqWQLb0BjUps6MjC4vQX+MsDa2qcD7KlIN1yuvP69ojA0+Tqx76k4NixrC0BzMrhg5Y9Js173Itbjpbhp2VMpWbGLQK4Zo2nlBAAZChJPDMyJcX6xcm/krcrDz8Cv4KqqaZcpKSSKWe/UbX9NSM26NzcT+tGCfTlKj6K5e84/p1+aP+5VZInFg2FXOMmbLcWzWzdV72041JoPxPDfrR/Uak03PN9cR6sYB9GZmH0VLYrYcbwCAFkC5cjKRmUpbKekCDw1uNQTUydsqMWgI2YEEyc5ly6+F4N8pyFvJmy38lS+Aea0vOOS9o8uSaJWtme9m1AHkro+5knVih+1AD9TiuY3Lm8aj/wDtj/vVWSJUGiHwJUA342e56iMp6Pl69Pp0qR3kc/FC5tzcul+zm7X7eNPsNuawYF8TmW+oSJUzDrW5LWB4G2tjoRxqX2tsNJwupUqCAQL6NbMCP2V7q5z34PZ4Ka0tWMpcJgRg9qqioObuVAFujluHLc5qt81ujxtburbHICCyxohCqjAsoWM5TdVzWu5sdNSo8vCaG5LAgriLEag82bg9RBD1mLc6RQVGJWzcQYQwvqAbM2h1Oo11rnTPuvxnhVLKMt/6ga2D+XX0/VVmweDQ7svdIQvnaXPbgAuX7TRETlQnqArYqj5v/peIhr6icOEiUXhWabQYsMQtiDbye2nNdD55yxEOcWvbW9MuZygm9726rWtepKmU/gmgKs5QdtYmPE83HM8ahFNkbLcm9ySNagcG+PmGZMVLxtZscImv5FeVT18bVI8ov8sP92n20y3XWJHbEzFSsAzpGSM0sl7RqF4kA6kjhYV3UVicb3G+Nx2Ow8jRS4jEK62zKcQ72uAw1DkHQjroy5ON4J5mkimkMgUIyltWGYupW/EjoX111PkqvsZiGldpHN2YlmPaTqaLeSwfjE393F+9LU6iVGxZGbX3jxrTyBcRKqg3IUkBV06l6hfz2HpqXSTaDIHiM7LqUtIWaQi+ZQGcGRLEEFbkdhPCF2/jikhytc66dVj4JIuekLCx4kGxFuJRsWzLgX5pZwiXOJ5yNPcjMTdOb0Fo/CGcG99OytpULAx96cYGuJ5V8nOObW01udTprerB2DvDO+z5ZWYGWMSgMQNShYKxUaHgL9tVZj7c5J08/Tbp/L6R6fp4+mj3dj/yzEeef99qmaWwTBSferG3P4zKf2iv0LYdwrs209piMTFsUIj/ADhM2Sx4HNwt5agZjqfOas7C4lXlmxc8WMhX3OyyJMoXDAZAoWNibuGNrKF4m9W0l8GKwJg3uxoYWxMnEcTm+hr1ZOK21NJg4XR+akm5sFwobJmGZyga4vYEC9wL3sbVTUJ1Xzj66tSH+RYTzxf6bVzmlaNTGkU+KchVxWKdjwAKXPoSMVhp8SC34zigV8LpAZdQOkCmmpA9NYwuKEZa65ldGRhfKcrcbGxsdB1GpyWDnsysHDZoTIsYzMCw5uJWJ4iNBdv60nVatpEJt/JD4LamKWRW91zOBqUcQsjAakEiMMPOGHp4VNb0bXmvDHDIYc6yOzqqM9owlkXOrKLmQXJB0Xy3oeRMshW4OUsLjgbXFxUjvB+Vw/8Ac4j/APBUNblRk6GH4UxPjc5+aH1R11XEY4gFZcUQdQQqkHyj4vWuWzYkkfm30LghHJsEaxykjrBNh6aW0saWlcozBAcqAMQAijKn0AVeKJyY92DtnEc6Fed5FI8F1j08oKIpv5yaOLZkI7RVcbF/LL6fqqyIPArnJUzpB2h3h8Jlsb3t9tOqwtZodBUzl4emnlM5vBoAP3m3JGLkEqzc22UKbpnBtwI6S2qEbkvk8aQ/9ph/Gab7/wC2cQmJ5uOeRFCrojFOIvc5dSaHsHj8dM4jixGJd24KJpLnr+Vp566xUq5OTasKfgvk68UnzTH+OibdLdFcEHJk5x3sC2XKAq3ygC5+Udb9dVbLtrGRMyNiJwykqwM0hylTYjwrcRRpyd7xzzM8UzmSwVlLeELllIv1jo317amalW5qaMbW5NDJK7x4hVDG4VoySPJmDa91MTyWS+MR+q1Qm1t6MYZ3UYmUdIgKpyga6KoUcBw7fPSTEbVZQ6nGspFwymYqRxuCBYiqqZmxOfBbL4zGP2GP2ijDZm6yRYQ4UuWzB8zgWJZySWC621bQa8OuqoTerGjUYub0sD+8pqw9h7xzybPeVmBmQSqHKixZGZFdlFh1AkCw48KieXyUqILEclspJK4iM3PWjD7TXOXk0xbAK2JjYDgC0hA8wI0oek3vx19cZMfml/dQVtHvNtAqXE07KOLC5C28oFh6aupk7E9DyWTXBOIjGo4KzfRpfvo6l3eXmI4Ua3N5SpOt8oy6+cE1VGG31x4ZQMS2pA6Sow1PXdasbEbdlkwsTRtzby2uwAYp8W0jZQ1xeyEC4I1vY1Esr3N2OD7oS9Tp/iH2Vgbp4i5POJc3uczXN+NzbWopNo4q9vdc7HzQ3PoWIVuu1MVraeY21PgmwHWehpW+45+wlsJujIrAs6WHZcn6qkNtbvGUxujANGrpZr2KyBb6jgQUU8D19t6HcNt7EqbmZmA1KssZBA1IuFBGl9b8e3hUvvJtqVTHHE/N5kkkZgqs1ozEoUZgQLmW97HwfLUO7LjjRHNujiO2L12+5WPejif7L12+5TI7bxWv41J6sIt3R1q22MWP6VMOvVYeB4HWOr9xPsJvZO68scgeRksPkkknvAtRYgstA+wtu4gyhZJTIp4hkQEeUFFX6b0bobreold7nSNVsSYrNYFZoWKmc3gmndNJeFAU9yi/yw/qJ9Vdd18Vh0WJRM0c0kqc6TGWugkXLCrhhkU2BY9fmFFu8u5C4uTnVmMbWAIyZwbcDxFqhW5Ln8aX5k/frspxqjli7BDelV91TlHzgySEnKVsS7Er5bdvXU9yYD8Yf+7T96Sn/wAFz+NL80fv0TbqborgwxMnOO1gWy5QALkAC56ydb1M5JqkFFlS4w/jMvaecA85DAfXb00XbMjbFrDFNhp4ETDhRilkdY1RELrIylQjKb9t9e55tXkzMkrvHiQoYkhWjJtfqzBte6mc3JzicuVsauTsOfL3XqskKYA5tB2//LD6++rA3YP/AIXOf60/+q9cYuTGQ8MTH6EY/bRls7dVYsI2Fzk5w+Z7W6TksSB2XOgqJyTqjUikHtmN9BfXya1Zm28Osk8uEikxcKx4fNGVYLhMgjBsUAGZWBILkm5JHVUdiOS7EXJWeIgk8Q6+2urbhbRMfMnGKYuHN85Lkt2ZbWt5KtyXZlMruJukp8o+urUi0w2FHlA/9vLUNFyV4i4zTxAX6g7HuIFHmJ3dBijjRrGMgqT12VkINvIx9Nqick2hTAtScwy3zXGW3G99LeW9T22NY3yZecDKcXl62AFiOoqGzZrfna1j3q4hTdXQEcCGYEekDStsNuxOpJJQ3BBs7DQ6k3sL+brrbRCiyDwaA5ieAVge3pK1reXQ8dKkt5QOdi14QTf6mFqQwm7s4e7NGAbXKsxNgLWAKjj59LDSnW3N32lKNGy3VHSzEgEOY2vcA6gxDvNQ2rKUXQHYUx3PO85lt/N5b3045tLWvUntvm9Oc9085za83nMOXJrlvk1tx8tbNuniexPX/wCK1O6mJPUvpf8A4q8kRi+hhsE/HDzeyrIh8ChPY+7EscmeQqAB1Ek/VRagstq5yds6wTSJMVmsUrUOgiKZyno09plP4NAVZv7tTELiSiYiVFAWyxuYxqASTlsSbk6k9lDmHxWLkYJHPi3Y8FWeck+gNUvygn8bb9Vf3RXHcjFumMhRXKrJIocA2zAEkA+SvQorHg4tuyL/AAxik0904kEcQZ5TqPO1HHJ5vFPKWimcyWsVZvCAJIsT1+DfXtqvca3Tf9ZvrNFXJl+Xf9Vf3nqdRKhFsi9rbz4znX/GpQMx0UhAL65QFA0HDXXTUk01O2sflEgmxJUfnFnK66cTpfWmO1W+Nk/WNFew58Rh4DiMQ8rRtE0WHw5LMJQVyhinBYlBve2ulvLtLoWyD99GLUnm8TMg0sC6udAAWJZCCSQTwtrVhbK3kmk2c8xKmZA6hiuhZWKK7KLdgJAt18KqS+lH+7p/8Mn/AFpP9U1E0lRSYNy744+5/HJT+zCP3YxXI74Y7xuX/L+5UNIeNHOztotiMHi1bESYh1w5ZopUCxxZSLvG4zdJbdEdG/o0pxRlkJhN9toB1/G3bUaOkBXU9dow1vTVmYjbkjwRtGcjy21ADFegznKDoT0ba9tUrg0zSItwLsoueGpHGraVlXD4cEcbjuglPlsOj5wfpiaVo23RqcbOOOMmOmvRw6gi56S/FasOBHk4ddNJ9rYkno4uYC5/Ng1v1WMZ0HV5ON64TzFvN2X8lrntPlrgTW4o5ObJPB7exKkFpzIARcOkeouL6oqkG17VK7xbclRkjiYJeN5GbKGPRaNQqg6C5kuSQfB8tC6dfmqT3gPxqH+wk+mSD2VDSstSdDZtu4vxp/m4P9utPw7jPG39McH3Kb4MRljzrOq20KAMb3HUSNLXrviYsMFJSWUt1BowAT5SG0q8UTkyR2FvBiWlCySLIhGvxYRh2EFTY+a3X5NTRWutVtsA/Hej7RVkQ+DXOSpnSDtEqKzSrFDoZplL4NPaYzno0AI7ybkjFyc6sxjNgCCmcG2gPhC2lQx5MZOrFL82R/HTHfjbGIXFNHHPKiqF0Ryg1AJOnHjUVhTtGZS8cmMdV4lZJiL8bCx1PkFdkpVycW1YQ/BfJ14pfmif46JN1dz1wYJMhkduLZcosL2AW57T11V67dxQH8qxFxwvLIbnykns6qOOTnb883OxzSGTIUKlvCAZdVv16i+uuvmqZqVblRoa7U5MneR3jxChWJIVkNwD1XB1rePczaaAKm0WCgAACWcAAaAAcAB2UMbU3rxnOuPdEgsxFlOUCx4AC2lNPfPjPGZvnG9tbUjLQRLyW4jrniHmDn7BRls/dRYsIcLnJzBrva3SYliQL8LnhequXe3HDhipfSVb95TVg7L3kmfZ7TEgyrnTMV0JVzGHKiw6gbCw81RPL5KVAzLyX4oG6TwmxuCc6nyHRTaneP3T2xMhjlxcboeKmWQA/rWjGb01AYXebacriNMXK7seiAkAJ6+qMDgK5De7HW/lkt+roxG5v19CrqROxKYfkvxeYFpYFFxcgux7igv30V7ewQhjwsYN8rlQT2DDTigOLfPaSPY4liQbFJIobXvazWRX48RcGjfb+P56PByWtnkJt2XwuINqiV2rN2ohSad4/CBY4WUG8iFm69QxGnZpXLCY+SEkxOVJFjYA3HHrFSuO3hm5uHJOcxVucsFvmzaX000qziqITD6taibauwXlyPGVuEZCGJGjFGBBsdQUtY9vHTUZgcl7nUm5PlJ1NEO2duTRmOOEqt42dmZc5spRQqi4Gpe9z2eWolydIVW5Gtutifkr6wrQ7q4n5C+sta++PFeMd0UfsrX3yYzxgemKP7LVvuM9hJbF3amjkzSZQLdRueIP2UYILLQbsPeTEPJklMbqexCjDy3zEHzW9PVRkpuKiV3udI1WxK0qVKhYqY4jwafUwmPFaAprfd/x2X9n9xal8LhpZo9mthwWWJm5wrwifnQzNJ8m663PVUzvDuIuJmaZZ2jLWuMoYXAA01BHCohuTF+rFj5o/frspqjji7BbeKZHxU7R2yNK5UjgQWOo8h40R8lY+OxP/a/crsOTF+vFr80fv0TbqbrJglf4wyPIQWYgKLKoVVA17L6k8TUzmmqRsUyoton46T9dvrNPQcJ7k4Se6s/H8zJ9XD038lFe0OTRmkd48SoDMSA0ZuLm9rhtfPamZ5M8R+ni7n9lXnEUwLNH+wP/ACqX9eT/AFzTUcmU/XiIx5lY+yjLB7srHhDhc5Nw12t+czFybdmY8OyonJOjUitt0dvphZU5yKJkzFjIY80yXXLaNr6Dhp5TW2J2iECYloIIpmX8XjiTIqi5/GpFJNz1J22zcALvvg3xasCHw7gEGzmQBrG9mAXgevWn+H3d2uhciXCEyNmbOqvrYKLZoTlAAACjQAVWSMplfrJdgSbksCSTck31JNWTjj+L7PH9f/8AUxBqOxG4WOnlEk0uHU6A82pGg7EVFF6Msfu4HjhRWsYDdb8D8VJDrbySE+ionJNo1LYD2NaE1Ltuvif6h/a/4rQ7r4nsX1hVWjliyPwx6XoP1VI7e/Kr/cN/qRV3wO604cFygXrsbn0aVIbd2BJIytGRpGyEHTiyMCD+wRby1DastRdA3smHO5HNc70ScvOiK2o1zHjx4eXyU/x+zssbN7jZLDwvdKOF8uUC5ps+6+K+QD+0vtrn72MV+iHrJ7au0RT6OewT8cPN9oqyYvBoJ2Ju7OkoaRQoA+UD2dlGycLVylyddNUiXpUqxQ6CauM+HDa8D/8AONd6VARbYSQcLH0+2tTFIPzT9BqWpVlAhjn+S3ca1Mh6wam6VKBBc/WTNU0VHYK1MC/JXuFKBDiWlztSpwifIHdWPcUfyR9NKBF86KXOCpP3BH8n6T7ax+D4/k/SfbSgR3OVnnKkPwfH8n6T7aX4Pj+T9J9tKBD42awHHj1XHUeysYOXwrm/Dt7NQL1M/g+P5P0n21n3BH8n6T7aUCO50VjnhUkMFH8kfTW4wifIXupRhFc+Kx7oFTAgX5K9wrcKB1ClGkGJCeAJ8wp7g8Ib5n07B9pp8WralAwtZpVitBmlVI/DPivFoO+T20vhnxXi0He/trcWZaLurFUl8M+K8Wg75PbS+GfFeLQd7+2mLFou6lVI/DNivFoO+T20vhnxXi0He/tpixaLupVSPw0YrxaDvf20vhnxXi0HfJ7aYsWi7qVUj8M+K8Wg75PbS+GfFeLQd8ntpixaLtrNUj8M+K8Wg75PbS+GbFeLQd8ntpixaLtrNUj8M+K8Wg75PbS+GfFeLQd8ntpixaLupVSPwz4rxaDvk9tL4Z8V4tB3ye2mLFou6sVSXwz4rxaDvk9tL4Z8V4tB3ye2mLFou6tc1Un8M2K8Wg75PbWPhmxXi0HfJ7aYsWi7FWt6pH4Z8V4tB3v7aXwz4rxaDvk9tMWLRd1KqR+GfFeLQd8ntpfDPivFoO+T20xYtFZ0fYCDCQYNp+bh5xsG4aOSYl52IjZioinNkLC1ssbqdNdaAaVqsksqPZOzsPiFnUwtDG8YBbEBrS+7gpuue7BcOQ2oy2W5vrW0OBwM5iEsOGXm4mXJHib3dcU3Oqb4hb2ibOGLa5yekECis7UrUoB7htlbPdmjWOPKIEcTyYjRWkDs3PKs6aqFRQYw1iTmU3FN9sRYXESFI44YmeKB1n5/4mKJIEEhKc5rLzi82YyL2XNa5JIValQFlRbPwK+6IiII4XWLmWGLznE5ZY2iMtmYwKz6M4Aspc2XLeq5xB6R6CoRoVUki40OpZuvsNuyudqVAKlSpVoFSpUqAVKlSoBUqVKgFSpUqAVSeBx7qgQCCwvq69I3Jbjx0J41GUqAlJcSz2uYBYs3pa9x5tfoHYK5862msPb9nD01H0qA7Yo63JU6fm8BXGlSoD//2Q==",
               "filename": "failure_screenshot.jpg",
               "contentType": "image/jpeg"
              }, 
              {
               "data": "dGhpcyBpcyBhIGR1bW15IGxvZw==",
               "filename": "log.txt",
               "contentType": "text/plain"
              }
         ]
       }
    ]
}

Implementation Checklist

Make sure your tool provides these:

Xray and Jira configuration
- Xray server/DC or Xray Cloud ?
- Server/DC specific settings
  - Use Jira username/password or Personal Access Token?
  - Jira username
  - Jira password
  - Personal Access Token
- Cloud specific settings
  - client id
  - client secret
On the test tool, associate the project to a Jira project
1. identify Jira project by its key,
2. specify the Jira project version, to identify the version of the application that is being target by testing
On each test on the tool, associate it to an existing Xray Test issue
1. specify Test issue key
If the tool provides a concept of a group of tests to be executed/orchestrated, and where then users can see the results of each iteration, then optionally associate to this group:
1. Test Plan, by specifying its issue key
2. Test Environment, by specifying its name; in Xray Cloud, the Test Environment needs to be defined on the Xray project settings

Provide links on Xray entities to external entities to facilitate navigation

Sometimes it may be useful to provide a way to quickly navigate from a Xray entity (e.g. a Test, Test Execution, Test Plan) to the corresponding entity in the external tool.

The general flow can be depicted on the following diagram.

There are several ways to accomplish it; however, we recommend creating a new specific custom field (e.g. "ExternalRef") for this purpose. You can reuse this field for different issue types, in case they have their corresponding flavour on the external tool.

Create a text-based custom field.

This can be done from Jira administration page, under Fields > Custom fields, or right from the issue screen where you want to provide it.

... or ...

Name it (e.g. "ExternalRef") and add a friendly description.

You need to know the custom field id (e.g customfield_xxxx) that you need to pass on the API requests. This can be obtained by making a call to the Jira REST API (GET /rest/api/2/customFields), or from Jira administration page under Fields > Custom fields > Edit. Note it down.

We need to make sure to use the Wiki Style Renderer, so we can have links on the field. For that we need to go in Jira administration Fields > Field Configurations > xxx > Configure.

Create a text-based custom field.

This can be done from Jira administration page, under Fields > Custom fields, or right from the issue screen where you want to provide it.

Name it (e.g. "ExternalRef") and add a friendly description.

Associate it to the screens where you want to have it available and save it.

Later on, you'll need to know the custom field id (e.g customfield_xxxx) that you need to pass on the API requests. This can be obtained by making a call to the Jira REST API (GET /rest/api/2/customFields), or from Jira administration page under Fields > Custom fields > Edit details. Note it down.

We need to make sure to use the Wiki Style Renderer, so we can have links on the field. For that we need to go in Jira administration Fields > Field Configurations > xxx > Configure, and then configure the renderer for the custom field.

Examples

Scenario1: provide a link on Xray Test issues to the corresponding test in the external tool

In this case we're assuming:

Test issue(s) already exist in Xray (they could have been created as usual by a user or by some automation process)
An automated test equivalent exist in the external tool, for each Test

And we want to...

be able to identify the test ID on the external tool and easily navigate to it right from the Test issue screen

The implementation would be exactly the same in case we want to provide this capability for other Xray entities, such as Test Execution or Test Plan issues, if they have a counterpart entity on the external tool.

Xray server/DC

As the API call is made on Jira server/DC REST API, we need to use its authentication mechanisms (e.g. basic authentication or Personal Access Tokens).

example of a API request with "curl" using basic authentication

curl -H "Content-Type: application/json" -X PUT -u jira_username:jira_password --data @data.json https://jiraserver.example.com/rest/api/2/issue/<test_issue_key>

example of a API request with "curl" using Personal Access Tokens

curl -H "Accept: application/json" -X PUT -H "Authorization: Bearer MDkzNjIyMTEwNjQzOqb+ApTGm3af+z3eGdcyDm7xHv6R" --data @data.json https://jiraserver.example.com/rest/api/2/issue/<test_issue_key>

JSON body payload

{
    "update" : {
        "customfield_13000" : [{ "set": "[EXTOOL_1.2|http://www.example.com]"}]
    }
}

After making the request, the Test issue will have the our field (i.e. ExternalRef) with a link to the test on the external tool.

Xray Cloud

As the API call is made on Jira Cloud REST API, we need to use a Jira Cloud API token created manually beforehand.

example of a API request with "curl" using basic authentication

JIRA_API_BASE_URL=<your_jira_cloud_url>/rest/api/2
JIRA_USERNAME=<jira_username>
JIRA_API_TOKEN=<jira_api_token>
TEST_ISSUE_KEY=<test_issue_key>

curl -H "Content-Type: application/json" -X PUT -u $JIRA_USERNAME:$JIRA_API_TOKEN   --data @"payload_cloud.json" $JIRA_API_BASE_URL/issue/$TEST_ISSUE_KEY

JSON body payload

{
    "update" : {
        "customfield_10033" : [{ "set": "http://www.example.com"}]
    }
}

After making the request, the Test issue will have the our field (i.e. "ExternalRef") with a link to the test on the external tool.

Implementation Checklist

For this purpose, please make sure your tool provides these:

Xray and Jira configuration
- Xray server/DC or Xray Cloud ? (which implictly will define if Jir server/DC or Jira Cloud are being used)
- Server/DC specific settings
  - Use Jira username/password or Personal Access Token?
  - Jira username
  - Jira password
  - Personal Access Token
- Cloud specific settings
  - Jira username
  - Jira API Token
On each test on the tool, associate it to an existing Xray Test issue
1. specify Test issue key

Provide links on external tools to Xray entities to facilitate navigation

Sometimes, some external tools may refer to Xray entities, such as Tests, Test Runs, or others. It may be useful to provide a direct link from the external tool to Xray, to be able to see the target entity on Xray side.

The link to the UI showing the details of the target entity may not returned on the API calls; however, it's possible to easily build it following some rules.

Building the URL for the target link depends on the entity we aim to navigate to; the following table sums this information.

Entity

Xray Server/DC

Xray Cloud

Notes

Test, Precondition, Test Set, Test Execution, Sub Test Execution, Test Plan

<Jira_base_URL>/browse/<issue_key>

Example: https://mylocaljira/browse/EWB-399

<Jira_base_URL>/browse/<issue_key>

Example: https://myjiracloud.atlassian.net/browse/EWB-399

Test Run

<Jira_base_URL>/secure/XrayExecuteTest!default.jspa?testExecIssueKey=<testexecution_key>&testIssueKey=<test_key>

Example: https://mylocaljira/secure/XrayExecuteTest!default.jspa?testExecIssueKey=EWB-399&testIssueKey=EWB-344

<Jira_base_URL>/projects/<project_key>?selectedItem=com.atlassian.plugins.atlassian-connect-plugin:com.xpandit.plugins.xray__testing-board&ac.testExecutionKey=<testexecution_key>&ac.testKey=<test_key>#!page=test-run&testExecutionKey=<testexecutuion_jet>&testKey=<test_key>&testPlanId=<testplan_key>

Example: https://myjiracloud.atlassian.net/projects/EWB?selectedItem=com.atlassian.plugins.atlassian-connect-plugin:com.xpandit.plugins.xray__testing-board&ac.testExecutionKey=EWB-399&ac.testKey=EWB-344#!page=test-run&testExecutionKey=EWB-399&testKey=EWB-344&testPlanId=

Example assuming that we're working on project EWB and that we aim to show the Test Run related to Test EWB-344 in the context of Test Execution EWB-399

Create/update test(s) in Xray

In Xray Tests are usually associated with Requirements (either being a User Story, Task, Epic, or other issues your team uses), this is usually how teams work and will allow Xray to calculate the Coverage.

Regarding the flow of interacting with external tools the most common case is the one where the Test is already defined in Xray and the execution is done in another tool (like a test automation tool), having the need to import those execution results back to Xray (as described above).

Ideally when you create a new test in the test automation tool you should create a Test in Xray that will be his counterpart (this is to prevent having the need to manually create those), in order to link the execution results to the same Tests. However this may imply the need to update the Test (in Xray) each time it is changed in the eternal tool, so please take this into consideration when choosing to implement this integration.

At high-level, the flow can be depicted in the following diagram.

In Xray we can create three types of Tests:

Manual, are user-assisted tests, listing an assortment of guidelines the user must follow, and the results that such behavior is expected to accomplish. They use steps with action, data and expected result to define a manual Test.
Generic, is an agnostic Test that does not contain any structure. The only field that is related to a Generic Test is its Definition. For an automated test, the Definition field can have an ID or name that maps to the external automated Test case. An example might be a JUnit test created automatically by Xray when the JUnit XML report was imported with test results. Usually the ones used for automation as we only have one field where an unique identifier is expected that will be used by Xray to be able to connect back the executions.
Cucumber, Cucumber is a tool that executes plain-text functional descriptions as automated tests. The language that Cucumber understands is called Gherkin, which is a business-readable, domain-specific language that describes the software’s behavior without detailing how that behavior is implemented. Gherkin is a line-oriented language that uses indentation to define the structure. Line endings terminate statements (e.g., steps). Either spaces or tabs may be used for indentation (but spaces are more portable).

For the following examples we will provide one example of each type.

Before implementing the integration, review the following questions to help guide your implementation approach.

Questions	Possible answers	Tips
Do I want to link the Test to a Requirement?	No, we only want to see the execution results in Xray and do not use the coverage report or will link it by hand afterwards. Yes, I will provide the requirement in the request to create the issue to be linked since the start.	Coverage is a great asset that will bring visibility to all the elements of the team about the overall status of the release/project.
Do I need to know the type of test whenever creating a new Test?	Yes, because the format for creating tests is different from test type to test type, due to their kind (i.e. whether they are composed of steps or not, or if they are gherkin)
Can I differentiate these Tests from the “normal” ones created in Xray?	Yes, you can add an extra label that can be used in filters or add a link to the corresponding Test detail of your tool.	Adding an endpoint in the Test for the users to be able to jump from one tool to another easily is very useful.

Context	Xray Server/DC	Xray Cloud	Notes
Requests to perform the creation/update of the test	Use Jira Rest API.	Use GraphQL mutations to achieve it.
Linking to a requirement upon creation	Can be achieved within the same request.	We must perform a separate request to achieve it.
Usage of custom fields	As we are using Jira API to perform the requests, we need to know the custom field ids to be able to set values to them.	For Cloud Xray uses its internal fields.	Both use special fields, but in one case we use custom Jira fields and to interact with those we need to know their ids (Server/DC), and in the other case (Cloud) we use internal Xray fields that are surfaced in the GraphQL queries.

Xray Server/DC

In order to find the custom field ids you must go to the administration menu and click in "Issues"

Once there, click on the entry "Custom Fields" in the left menu (under "FIELDS")

The Custom fields page should appear, in here you can filter the custom fields to find those you want by typing, foe example, "test" in the search box

Finally to find out the ids of the custom field you want you must press the "Actions" gear and choose one of those entries (with exception of the Delete one for obvious reasons), in our case we are clicking on "Configure".

When you are overing that entry you can already see in the bottom left of the page the endpoint with the custom field id in it.

Check the URL of the configuration page as you can see in the screenshot (repeat to find all custom field ids that you need).

Examples

Scenario1: creating Test(s) in Xray

In this case we're assuming:

Requirements(s) already exist in Xray (they could have been created by a user or by some automation process)
An automated test equivalent exist in the external tool, for each Test

And we want to...

Create the “same” test(s) in Xray

The following request shows how to create Tests (of different types) in Server/DC and Cloud.

Xray server/DC

As the API call is made on Jira server/DC REST API, we need to use its authentication mechanisms (e.g. basic authentication or Personal Access Tokens).

example of a API request with "curl" using basic authentication

curl -H "Content-Type: application/json" -X PUT -u jira_username:jira_password --data @data.json https://jiraserver.example.com/rest/api/2/issue

example of a API request with "curl" using Personal Access Tokens

curl -H "Accept: application/json" -X PUT -H "Authorization: Bearer MDkzNjIyMTEwNjQzOqb+ApTGm3af+z3eGdcyDm7xHv6R" --data @data.json https://jiraserver.example.com/rest/api/2/issue

What will change between each type of Test is the payload, so we are providing one example for each type.

Manual Test

JSON body payload

{
   "fields": {
       "project":
       {
          "key": "XT"
       },
       "summary": "T4",
       "description": "T4: example of manual test. Details of the [external tool|http://www.example.com]",
       "labels": ["External_Tool"],
       "issuetype": {
          "name": "Test"
       },
       "customfield_12100": "Calculator",
       "customfield_10000": { "value": "Manual" },
       "customfield_10004": {
            "steps": [
				{
                    "index": 1,
                    "fields": {
                        "Action": "Step 1",
                        "Data": "input Data 1",
                        "Expected Result": "Excepted result 1"
                    }
                },
                 {
                    "index": 2,
                    "fields": {
                        "Action": "Step 2",
                        "Data": "input Data 2",
                        "Expected Result": "Excepted result 2"
                    }
                },
                 {
                    "index": 3,
                    "fields": {
                        "Action": "Step 3",
                        "Data": "input Data 3",
                        "Expected Result": "Excepted result 3"
                    }
                },
                 {
                    "index": 4,
                    "fields": {
                        "Action": "Step 4",
                        "Data": "input Data 4",
                        "Expected Result": "Excepted result 4"
                    }
                }
            ]
        }
   },
        "update":{
            "issuelinks":[
                {
                    "add":{
                    "type":{
                        "name":"Tests"
                    },
                    "outwardIssue":{
                        "key":"XT-227"
                    }
                    }
                }
            ]
        }
}

Keys	Values	explanation
project	XT	The project key of the project you are using in Jira.
summary	T4	The summary that you want to appear in the summary of the Test in Xray.
description	T4: example of manual test. Details of the [external tool\|http://www.example.com]	Description that will appear in Xray Test, in this case, as the description field accepts wiki markup, we are adding a link to the issue in the external tool (one possibility, please check this section for other possibilities) .
labels	External_Tool	This field will be used to populate the labels in your Jira issue, in this case it will add a label "External_Tool" to the Test (useful for filtering purposes).
issuetype	"name": "Test"	Please check in your Jira installation what is the name given to the Test issue (usually is Test).
customfield_12100	Calculator	This entry will add a link to the Test Repository path where the test is present (if the Test Repository entry does not exist it will create one). It corresponds to the "Test Repository" custom field (please check the correct customfield id for your Jira instance). The key is a concatenation of "customfield_" with the id of the custom field.
customfield_10000	"value": "Manual"	This entry corresponds to the "Test Type" custom field (please check the correct customfield id for your Jira instance). The key is a concatenation of "customfield_" with the id of the custom field.
customfield_10004	"steps": [ .....	This entry corresponds to the "Test Steps" custom field (please check the correct customfield id for your Jira instance). In this case (Test of type Manual with test steps) we define the test steps of the manual Test.
update	"issuelinks":[......	The update object is native o Jira and allows a lot of operations, in this case we are using it to define a link to the requirement. You can discard it if you have no intention of defining a link to the requirement.

After making the request, the Test will be created and the response will provide the id in the Jira side that you can use to access it.

example of an API response

{
    "id": "21543",
    "key": "XT-273",
    "self": "https://jiraserver.example.com/rest/api/2/issue/21543"
}

In Jira you can check the Test created.

Generic

JSON body payload

{
   "fields": {
       "project":
       {
          "key": "XT"
       },
       "summary": "T3",
       "description": "T3: example of generic test. Details of the [external tool|http://www.example.com]",
       "labels": ["External_Tool"],
       "issuetype": {
          "name": "Test"
       },
       "customfield_12100": "Calculator",
       "customfield_10000": { "value": "Generic" },
       "customfield_10003": "sum_script.sh"        
   },
        "update":{
            "issuelinks":[
                {
                    "add":{
                    "type":{
                        "name":"Tests"
                    },
                    "outwardIssue":{
                        "key":"XT-227"
                    }
                    }
                }
            ]
        }
}

Keys	Values	explanation
project	XT	The project key of the project you are using in Jira.
summary	T3	The summary that you want to appear in the summary of the Test in Xray.
description	T3: example of generic test. Details of the [external tool\|http://www.example.com]	Description that will appear in Xray Test, in this case, as the description field accepts wiki markup, we are adding a link to the issue in the external tool (one possibility, please check this section for other possibilities) .
labels	External_Tool	This field will be used to populate the labels in your Jira issue, in this case it will add a label "External_Tool" to the Test (useful for filtering purposes).
issuetype	"name": "Test"	Please check in your Jira installation what is the name given to the Test issue (usually is Test).
customfield_12100	Calculator	This entry will add a link to the Test Repository path where the test is present (if the Test Repository entry does not exist it will create one). It corresponds to the "Test Repository" custom field (please check the correct customfield id for your Jira instance). The key is a concatenation of "customfield_" with the id of the custom field.
customfield_10000	"value": "Generic"	This entry corresponds to the "Test Type" custom field (please check the correct customfield id for your Jira instance). The key is a concatenation of "customfield_" with the id of the custom field.
customfield_10003	sum_script.sh	This will be the unique identifier that Xray uses to identify the test. Make sure you add an identifier that is immutable and do not change.
update	"issuelinks":[......	The update object is native o Jira and allows a lot of operations, in this case we are using it to define a link to the requirement. You can discard it if you have no intention of defining a link to the requirement.

After making the request, the Test will be created and the response will provide the id in the Jira side that you can use to access it.

example of an API response

{
    "id": "21545",
    "key": "XT-274",
    "self": "https://jiraserver.example.com/rest/api/2/issue/21545"
}

In Jira you can check the Test created.

Cucumber

JSON body payload

{
    "fields": {
       "project":
       {
          "key": "XT"
       },
       "summary": "Sum of two numbers",
       "description": "example of cucumber automated test - Scenario Outline. Details of the [external tool|http://www.example.com]",
       "labels": ["External_Tool"],
       "issuetype": {
          "name": "Test"
       },
       "customfield_12100": "Calculator",
       "customfield_10000": { "value": "Cucumber" },
       "customfield_10001": { "value": "Scenario Outline" },
       "customfield_10002": "Given I have entered <input_1> into the calculator\nAnd I have entered <input_2> into the calculator\nWhen I press <button>\nThen the result should be <output> on the screen\n\n  Examples:\n    | input_1 | input_2 | button | output |\n    | 20      | 30      | add    | 50     |\n    | 2       | 5       | add    | 7      |" 
   
   },
        "update":{
            "issuelinks":[
                {
                    "add":{
                    "type":{
                        "name":"Tests"
                    },
                    "outwardIssue":{
                        "key":"XT-227"
                    }
                    }
                }
            ]
        }
}

Keys	Values	explanation
project	XT	The project key of the project you are using in Jira.
summary	Sum of two numbers	The summary that you want to appear in the summary of the Test in Xray.
description	example of cucumber automated test - Scenario Outline. Details of the [external tool\|http://www.example.com]	Description that will appear in Xray Test, in this case, as the description field accepts wiki markup, we are adding a link to the issue in the external tool (one possibility, please check this section for other possibilities) .
labels	External_Tool	This field will be used to populate the labels in your Jira issue, in this case it will add a label "External_Tool" to the Test (useful for filtering purposes).
issuetype	"name": "Test"	Please check in your Jira installation what is the name given to the Test issue (usually is Test).
customfield_12100	Calculator	This entry will add a link to the Test Repository path where the test is present (if the Test Repository entry does not exist it will create one). It corresponds to the "Test Repository" custom field (please check the correct customfield id for your Jira instance). The key is a concatenation of "customfield_" with the id of the custom field.
customfield_10000	"value": "Cucumber"	This entry corresponds to the "Test Type" custom field (please check the correct customfield id for your Jira instance). The key is a concatenation of "customfield_" with the id of the custom field.
customfield_10001	"value": "Scenario Outline"	This entry corresponds to the "Cucumber Scenario" custom field (please check the correct customfield id for your Jira instance). The key is a concatenation of "customfield_" with the id of the custom field. Possible values: Scenario Outline, Scenario.
customfield_10002	"Given I have entered <input_1> ......	This is where you will add the gherkin definition of the test.
update	"issuelinks":[......	The update object is native o Jira and allows a lot of operations, in this case we are using it to define a link to the requirement. You can discard it if you have no intention of defining a link to the requirement.

After making the request, the Test will be created and the response will provide the id in the Jira side that you can use to access it.

example of an API response

{
    "id": "21547",
    "key": "XT-275",
    "self": "https://jiraserver.example.com/rest/api/2/issue/21547"
}

In Jira you can check the Test created.

Please find more information in the documentation.

Xray Cloud

As the API call is made on GraphQL API of Xray Cloud, so the requests made to Xray's GraphQL API must also be authenticated using the same end point for REST API authentication.

The authentication process uses a Client Id and a Client Secret, both of which are based on an API Key created for a specific user in Xray Global Settings: API Keys global settings.

Thus, the first step for you is to obtain the token based on the Client Id and Client Secret of your assigned API Key. You can then use any GraphQL or REST API end points.

example of a API request with "curl" using basic authentication

token=$(curl -H "Content-Type: application/json" -X POST --data '{ "client_id": "32A27E69B0AC4E539C14016437000000","client_secret": "d62f81eb9ed859e11e54356dd8a00e4a5f0d0c2a2b52340776f6c7d6d7000000" }'  https://xray.cloud.getxray.app/api/v1/authenticate)

curl -H "Content-Type: application/json" -X POST -H "Authorization: Bearer $token"  --data @"payload.json" "https://xray.cloud.getxray.app/api/v1/graphql

What will change between each type of Test is the payload, so we are providing one example for each type.

Manual Test

The request for the GraphQL API is made using the createTest mutation as we can see below:

JSON body payload

mutation {
    createTest(
        testType: { name: "Manual" },
        steps: [
            {
                action: "Create first example step",
                result: "First step was created"
            },
            {
                action: "Create second example step with data",
                data: "Data for the step",
                result: "Second step was created with data"
            }
        ],
        jira: {
            fields: { summary:"Manual Test", project: {key: "COM"}, description:"Check details in the [external tool|http://www.example.com]", labels:["External_Tool"] }
        }
    ) {
        test {
            issueId
            testType {
                name
            }
            steps {
                action
                data
                result
            }
            jira(fields: ["key"])
        }
        warnings
    }
}

Keys

Values

explanation

testType

Type of test that will be created.

Possible values: Manual, Generic, Cucumber.

steps

action

data

results

List of steps for the manual test, divided between action, data and results.

jira

fields

Setting that will enable the definition of native Jira fields, such as, summary, project, description, labels, etc

After making the request, the Test will be created and the response will provide the id in the Jira side that you can use to access it.

example of an API response

{
    "data": {
        "createTest": {
            "test": {
                "issueId": "10134",
                "testType": {
                    "name": "Manual"
                },
                "steps": [
                    {
                        "action": "Create first example step",
                        "data": null,
                        "result": "First step was created"
                    },
                    {
                        "action": "Create second example step with data",
                        "data": "Data for the step",
                        "result": "Second step was created with data"
                    }
                ],
                "jira": {
                    "key": "COM-122"
                }
            },
            "warnings": []
        }
    }
}

In Jira you can check the Test created.

Generic Test

The request for the GraphQL API is made using the createTest mutation as we can see below:

JSON body payload

mutation {
    createTest(
        testType: { name: "Generic" },
        unstructured: "script.sh",
        jira: {
            fields: { summary:"Generic Test", project: {key: "COM"},, description:"Check details in the [external tool|http://www.example.com]", labels:["External_Tool"] }
        }
    ) {
        test {
            issueId
            testType {
                name
            }
            unstructured
            jira(fields: ["key"])
        }
        warnings
    }
}

Keys

Values

explanation

testType

Type of test that will be created.

Possible values: Manual, Generic, Cucumber.

unstructured

script.sh

This will be the unique identifier that Xray uses to identify the test. Make sure you add an identifier that is immutable and do not change.

jira

fields

Setting that will enable the definition of native Jira fields, such as, summary, project, description, labels, etc

After making the request, the Test will be created and the response will provide the id in the Jira side that you can use to access it.

example of an API response

{
    "data": {
        "createTest": {
            "test": {
                "issueId": "10136",
                "testType": {
                    "name": "Generic"
                },
                "unstructured": "script.sh",
                "jira": {
                    "key": "COM-124"
                }
            },
            "warnings": []
        }
    }
}

In Jira you can check the Test created.

Cucumber Test

The request for the GraphQL API is made using the createTest mutation as we can see below:

JSON body payload

mutation {
    createTest(
        testType: { name: "Cucumber" },
        gherkin: "Given I have entered <input_1> into the calculator\nAnd I have entered <input_2> into the calculator\nWhen I press <button>\nThen the result should be <output> on the screen\n\n  Examples:\n    | input_1 | input_2 | button | output |\n    | 20      | 30      | add    | 50     |\n    | 2       | 5       | add    | 7      |",
        jira: {
            fields: { summary:"Sum of two numbers", project: {key: "COM"}, description:"Check details in the [external tool|http://www.example.com]", labels:["External_Tool"] }
        }
    ) {
        test {
            issueId
            testType {
                name
            }
            gherkin
            jira(fields: ["key"])
        }
        warnings
    }
}

Keys

Values

explanation

testType

Type of test that will be created.

Possible values: Manual, Generic, Cucumber.

gherkin

Given.......

Gherkin test definition.

jira

fields

Setting that will enable the definition of native Jira fields, such as, summary, project, description, labels, etc

After making the request, the Test will be created and the response will provide the id in the Jira side that you can use to access it.

example of an API response

{
    "data": {
        "createTest": {
            "test": {
                "issueId": "10137",
                "testType": {
                    "name": "Cucumber"
                },
                "gherkin": "Given I have entered <input_1> into the calculator\nAnd I have entered <input_2> into the calculator\nWhen I press <button>\nThen the result should be <output> on the screen\n\n  Examples:\n    | input_1 | input_2 | button | output |\n    | 20      | 30      | add    | 50     |\n    | 2       | 5       | add    | 7      |",
                "jira": {
                    "key": "COM-125"
                }
            },
            "warnings": []
        }
    }
}

In Jira you can check the Test created.

Please find more information in the documentation.

Scenario2: Update Test(s) in Xray

Once you have used the first scenario to create Tests in Xray (linked to tests of the external tool), you need to assure that they are always showing the same information, for each time a change occurs in the external tool (edition of the test) you should update the Test in Xray side.

Full synchronisation is hard to achieve, so after creation we suggest to update the minimal information, such as description or summary. We trust that the details will be safe in the external tool.

In this case we're assuming:

Test exists in Xray and another Test object exists in the external tool (linked to this one)

And we want to...

Update the “same” test(s) in Xray because something changed in the Test (edited via the external tool)

Xray server/DC

As the API call is made on Jira server/DC REST API, we need to use its authentication mechanisms (e.g. basic authentication or Personal Access Tokens).

example of a API request with "curl" using basic authentication

curl -H "Content-Type: application/json" -X PUT -u jira_username:jira_password --data @data.json https://jiraserver.example.com/rest/api/2/issue/<test_issue_key>

example of a API request with "curl" using Personal Access Tokens

curl -H "Accept: application/json" -X PUT -H "Authorization: Bearer MDkzNjIyMTEwNjQzOqb+ApTGm3af+z3eGdcyDm7xHv6R" --data @data.json https://jiraserver.example.com/rest/api/2/issue/<test_issue_key>

JSON body payload

{
    "update" : {
        "description" : [{ "set": "[EXTOOL_1.2|http://www.example.com]"}]
    }
}

After making the request, the Test issue will have the our field (i.e. description) with a link to the test on the external tool.

Xray Cloud

As the API call is made on Jira Cloud REST API, we need to use a Jira Cloud API token created manually beforehand.

example of a API request with "curl" using basic authentication

JIRA_API_BASE_URL=<your_jira_cloud_url>/rest/api/2
JIRA_USERNAME=<jira_username>
JIRA_API_TOKEN=<jira_api_token>
TEST_ISSUE_KEY=<test_issue_key>

curl -H "Content-Type: application/json" -X PUT -u $JIRA_USERNAME:$JIRA_API_TOKEN   --data @"payload_cloud.json" $JIRA_API_BASE_URL/issue/$TEST_ISSUE_KEY

JSON body payload

{
    "update" : {
        "description" : [{ "set": "[EXTOOL_1.2|http://www.example.com]"}]
    }
}

After making the request, the Test issue will have the our field (i.e. "description") with a link to the test on the external tool.

Xray Server/DC

Update Test Steps

Xray Cloud

Tips

Use wiki markup to not only to format text but also to provide additional information or access to it

The Description field support Jira wiki markup, so you can use formatting to insert. a link for instance.

There can be some subtle differences between Jira server/DC and Jira Cloud, so we advise you to try it out on both deployment types.

What can it be used for?

Some ideas:

On the Description field,
- provide a link to the external tool, where users can see the correspondent Test in the external tool.
- provide some table, or some bullets, with some additional info about the context of the batch execution and that thus apply to all results (i.e. Test Runs)
- embed a image (e.g. a chart); image must be stored in some accessible URL

Use labels

When creating a new Test you have the ability to associate a label to it.

What can it be used for?

Some ideas:

This will make it possible to perform analysis over the origin of the Tests or to easily identify the ones that were created via automation to apply some action over those.
This will enable you to filter those Tests using Jira queries.

Implementation Checklist

For this purpose, please make sure your tool provides these:

Xray and Jira configuration
- Xray server/DC or Xray Cloud ? (which implicitly will define if Jira server/DC or Jira Cloud are being used)
- Server/DC specific settings
  - Use Jira username/password or Personal Access Token?
  - Jira username
  - Jira password
  - Personal Access Token
- Cloud specific settings
  - Jira username
  - Jira API Token
On each test on the tool, associate it to an existing Xray Test issue
1. specify Test issue key
Discover the custom fields ids of the entities being used

Xray internals

How is Xray data stored

Xray server/DC

Xray data is stored on Jira's database, mostly in Xray specific tables (e.g., for test runs).

As Xray reuses Jira concepts (e.g., issue types, links, custom fields), some of its information is stored on standard Jira entities/tables (e.g., Xray issues standard Jira metadata - summary, description, fixVersion, component, issue links used for coverage purpose).

Xray also provides a bunch of custom fields (CFs) to present or manage some information; some of these are calculated, some are stored in the database (on Jira tables for CFs).

Therefore, a Jira database's dump contains all relevant information from Jira and Xray.

Some of this information is exposed and managed using the Jira's REST API or through Xray's REST API.

Xray Cloud

Due to the way Atlassian cloud architecture works, apps such as Xray manage their data in their own way (e.g., in the app cloud infrastructure or in a cloud DB provider). Jira related information (e.g., issues, links) is managed by Atlassian and stored in Atlassian's cloud imfrastructure.

Some of this information is exposed and managed using different APIs:

Jira's REST API: for Jira managed information (e.g., issues, standard fields, CFs, links, attachments, comments)
Xray's REST API: mostly to import test results and other more "operational" tasks
Xray's GraphQL API: for most of Xray data and their relations; a quite flexible API

Depending on the use case, one or a combination of the previous may be required. Note that Jira's REST API uses the authentication mechanisms of Jira while Xray's APIs use their own authentication mechanism based on an API key pair (client_id+client_secret).

It's possible to generate a dump of the Xray managed data; note that this dump doesn't contain the information managed by Jira/Atlassian as that follows a different mechanism.

Custom fields

Generically speaking, custom fields (CFs) are like custom attributes. They exist as a means to add additional, well-defined information, to certain entities.

CFs can be seen as way to expose some internal information, to assign information, connected with some entities.

But whenever "custom fields" come to mind, we may be referring to different kinds of CFs (as Xray also provides some kind of custom fields), and this may cause some misunderstanding. These are the types of fields we may be referring to:

Jira's traditional "custom fields": In Jira terminology, custom fields (CFs) are a way to collect information that isn't available on the default fields. In other words, Jira CFs are a way to add additional metadata to issues.; we can think on them as issue level "attributes", having a id, name, type, and value. Jira CFs are defined at administration level (server/DC, cloud), and may be included on some screens, for certain issue types. These fields can easily be included on Jira standard gadgets, listings, etc. A field has a visual representation and a text representation. Some fields may be searchable and included on JQL queries. Some fields store data, while others are calculated. It's usual for Jira apps/plugins to create specific Jira CFs to present or manage some of their data as seen ahead.

Xray managed Jira CFs: Xray data managed as a Jira CF (server/DC, cloud); some of these CFs are read-only, while other are read/write. These are also Jira CFs in strict sense. Xray server/DC has many of these; Xray cloud has very few, due to the way apps integrate with Jira cloud. On Xray server/DC some of these fields are stored in Jira's database, others are stored on lucene.

Xray's Test step custom fields: Xray proprietary way of defining named attributes for steps on "manual" scripted tests; definable at project level. These are not listable/includable on Jira issues/gadgets nor exportable using standard Jira mechanisms; they are also not visible to Jira's REST API.
Xray's Test Run custom fields: Xray proprietary way of defining named attributes for Test Runs; definable at project level. These are not listable/includable on Jira issues/gadgets nor exportable using standard Jira mechanisms; they are also not visible to Jira's REST API.

	Xray server/DC		Xray Cloud
	read	update	read	update
Jira CFs (issue level)	Jira's REST API	Jira's REST API	Jira's REST API	Jira's REST API
Xray data managed as Jira CFs (issue level)	Jira's REST API	Jira's REST API	Jira's REST API	Jira's REST API
Xray test step CFs	on a given Test: Xray REST API v2 settings: N/A	on a given Test: Xray REST API v2 settings: N/A	on a given Test: Xray's GraphQL API settings: Xray's GraphQL API	on a given Test: Xray's GraphQL API settings: N/A
Xray's Test Run CFs	on a given test run: Xray REST API v2 settings: N/A	on a given test run: Xray REST API v2 settings: N/A	on a given test run: Xray's GraphQL API settings: Xray's GraphQL API	on a given test run: Xray's GraphQL API settings: N/A

Xray managed CF	Xray server/DC	Xray Cloud
Begin Date	x (rw)	x (rw)
Conditions	x (rw)	-
Cucumber Scenario	x (rw)	-
Cucumber Test Type	x (rw)	-
Dataset values	x	-
End Date	x (rw)	x (rw)
Generic Test Definition	x (rw)	-
Manual Test Steps	x (rw?)	-
Pre-Condition Type	x (rw)	-
Pre-Condition association with a Test	x (rw?)	-
Requirement Status	x (r)	-
Revision	x (rw)	x
Steps Count	x (r)	-
Test Type	x (rw?)	-
TestRunStatus	x (r)	-
Test Plans associated with a Test	x (rw?)	-
Test Repository Path	x (rw)	-
Test Sets associated with a Test	x (rw?)	-
Test Count	x (r)	-
Tests association with a Pre-Condition	x (rw?)	-
Tests associated with a Test Plan	x (rw?)	-
Tests association with a Test Set	x (rw?)	-
Tests association with a Test Execution	x (r)	-
Test Environments	x (rw)	-
Test Execution Defects	x (r)	-
Test Execution Status	x (r)	-
Test Set Status	x (r)	-
Test Plan Status	x (r)	-

Tips

try to use the proper APIs as much as possible and avoid using CFs to obtain or modify data

References

Xray Cloud
Xray server/DC

Recommendations, tips, and concerns

General integration tips

Provide links from your tool to Xray and the other way around. If facilitates navigation and access to information; we don't need to duplicate information in each tool.
- Example: if you're using an external tool to implement test automation and reports the results back to Xray, the Test Run details may contain a URL to the details on the external tool. That link may appear also at the Test Execution level, if it applies to all the tests run in that batch. On the external tool, a link could be provided so users can jump to Xray to see either the Test Execution or the details of a specific Test Run.
Jira instances are highly customizable. This means that Jira administrators may define some rules to make certain custom fields mandatory on certain issue types, such as on Test or Test Executions, which will impact certain API calls that create Jira issues (e.g. importing results)
- In this case, the "multipart" endpoints for importing results should be used as they allow us to specify fields for the Test Execution, and also for the Tests, that may be created.

Avoid performing many calls to achieve a certain flow

Xray Server/DC tips

Support basic authentication, using Jira username and password, but also support Personal Access Tokens, a recent feature that may be better suited for integration purposes

Xray Cloud tips

Authenticate once to obtain a token and use that token in all following requests; Xray Cloud tokens are valid for 1 day
Xray Cloud has a rate limit of 60 requests/minute. Please don't exceed this limit or else you'll receive a HTTP 429 status error code; in that case, the "Retry-After" HTTP header contains the number of seconds to wait until a new request can be made

Topics to cover/add in the future

Data-driven testing and datasets
use case for exporting data to BI tools and similar

Useful resources

Xray Postman Collections (Postman collections showcasing usage of APIs both for Xray server/DC and Xray Cloud, including REST and GraphQL examples)
RestMan (a Chrome extension very useful to try out REST API call)
Postman (API client with good support for REST APIs)
Insomnia (API client that provides good support for GraphQL)

References

Xray server/DC
- Jira server/DC REST API
- Xray server/DC REST API
- Xray usage tips for improving performance (see integration and REST API sections)
Xray Cloud

Content

Space Tools

Xray Technical Integration Kit

Overview

Background

Xray concepts

Xray server/DC vs Xray Cloud

Preparing a environment with Xray to test the integration

Integration possibilities

Xray server/DC

Jira's REST API and Xray REST API

Authentication

Xray Cloud

Authentication

Typical use cases

Importing test automation results to Xray

Examples

Scenario1: report test automation results to existing Tests in Xray

Xray server/DC

Xray Cloud

Scenario2: report test automation results to existing Tests in Xray, with additional step-like detail

Xray server/DC

Xray Cloud

Tips

Use wiki markup to not only to format text but also to provide additional information or access to it

Attach files (e.g. screenshots) as evidence

Implementation Checklist

Provide links on Xray entities to external entities to facilitate navigation

Examples

Scenario1: provide a link on Xray Test issues to the corresponding test in the external tool

Xray Cloud

Implementation Checklist

Provide links on external tools to Xray entities to facilitate navigation

Create/update test(s) in Xray

Examples

Scenario1: creating Test(s) in Xray

Xray server/DC

Manual Test

Generic

Cucumber

Xray Cloud

Manual Test

Generic Test

Cucumber Test

Scenario2: Update Test(s) in Xray

Xray server/DC

Xray Cloud

Tips

Use wiki markup to not only to format text but also to provide additional information or access to it

Use labels

Implementation Checklist

Xray internals

How is Xray data stored

Xray server/DC

Xray Cloud

Custom fields

Tips

References

Recommendations, tips, and concerns

General integration tips

Xray Server/DC tips

Xray Cloud tips

Topics to cover/add in the future

Useful resources

References