Overview

Slack is a popular communication platform adopted by many development and non-development teams.

Teams use it daily to collaborate, make calls, share information, ask for help, and be notified of certain events to increase visibility of the status of the projects they're working on.

In this article, we'll see an everyday use case: sharing test results with the team in their collaboration tool (i.e., "Slack"). However, Xray and Jira's built-in automation capabilities can implement many more use cases.

XRAY 7.7.0 > Integration with Slack > image2023-8-14_11-29-4.png

Prerequisites

To integrate with Slack, we can set up an incoming webhook in a channel and send notifications (i.e., "messages").

Go to Slack API site to create a new app
Name your app (e.g., "Xray Notifications") and select a workspace where to install it
Add "Incoming Webhooks" feature and enable it
Add the webhook to your workspace and associate it with a channel
Copy the generated webhook URL and keep it safe
At any time, under the "Basic Information" on the application settings, you can change the name and associate a image with the app by uploading an image
1. Note: You can use the following image containing the Xray logo

Use cases

Share test results to a channel

This common use-case is about sharing information about test results, usually from test automation, to a Slack channel so the team can be aware of testing progress and analyze the results if needed.

At high-level, we will:

obtain the Test Execution details, including recorded test results, using information stored as JSON on the "Test Execution Status" custom field of the corresponding Test Execution issue;
build an HTTP request to send a notification with the test results summary by invoking a webhook on Slack

Step-by-step implementation with Jira automation

In Jira's project settings, under Automation, create the Jira automation rule

define the trigger; we can have a manual trigger that will provide an action from the issue screen or a trigger based on creating a specific issue type. In this case, it would probably make the most sense to have a trigger based on the transition to some status (e.g., "Done"). We can also restrict this to the issue types "Test Execution" and "Sub-Test Execution" using an IF condition.

Xray provides an automatic mechanism to transition Test Execution issues if all of its tests have a final status (i.e., if all tests have reported results). This is useful whenever uploading test automation results from a CI/CD pipeline. Please read the available configuration options to enable this under the Miscellaneous tab in Xray global settings.

make a IF THEN ELSE block to adjust the text message and image of the notification, to tailor our notification for successful and unsuccessful testing case
1. We can compare the number of tests that passed vs the total number of tests on the Test Execution, to identify wether all test results were successful or not. We'll use information from the "Test Execution Status" custom field that is associated with the Test Execution issue; if you want to see what's within this field, you can use the "Add value to the audit log"
  3. {{Test Execution Status.statuses.get(0).statusCount}} ... {{Test Execution Status.count}}
2. Note: we could also do it by looking at the number of failed tests using the expression {{Test Execution Status.statuses.get(1).statusCount)}}

send the notification by using a "Send web request" to invoke the webhook on Teams (two requests, one for success and another if tests failed)

If not all tests passed, we can send the following content.

XRAY 7.7.0 > Integration with Slack > image2023-8-14_11-32-31.png

{
	"blocks": [
		{
			"type": "context",
			"elements": [
				{
					"type": "mrkdwn",
					"text": "Test results were reported for project *{{project.name}}*"
				}
			]
		},
		{
			"type": "context",
			"elements": [
				{
					"type": "mrkdwn",
					"text": "{{issue.summary}}"
				}
			]
		},
		{
			"type": "section",
			"text": {
				"type": "mrkdwn",
				"text": "Some tests reported by {{issue.reporter.displayName}} failed. Please check the results."
			},
			"accessory": {
				"type": "image",
				"image_url": "https://docs.getxray.app/s/e1tqew/8402/f0863dd17de361916f7914addff17e0432a0be98/_/images/icons/emoticons/error.png",
				"alt_text": "test results with failures"
			}
		},
		{
			"type": "divider"
		},
		{
			"type": "section",
			"text": {
				"text": "Test Execution details, including its Test Runs",
				"type": "mrkdwn"
			},
			"fields": [
	
                 {
              		"type": "mrkdwn",
                    "text": "*Test Execution*\n<{{issue.url}}|{{issue.key}}>"
                  }, {
                    "type": "mrkdwn",
                      "text": "*Version*\n{{issue.fixVersions.name}}"
                  }, {
                    "type": "mrkdwn",
                      "text": "*Revision*\n{{issue.Revision}}"
                  }, {
                    "type": "mrkdwn",
                      "text": "*Test Environment(s)*\n{{issue.Test Environments}}"
                  }, {
                    "type": "mrkdwn",
                      "text": "*Test Plan*\n<{{baseUrl}}/browse/{{issue.Test Plan}}|{{issue.Test Plan}}>"
                  }, {
                    "type": "mrkdwn",
                      "text": "*Total tests*\n{{Test Execution Status.count}}"
                  }, {
                    "type": "mrkdwn",
                      "text": "*Passed tests*\n{{Test Execution Status.statuses.get(0).statusCount}}"
                  }, {
                    "type": "mrkdwn",
                      "text": "*Failed tests*\n{{Test Execution Status.statuses.get(1).statusCount}}"
                  }, {
                    "type": "mrkdwn",
                      "text": "*Other tests*\n{{#=}}{{Test Execution Status.count}} - {{Test Execution Status.statuses.get(0).statusCount}} - {{Test Execution Status.statuses.get(1).statusCount}}{{/}}"
                  }
              
			]
		}
	]
}

If all tests passed, within the "ELSE" section of the IF block, we can send the following content.

XRAY 7.7.0 > Integration with Slack > image2023-8-14_11-34-56.png

{
	"blocks": [
		{
			"type": "context",
			"elements": [
				{
					"type": "mrkdwn",
					"text": "Test results were reported for project *{{project.name}}*"
				}
			]
		},
		{
			"type": "context",
			"elements": [
				{
					"type": "mrkdwn",
					"text": "{{issue.summary}}"
				}
			]
		},
		{
			"type": "section",
			"text": {
				"type": "mrkdwn",
				"text": "All tests reported by {{issue.reporter.displayName}} passed. Nothing to worry about."
			},
			"accessory": {
				"type": "image",
				"image_url": "https://docs.getxray.app/s/e1tqew/8402/f0863dd17de361916f7914addff17e0432a0be98/_/images/icons/emoticons/check.png",
				"alt_text": "test results without failures"
			}
		},
		{
			"type": "divider"
		},
		{
			"type": "section",
			"text": {
				"text": "Test Execution details, including its Test Runs",
				"type": "mrkdwn"
			},
			"fields": [
	
                 {
              		"type": "mrkdwn",
                    "text": "*Test Execution*\n<{{issue.url}}|{{issue.key}}>"
                  }, {
                    "type": "mrkdwn",
                      "text": "*Version*\n{{issue.fixVersions.name}}"
                  }, {
                    "type": "mrkdwn",
                      "text": "*Revision*\n{{issue.Revision}}"
                  }, {
                    "type": "mrkdwn",
                      "text": "*Test Environment(s)*\n{{issue.Test Environments}}"
                  }, {
                    "type": "mrkdwn",
                      "text": "*Test Plan*\n<{{baseUrl}}/browse/{{issue.Test Plan}}|{{issue.Test Plan}}>"
                  }, {
                    "type": "mrkdwn",
                      "text": "*Total tests*\n{{Test Execution Status.count}}"
                  }, {
                    "type": "mrkdwn",
                      "text": "*Passed tests*\n{{Test Execution Status.statuses.get(0).statusCount}}"
                  }, {
                    "type": "mrkdwn",
                      "text": "*Failed tests*\n{{Test Execution Status.statuses.get(1).statusCount}}"
                  }, {
                    "type": "mrkdwn",
                      "text": "*Other tests*\n{{#=}}{{Test Execution Status.count}} - {{Test Execution Status.statuses.get(0).statusCount}} - {{Test Execution Status.statuses.get(1).statusCount}}{{/}}"
                  }
              
			]
		}
	]
}

Example of the output

XRAY 7.7.0 > Integration with Slack > image2023-8-14_11-28-52.png

Further ideas to try out

send the notification only if there are test failures
send the notification only if there is a specific label or custom field on the Test Execution
add comments and/or logs of the failed tests to the notification

Tips

Use Slack's Block Kit Builder tool to sketch the layout of the notification and the respective payload you need to send to Slack's webhook endpoint
To get information about test results using the "Test Execution Status" custom field, the scripts above use the "Test Execution Status" custom field that has an internal representation about the overall testing progress. Note that this internal representation may change in the future. The table below presents some information you may obtain and how to perform it using Jira automation smart values/expressions. Please see the code snippets above to see how additional information.

Information to obtain	Jira Automation smart values	Notes
Total Tests	`{{Test Execution Status.count}}`
Number of tests passing (i.e., reported as "Pass")	`{{Test Execution Status.statuses.get(0).statusCount}}`	A more accurate expression would be: `{{#issue.Test Execution Status.statuses}}{{#if(equals(name,"PASS"))}}{{statusCount}}{{/}}{{/}}`
Number of failed tests (i.e., reported as "FAIL")	`{{Test Execution Status.statuses.get(1).statusCount}}`	A more accurate expression would be: `{{#issue.Test Execution Status.statuses}}{{#if(equals(name,"FAIL"))}}{{statusCount}}{{/}}{{/}}`
Test Environments	`{{issue.Test Environments}}`
Linked Test Plan(s)	`{{issue.Test Plan}}`