Overview

Slack is a popular communication platform adopted by many development and non-development teams.

Teams use it daily to collaborate, make calls, share information, ask for help, and be notified of certain events to increase visibility of the status of the projects they're working on.

In this article, we'll see an everyday use case: sharing test results with the team in their collaboration tool (i.e., "Slack"). However, Xray and Jira's built-in automation capabilities can implement many more use cases.


Prerequisites

To integrate with Slack, we can set up an incoming webhook in a channel and send notifications (i.e., "messages").

  1. Go to Slack API site to create a new app 
  2. Name your app (e.g., "Xray Notifications") and select a workspace where to install it
  3. Add "Incoming Webhooks" feature and enable it
  4. Add the webhook to your workspace and associate it with a channel
  5. Copy the generated webhook URL and keep it safe
  6. At any time, under the "Basic Information" on the application settings, you can change the name and associate a image with the app by uploading an image
    1. Note: You can use the following image containing the Xray logo
      1.  
      2.  


Use cases

Share test results to a channel

This common use-case is about sharing information about test results, usually from test automation, to a Slack channel so the team can be aware of testing progress and analyze the results if needed.



At high-level, we will:

  1. make an authentication request to Xray's API to get an authentication token to be used on the GraphQL query
  2. make a GraphQL query to obtain the Test Execution details, including recorded test results
  3. process the response, storing relevant information on some variables
  4. build a HTTP request to send a notification with the test results summary by invoking a webhook on Slack

Step-by-step implementation with Jira automation

  1. .In Jira's project settings, under Automation, create the Jira automation rule
    1. define the trigger; we can have a manual trigger, that will provide an action available from the issue screen, or a trigger based on the creation of a specific issue type. In this case, it would probably make most sense to have a trigger based on the transition to some status (e.g., "Done") . We can also restrict this to the issue types "Test Execution" and "Sub Test Execution" using an IF condition.
    2. create an action using the "Send web request" template to make the authentication request and obtain a token to be used on the GraphQL queries; we need a client_id and a client_secret (please see how to create API keys).
    3. save the response on a variable (e.g., "token")
    4. make a GraphQL query to obtain the details of the Test Execution and its results
      1. Use the GraphQL API, namely the getTestExecution function. We'll need the issue id of the Test Execution that was triggered; for that, we can obtain the smart values feature from Jira automation. We need to also obtain the custom field id of the "Revision" custom field under the "Issue Fields" section of your Jira administration. 
        1. sample GraphQL query
          {
              getTestExecution(issueId: "{{issue.id}}") {
          				
                  issueId
                  jira(fields: ["key","customfield_10033"])
              
          				
                    testRuns(limit: 100){
                      results{
                        status{
                          name
                        }
            
                        test {
                            jira(fields: ["key"])
                              }
          							
                        }
                  }
          		
          				testEnvironments
          				testPlans(limit: 1) {
          					results{
          							jira(fields: ["key"])
          					}
          			
          				}
          
              }
          }

           

      2. Escape the GraphQL query (e.g., using the online GraphQL to JSON Body Converter tool)
  2. get the total tests and store it in a variable(note that this is an approximate value as GraphQL results can be limited and paginated)
    1. {{webResponse.body.data.getTestExecution.testRuns.results.size|0}} 
  3. get the total tests passing and store it in a variable (note that this is an approximate value as GraphQL results can be limited and paginated)
    1. {{webResponse.body.data.getTestExecution.testRuns.results.status.name.match(".*(PASSED).*").size|0}} 
  4. get the total failed tests and store it in a variable (note that this is an approximate value as GraphQL results can be limited and paginated)
    1. {{webResponse.body.data.getTestExecution.testRuns.results.status.name.match(".*(FAILED).*").size|0}} 
  5. get the total tests in TO DO and store it in a variable (note that this is an approximate value as GraphQL results can be limited and paginated)
    1. {{webResponse.body.data.getTestExecution.testRuns.results.status.name.match(".*(TO DO).*").size|0}} 
  6. get the total tests in other statuses and store it in a variable (note that this is an approximate value as GraphQL results can be limited and paginated)

    1. {{#=}}{{totalTests}} - {{passedTests}} - {{failedTests}} - {{todoTests}}{{/}}
  7. get the linked Test Plan and store it in a variable
    1. {{webResponse.body.data.getTestExecution.testPlans.results.jira.key}}
  8. Use a "IF" block to store the URL of the linked Test Plan, if any
  9. get the associated Test Environments and store it in a variable
    1. {{webResponse.body.data.getTestExecution.testEnvironments.join(",")}}
  10. get the list of distinct statuses reported and store it in a temporary variable
    1. {{webResponse.body.data.getTestExecution.testEnvironments.join(",")}}
  11. make a IF THEN ELSE block to adjust the text message and image of the notification, to tailor our notification for successful and unsuccessful testing cases
    1. As a mere example, for the "failure" image we can use the following one.

    2. https://docs.getxray.app/s/e1tqew/8402/f0863dd17de361916f7914addff17e0432a0be98/_/images/icons/emoticons/error.png
    3. As a mere example, for the "success" image we can use the following one.

    4. https://docs.getxray.app/s/e1tqew/8402/f0863dd17de361916f7914addff17e0432a0be98/_/images/icons/emoticons/check.png
  12. send the notification by using a "Send web request" to invoke the webhook on Slack; use the webhook URL that was configured on Slack
    1. {
      	"blocks": [
      		{
      			"type": "context",
      			"elements": [
      				{
      					"type": "mrkdwn",
      					"text": "Test results were reported for project *{{project.name}}*"
      				}
      			]
      		},
      		{
      			"type": "context",
      			"elements": [
      				{
      					"type": "mrkdwn",
      					"text": "{{issue.summary}}"
      				}
      			]
      		},
      		{
      			"type": "section",
      			"text": {
      				"type": "mrkdwn",
      				"text": "{{notificationMsg}}"
      			},
      			"accessory": {
      				"type": "image",
      				"image_url": "{{notificationImageUrl}}",
      				"alt_text": "test results"
      			}
      		},
      		{
      			"type": "divider"
      		},
      		{
      			"type": "section",
      			"text": {
      				"text": "Test Execution details, including its Test Runs",
      				"type": "mrkdwn"
      			},
      			"fields": [
      	
                       {
                    		"type": "mrkdwn",
                          "text": "*Test Execution*\n<{{issue.url}}|{{issue.key}}>"
                        }, {
                          "type": "mrkdwn",
                            "text": "*Version*\n{{issue.fixVersions.name}}"
                        }, {
                          "type": "mrkdwn",
                            "text": "*Revision*\n{{issue.Revision}}"
                        }, {
                          "type": "mrkdwn",
                            "text": "*Test Environment(s)*\n{{testEnvironments}}"
                        }, {
                          "type": "mrkdwn",
                            "text": "*Test Plan*\n{{testPlanSlackUrl}}"
                        }, {
                          "type": "mrkdwn",
                            "text": "*Total tests*\n{{totalTests}}"
                        }, {
                          "type": "mrkdwn",
                            "text": "*Passed tests*\n{{passedTests}}"
                        }, {
                          "type": "mrkdwn",
                            "text": "*Failed tests*\n{{failedTests}}"
                        }, {
                          "type": "mrkdwn",
                            "text": "*To Do tests*\n{{todoTests}}"
                        }, {
                          "type": "mrkdwn",
                            "text": "*Other tests*\n{{otherTests}}"
                        }
                    
      			]
      		}
      	]
      }

Example of the output

Further ideas to try out

  • send the notification only if there are test failures
  • send the notification only if there is a specific label or custom field on the Test Execution
  • add comments and/or logs of the failed tests to the notification

Tips

  • Use Slack's Block Kit Builder tool to sketch the layout of the notification and the respective payload you need to send to Slack's webhook endpoint 

References



  • No labels