Overview

Boozang is a codeless testing tool that enables you to define and execute UI/API automated tests without the need to code them. It also has support for Cucumber tests and has a modular approach towards testing.

Integration with CI/CD tools is also possible through the scripts generated in Boozang to be used in your CI/CD tool.

More details about Boozang here.

Main features

This integration provides:

• Integration with Xray cloud or Server/DC
• Ability to export Cucumber tests from Xray to Boozang
• Automate the tests in Boozang
• Import the automation results back to Xray 

Mapping of concepts

Flow

If you are using Xray as the source of truth for your Cucumber tests you will be defining your Cucumber tests, writing those in Gherkin, in Xray and then you will import that specification into Boozang to automate the steps and execute them.

Once the execution is done you will import the test results back to Xray, so the flow will be the below one:

Setup

The first step that you need is to have Cucumber Tests defined in Xray, if you do not have those yet defined please follow this document in order to add some Cucumber tests in Xray.

The second step is to define a JQL query, in Jira/Xray, to select those tests in order to import them to Boozang (the integration is based on this filter), to do so select "Search for issues" from the "Filters" entry in the top menu

Introduce the query that will allow you to select the Cucumber Tests that you want to export to Boozang (in order to automate those), in my case it will look like this:

Once you have the filter defined save it using the "Save as" option.

Provide a name and save the id that Jira will associate to your filter (important as we will use this id in the integration with Boozang to import the Cucumber Tests)

At this stage we have defined the Cucumber Tests in Xray and created a filter to extract those fields, so it is now time to switch to the Boozang application.

Access Boozang (we are using the Europe region but use the one that makes sense for you) and create a new project where we will create our test automation, in my case I created the Xray project, next I will open the Boozang tool by pressing in the "Launch Tool" option:

This will open the tool (actually the first time you press this option it will take you to the installation option of the Boozang AI Chrome extension, after you have installed it, it will open the tool right away)

In order to configure the integration with Xray we must access the main page and select the configuration option in the bottom left and then configure the Feature File Server by pressing the configuration icon next to it

This will open a pop up to configure the different integrations available in Boozang, in our case we are going to choose the "Jira/Xray" option

In more detail:

• Type: Choose Jira/Xray option
• File List Url: Replace by your Jiran instance URL plus "/rest/raven/1.0/export/test?filter" and the JQL filter that we have saved in Jira/Xray
• Token: Insert the usernam and password for your Jira instance
• Client ID: Leave blank
• Client Secret: Leave blank
• Match File: Leave the "*.feature"

You can press Check to validate that the configuration is ok, once everything is done press Done.

Now get back to the main page of Boozang application by pressing the first option in the left menu

Import Xray Cucumber Tests into Boozang

Now that the configuration is set we will export the Cucumber tests from Xray and import them into Boozang, to do so use the option in the right menu (as shown in the screenshot below) and choose "Import Features" option in the pop up that appears

Another pop up will appear to choose the import type, in our case we will choose "Sync from server" and press Load

At this stage Boozang will connect to Xray and list all the requirements that are covered by the Cucumber tests present in the filter you have provided, in my case my two Cucumber Tests are covering the XT-45 User Story

So in the pop up it will show a preview of what is the requirement from where it will import the Tests from

When we press "Start" Boozang will show you the items it will create and proceed with the creation of the features.

Now that we have the features imported into Boozang, we need to implement the code that will be executed to perform the actual validations, let's get back to the main page and check the Features created

When I press the Feature it will take me to the actual Tests that where imported and if I click on either of them I will taken to the detail view of the Tests, where I can see that the steps need to be automated (in red in the screenshot)

Automate Cucumber Tests in Boozang

I will describe one of the ways to automate the tests and link that automation to the steps in Boozang but you must check the Boozang documentation to see what other ways may be available that are more suited to your needs, what matters for this example is to have the steps automated so that they can be executed and produce a report of the execution to be imported to Xray.

My approach was to create new Modules with a test in them that will represent each step in the Cucumber Test in order to be able to link those to each step and reuse them in both Tests.

For the creation of each Test a used the record capability of Boozang, where I defined the automated steps and the validations that I required for each Test.

Once the Modules and Tests are ready we need to link those to each Step in the Cucumber Test, for that we need to go into the details of each Cucumber Test and click in the "NOT IMPLEMENTED! -..." steps, that will load, in the upper part of the application, in the Main submenu of the "Action Setting" tab.

Clicking on the "Goto Test" entry we will have the possibility to choose, from the dropdown list, which Test will be associated to this step.

The final result looks like the following, where all the steps have automated tests associated and are ready to be executed.

Import results to Xray

Finally the Tests we have exported from Xray into Boozang are automated and now we need the results from the executions to be sent back to Xray, to do that we will use the CI part of the Boozang tool, in this are we will be able to generate the necessary scripts, for different CI/CD tools, that will execute the Boozang Test Runner.

In Boozang access the CI entry present in the bottom of the left menu

Jenkins

In our case, and to take advantage of the Xray Jenkins Plugin available, we generate the script needed to include in Jenkins by clicking the bottom link "Get Jenkins config".

A new pop up will appear requiring details to generate the script

The majority of the options will be the default ones, we are going to choose the "Test case" to be the feature we have automated (in the drop down list)

Fill in the password, click "Submit" and click on "Generate", this will produce the necessary script that we will copy and paste into Jenkins.

In Jenkins we created a simple Project where we add one step to insert the script generated, more specifically by adding a "Execute shell" in the build step section and inserting the generated code there

cucumber-json-merge-multiworker report_cucumber-m3-t1.json report_cucumber-m3-t2.json

The last step before executing the pipeline is to add the Xray Plugin to import the results back to Xray, to do so make sure you have previously installed the plugin, and in your pipeline you just need to add in the Post Build Actions, the "Xray: Results Import Task" and configure properly with:

• the Jira/Xray instance that you will use to upload the results
• the Format of the results file you want to import, in our case Cucumber JSON
• The path and name of the file to be uploaded

Once you build this pipeline it will execute the tests in Boozang and import the results into Xray as you can verify in the "Console Output" of the build:

We can see the response from Xray, where we can find the Test Execution created for this execution.

In Xray, if we verify the Test Execution that was created we can see the uploaded results with the total number of Tests and the Overall Execution Status (in this case not really relevant because we only have one Test but if we had several Tests it will the overall result of all the Tests)

To have a better understanding of the details of the execution we can press the details button (one below the red arrow) that will take us to the execution panel of the Test Execution, in there we can see the details of the execution:

• The requirement that is covered by the Test, in this case XT-45
• The scenario definition
• The details of the execution of each step

Command line

If Jenkins (or another CI/CD tool) is not part of your workflow you can use the Xray API to send the result back to Xray through the command line, to do so you must take two steps:

• Authenticate to obtain the token
• Send the results to Xray (using the token obtained)

Authenticate 

The authentication int he Server/DC version is done using the username and password in every request, so no special step to perform before executing the import request.

Send results to Xray

Finally to upload the results back into Xray you must use the following request

curl -H "Content-Type: application/json" -X POST -u admin:admin --data @‘xrayResults.json’ ‘https://xray-demo3.xpand-it.com/api/v2/import/execution'

When successful the answer will have information of the Test Execution created.

In Xray we can see the detailed results

Learn more

• Boozang home page
• Boozang features overview