Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

In this tutorial, we will create some tests in Behave, which is a Cucumber variant for Python.

The test (specification) is initially created in Jira as a Cucumber Test and afterward, it is exported using the UI or the REST API.

We'll show you how to use both the Behave JSON report format and also the Cucumber JSON report format, in case you need itBehave generates a Behave JSON report but we will use a custom formatter to generate a Cucumber JSON report that can be processed by Xray.


Note
iconfalse
titleSource-code for this tutorial
typeInfo

Code is available in GiHub; the repo contains some additional tests beyond the scope of this tutorial and some auxiliary scripts.

...

  • use one of the available CI/CD plugins (e.g. see details of Integration with Jenkins; don't forget to define the issue keys or the filter id)
  • use the REST API directly (more info here)
    • Code Block
      languagebash
      titleexample of a shell script to export/generate .features from Xray
      collapsetrue
      #!/bin/bash
      
      token=$(curl -H "Content-Type: application/json" -X POST --data @"cloud_auth.json" https://xray.cloud.xpand-itgetxray.comapp/api/v2/authenticate| tr -d '"')
      curl -H "Content-Type: application/json" -X GET -H "Authorization: Bearer $token" "https://xray.cloud.xpand-itgetxray.comapp/api/v2/export/cucumber?keys=CALC-1206;CALC-1207" -o features.zip
      
      rm -rf features/*.feature
      unzip -o features.zip  -d features
  • ... or even use the UI (e.g. from a Test issue)

...

Code Block
languagepy
titlefeatures/steps/step_tutorial03.py
collapsetrue
# file:features/steps/step_tutorial03.py
# ----------------------------------------------------------------------------
# STEPS:
# ----------------------------------------------------------------------------
from behave   import given, when, then
from hamcrest import assert_that, equal_to
from blender  import Blender

@given('I put "{thing}" in a blender')
def step_given_put_thing_into_blender(context, thing):
    context.blender = Blender()
    context.blender.add(thing)

@when('I switch the blender on')
def step_when_switch_blender_on(context):
    context.blender.switch_on()

@then('it should transform into "{other_thing}"')
def step_then_should_transform_into(context, other_thing):
    assert_that(context.blender.result, equal_to(other_thing))


Running testsTo

In order to run the tests and produce a Cucumber JSON report, we need to use a custom reporter.

Behave does not provide, as of v1.2.6, the ability to generate compatible Cucumber JSON reports out-of-the-box. However, it provides the mechanism to use custom formatters. Thus, we can make our own implementation of a Cucumber JSON formatter.

there 2 options available:

  • Use the native Behave JSON (JSON pretty) report => recommended way
  • Or use a custom reporter that generates a compatible Cucumber JSON report

If you choose the latter, the The following code is based on a sample code provided by an open-source contributor "fredizzimo" (see original code here), with a small changes to make it handle correctly the JSON serialization of status results. You may create this cucumber_json126.py at the root of your project.

...

After running the tests and generating the Cucumber JSON Behave report, it can be imported to Xray via the REST API or  or the Xray - Import Execution Results action within the Test Execution.

Code Block
languagebash
titleexample of a Bash script to import results using the standard Cucumber Behave endpoint
collapsetrue
BASE_URL=https://xray.cloud.xpand-itgetxray.comapp
token=$(curl -H "Content-Type: application/json" -X POST --data @"cloud_auth.json" "$BASE_URL/api/v1v2/authenticate"| tr -d '"')
curl -H "Content-Type: application/json" -X POST -H "Authorization: Bearer $token"  --data @"results/cucumber.json" "$BASE_URL/api/v1/v2/import/execution/behave"


If we use the Cucumber JSON formatter instead, then the endpoint to be used needs to be changed accordingly.

Code Block
languagebash
titleexample of a Bash script to import results using the standard Cucumber endpoint
collapsetrue
BASE_URL=https://xray.cloud.getxray.app
token=$(curl -H "Content-Type: application/json" -X POST --data @"cloud_auth.json" "$BASE_URL/api/v2/authenticate"| tr -d '"')
curl -H "Content-Type: application/json" -X POST -H "Authorization: Bearer $token"  --data @"results/cucumber.json" "$BASE_URL/api/v2/import/execution/cucumber"

...