Page History

...

In this tutorial, we will create some tests in Behave, which is a Cucumber variant for Python.

The test (specification) is initially created in Jira as a Cucumber Test and afterward, it is exported using the UI or the REST API.

We'll show you how to use both the Behave JSON report format and also the Cucumber JSON report format, in case you need itBehave generates a Behave JSON report but we will use a custom formatter to generate a Cucumber JSON report that can be processed by Xray.

...

Behave (and Cucumber) can be used in diverse scenarios. Next, you may find some usage patterns, even though using Behave usage is mostly recommended only if you are adopting BDD.

1. Teams adopting BDD, start by defining a user story and clarify it using Scenario(s); usualyusually, Scenario(s)/Scenario Outline(s) are specified directly in Jira, using Xray
2. Teams adopting BDD but that favour favor a more Git-based approach (e.g. GitOps). In this case, stories would be defined in Jira but Behave .feature files would be specified using some IDE and would be stored in Git, for example
3. Teams not adopting BDD but still using Behave, more as an automation framework. Sometimes focused on regression testing; sometimes, for on non-regression testing. In this case, cucumber Cucumber would be used...
   1. With a user story or some sort of "requirement" described in Jira
   2. Without any story/"requirement" described in Jira

...

Before moving into the actual implementation, we you need to decide which workflow weyou'll use: do we you want to use Xray/Jira as the master for writing the declarative specification (i.e. the Gherkin based Scenarios), or do we you want to manage those outside using some editor and store them in Git, for example?

...

...

The test (specification) is initialy initially created in Jira as Cucumber Tests and afterwardsafterward, it is exported using the UI or the REST API.

This tutorial , has the following requirements:

...

To generate .feature file(s) based on Scenarios defined in Jira (i.e. Cucumber Tests and Preconditions), we can do it directly from Jira, by the REST API, or using a CI tool; we'll see that ahead in more detail.

Step-by-step

All Everthing starts with a user story or some sort of “requirement” that you wish to validate. This is materialized as a Jira issue and identified by the corresponding issue key (e.g. CALC-1206).

...

The coverage and the test results can be tracked in on the "requirement" side (e.g. user story). In this case, you may see that coverage changed from being UNCOVERED to NOTRUN (i.e. covered and with at least one test not run).

...

We repeat the process for additional "requirmentsrequirements" and/or test Scenarios.

The related statement's code is managed outside of Jira and stored in Git, for example.

...

You can then export the specification of the test to a Cucumber .feature file via the REST API, or the Xray - Export to Cucumber UI action from within the Test/Test Execution issue or even based on an existing saved filter. As a source, you can identify Test, Test Set, Test Execution, Test Plan, or "requirement" issues. A plugin for your CI tool of choice can be used to ease this task.

...

• use one of the available CI/CD plugins (e.g. see details of Integration with Jenkins; don't forget to define the issue keys or the filter id)
  • 
• use the REST API directly (more info here)

  • Code Block
    language bash title example of a shell script to export/generate .features from Xray collapse true
    #!/bin/bash token=$(curl -H "Content-Type: application/json" -X POST --data @"cloud_auth.json" https://xray.cloud.xpand-itgetxray.comapp/api/v2/authenticate| tr -d '"') curl -H "Content-Type: application/json" -X GET -H "Authorization: Bearer $token" "https://xray.cloud.xpand-itgetxray.comapp/api/v2/export/cucumber?keys=CALC-1206;CALC-1207" -o features.zip rm -rf features/*.feature unzip -o features.zip -d features

• ... or even use the UI (e.g. from a Test issue)
  • 

...

# file:features/steps/step_tutorial03.py
# ----------------------------------------------------------------------------
# STEPS:
# ----------------------------------------------------------------------------
from behave   import given, when, then
from hamcrest import assert_that, equal_to
from blender  import Blender

@given('I put "{thing}" in a blender')
def step_given_put_thing_into_blender(context, thing):
    context.blender = Blender()
    context.blender.add(thing)

@when('I switch the blender on')
def step_when_switch_blender_on(context):
    context.blender.switch_on()

@then('it should transform into "{other_thing}"')
def step_then_should_transform_into(context, other_thing):
    assert_that(context.blender.result, equal_to(other_thing))

Running testsTo

In order to run the tests and produce a Cucumber JSON report, we need to use a custom reporter.

Behave does not provide, as of v1.2.6, the ability to generate compatible Cucumber JSON reports out-of-the-box. However, it provides the mechanism to use custom formatters. Thus, we can make our own implementation of a Cucumber JSON formatter.

there 2 options available:

• Using the native Behave JSON (JSON pretty) report => recommended way
• Using a custom reporter that generates a compatible Cucumber JSON report

If you choose the latter, the The following code is based on a sample code provided by an open-source contributor "fredizzimo" (see original code here), with a small changes to make it handle correctly the JSON serialization of status results. You may create this cucumber_json126.py at the root of your project.

...

After running the tests and generating the Cucumber JSON Behave report, it can be imported to Xray via the REST API or  or the Xray - Import Execution Results action within the Test Execution.

BASE_URL=https://xray.cloud.xpand-itgetxray.comapp
token=$(curl -H "Content-Type: application/json" -X POST --data @"cloud_auth.json" "$BASE_URL/api/v1v2/authenticate"| tr -d '"')
curl -H "Content-Type: application/json" -X POST -H "Authorization: Bearer $token"  --data @"results/cucumberbehave.json" "$BASE_URL/api/v1/v2/import/execution/behave"

If we use the Cucumber JSON formatter instead, then the endpoint to be used needs to be changed accordingly.

BASE_URL=https://xray.cloud.getxray.app
token=$(curl -H "Content-Type: application/json" -X POST --data @"cloud_auth.json" "$BASE_URL/api/v2/authenticate"| tr -d '"')
curl -H "Content-Type: application/json" -X POST -H "Authorization: Bearer $token"  --data @"results/cucumber.json" "$BASE_URL/api/v2/import/execution/cucumber"

...