Overview
In this tutorial, we will create some tests in Behave, which is a Cucumber variant for Python.
The test (specification) is initialy created in Jira as a Cucumber Test and afterwards, it is exported using the UI or the REST API.
We'll show how to use Behave JSON format and also how to generate a Cucumber JSON report, in case you need it.
Requirements
- Install Behave
- Install PyHamcrest
Code
Feature: Showing off behave (tutorial01) @ABC-119 Scenario: Run a simple test Given we have behave installed When we implement a test Then behave will test it for us!
# file:features/steps/step_tutorial01.py # ---------------------------------------------------------------------------- # STEPS: # ---------------------------------------------------------------------------- from behave import given, when, then @given('we have behave installed') def step_impl(context): pass @when('we implement a test') def step_impl(context): assert True is not False @then('behave will test it for us!') def step_impl(context): assert context.failed is False
After running the tests and generating the Behave JSON report (e.g., data.json), it can be imported to Xray via the REST API or the Import Execution Results action within the Test Execution.
behave --format=json -o data.json
The execution details displays the result of the Cucumber Scenario.
Learn more
See the available endpoints for importing Behave's results in Import Execution Results - REST.
Testing in BDD with Gherkin based frameworks (e.g. Cucumber) details the typical workflow for Cucumber-related tests.
Using Cucumber JSON reports
Please note
Cucumber JSON reports are supported by many tools, including some results parsers used by some CI tools. Besides it, as of Xray v3.1, the internal support for Cucumber JSON is more complete giving, for example, the ability to see step level information.
Behave does not provide, as of 2018, the ability to generate compatible Cucumber JSON reports. However, it provides the mechanism to use custom formatters. Thus, we can make our own implementation of a Cucumber JSON formatter.
The following code is based on a sample code provided by an open-source contributor "fredizzimo" (see original code here), with a small changes to make it handle correctly the JSON serialization of status results. You may create this cucumber_json.py
at the root of your project.
In this example, we'll use a demo.feature
file inspired in two Behave tutorials. The feature file needs to have the proper tags to the Test issue keys and, optionally, to the Test Execution in case you want to enfor the results to be submited to that same Test Executon. You may generate this feature from the UI of the Test Execution issue screen, by using the REST API.
@CALC-1958 Feature: demo @TEST_CALC-1957 Scenario Outline: Use Blender with <thing> Given I put "<thing>" in a blender When I switch the blender on Then it should transform into "<other thing>" Examples: Amphibians | thing | other thing | | Red Tree Frog | mush | | apples | apple juice | @TEST_CALC-1956 Scenario: Run a simple test Given we have behave installed When we implement a test Then behave will test it for us!
The corresponding steps implementation code lives in the following files.
After running the tests and generating the Cucumber JSON report (e.g., cucumber.json), it can be imported to Xray via the REST API or the Import Execution Results action within the Test Execution.
Running tests
behave --format=cucumber_json:PrettyCucumberJSONFormatter -o cucumber.json --format=json -o behave.json features/demo.feature
Import results via REST API
curl -H "Content-Type: application/json" -X POST -u user:password --data @cucumber.json https://sandbox.xpand-addons.com/rest/raven/1.0/import/execution/cucumber
The execution page provides detailed information, which in this case includes the results for the different examples along with the respective step results.