Page History
...
Your specification is made using Gherkin (i.e. Given, When, That) statements in Scenario(s) or Scenario Outline(s), eventually complemented with a Background. Implementation of each Gherkin statement (i.e. "step") is done in code; the Cucumber framework finds the code based on regular or cucumber expressions.
Note | ||||||
---|---|---|---|---|---|---|
| ||||||
Code is available in GiHub; the repo contains some auxiliary scripts. |
Usage scenarios
Cucumber is used in diverse scenarios. Next you may find some usage patterns, even though Cucumber usage is mostly recommended only if you are adopting BDD.
...
For the purpose of this tutorial, we'll use a simple, dummy Calculator implemented in a Java class as our target for testing.
Info | ||
---|---|---|
| ||
The code on this tutorial is available in the cucumber-java-calc GitHub repository. You can fork it and try it for youself. |
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
package com.xray.tutorials; public class Calculator { // Square function public static int Square(int num) { return num*num; } // Add two integers and returns the sum public static int Add(int num1, int num2 ) { return num1 + num2; } // Add two integers and returns the sum public static double Add(double num1, double num2 ) { return num1 + num2; } // Multiply two integers and retuns the result... this code is buggy on purpose public static int Multiply(int num1, int num2 ) { if ((num1==1) || (num2==1))0) { return num2; } else if (num2==0) { return num1; } else { return num1 * num2; } } public static int Divide(int num1, int num2 ) { return num1 / num2; } // Subtracts small number from big number public static int Subtract(int num1, int num2 ) { if ( num1 > num2 ) { return num1 - num2; } return num2 - num1; } } |
...
To generate .feature file(s) based on Scenarios defined in Jira (i.e. Cucumber Tests and Preconditions), we can do it directly from Jira, by the REST API or using a CI tool; we'll see that ahead in more detail.
...
Step-by-step
All starts with a user story or some sort of “requirement” that you wish to validate. This is materialized as a Jira issue and identified by the corresponding issue key (e.g. CALC-7931).
...
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
package calculator; import io.cucumber.java.en.Given; import io.cucumber.java.en.Then; import io.cucumber.java.en.When; import com.xray.tutorials.Calculator; import static org.junit.Assert.*; public class StepDefinitions { private Integer int1; private Integer int2; private Integer result; @Given("I have entered {int} into the calculator") public void i_have_entered_into_the_calculator(Integer int1) { this.int2 = this.int1; this.int1 = int1; } @When("I press add") public void i_press_add() { this.result = Calculator.Add(this.int1, this.int2); } @When("I press multiply") public void i_press_multiply() { this.result = Calculator.Multiply(this.int1, this.int2); } @Then("the result should be {int} on the screen") public void the_result_should_be_on_the_screen(Integer int1value) { // Write code here that turns the phrase above into concrete actions assertEquals((Integer)(this.int1 + this.int2)value, this.result); } } |
- the test runner is defined in the RunCucumberTest class. Cucumber options can be overriden from the command line, whenever executing Maven.
...
Code Block | ||
---|---|---|
| ||
mvn compile test -Dcucumber.plugin="json:report.json" -Dcucumber.features="features/" |
This will produce one Cucumber JSON report with all results.
...
Info | ||||
---|---|---|---|---|
| ||||
As the report format in Cucumber JSON is being deprecated in favour of Cucumber Messages, a protocol buffer based implementation, the previous command needs to be adapted slightly. The report starts by being generated in Cucumber Messages, using "-f message" argument, and then converted to the legacy Cucumber JSON report using the tool cucumber-json-formatter.
| ||||
title | example of a Bash script to import results using the standard Cucumber endpoint | |||
collapse | true | |||
curl -H "Content-Type: application/json" -X POST -u admin:admin --data @"report.json" http://jiraserver.example.com/rest/raven/1.0/import/execution/cucumber |
|
This will produce one Cucumber JSON report with all results.
After running the tests, results can be imported to Xray via the REST API, or the Import Execution Results action within an existing Test Execution, or by using one of the available CI/CD plugins (e.g. see an example of Integration with Jenkins).
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
curl -H "Content-Type: application/json" -X POST -u admin:admin --data @"report.json" http://jiraserver.example.com/rest/raven/1.0/import/execution/cucumber |
Info | ||
---|---|---|
| ||
To import results, you can use two different endpoints/"formats" (endpoints described in Import Execution Results - REST):
The standard cucumber endpoint (i.e. /import/execution/cucumber) is simpler but more restrictive: you cannot specify values | ||
Info | ||
| ||
To import results, you can use two different endpoints/"formats" (endpoints described in Import Execution Results - REST):
The standard cucumber endpoint (i.e. /import/execution/cucumber) is simpler but more restrictive: you cannot specify values for custom fields on the Test Execution that will be created. This endpoint creates new Test Execution issues unless the Feature contains a tag having an issue key of an existing Test Execution. The multipart cucumber endpoint will allow you to customize fields (e.g. Fix Version, Test Plan), if you wish to do so, on the Test Execution that will be created. Note that this endpoint always creates new Test Executions (as of Xray v4.2). In sum, if you want to customize the Fix Version, Test Plan and/or Test Environment of the Test Execution issue that will be created, you'll have to use the "multipart cucumber" endpoint. |
...
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
// Multiply two integers and retuns the result... this code is buggy on purpose public static int Multiply(int num1, int num2 ) { if ((num1==1) || (num2==1))0) { return 0num2; } else {if (num2==0) { return num1; } else { return num1 * num2; } } |
Info | ||
---|---|---|
| ||
If available, it is possible to see also attached screenshot(s). For this, you'll need to use Cucumber's API and do it in a After hook, for example (using The icon represents the evidences ("embeddings") for each Hook, Background and Steps. |
...
To import .features to Jira we can either use the REST API or a CI tool. To export tagged .features from Jira, we can do it directly from Jira, by the REST API or using a CI tool; we'll see that ahead in more detail.
...
Step-by-step
All starts with a user story or some sort of “requirement” that you wish to validate. This is materialized as a Jira issue and identified by the corresponding issue key (e.g. CALC-7931).
...
Having those to guide testing, we could then describe and implement the Cucumber test scenarios using our favourite IDE.
The related statement's code is managed outside of Jira and stored in Git, for example.
...
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
package calculator; import io.cucumber.java.en.Given; import io.cucumber.java.en.Then; import io.cucumber.java.en.When; import com.xray.tutorials.Calculator; import static org.junit.Assert.*; public class StepDefinitions { private Integer int1; private Integer int2; private Integer result; @Given("I have entered {int} into the calculator") public void i_have_entered_into_the_calculator(Integer int1) { this.int2 = this.int1; this.int1 = int1; } @When("I press add") public void i_press_add() { this.result = Calculator.Add(this.int1, this.int2); } @When("I press multiply") public void i_press_multiply() { this.result = Calculator.Multiply(this.int1, this.int2); } @Then("the result should be {int} on the screen") public void the_result_should_be_on_the_screen(Integer int1value) { // Write code here that turns the phrase above into concrete actions assertEquals((Integer)(this.int1 + this.int2)value, this.result); } } |
- the test runner is defined in the RunCucumberTest class. Cucumber options can be overriden from the command line, whenever executing Maven.
...
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
rm -f features.zip zip -r features.zip src/test/resources/calculator/ -i \*.feature curl -H "Content-Type: multipart/form-data" -u admin:admin -F "file=@features.zip" "http://jiraserver.example.com/rest/raven/1.0/import/feature?projectKey=CALC" |
...
Info | ||
---|---|---|
| ||
Each Scenario of each .feature will be created as a Test issue that contains unique identifiers, so that if you import once again then Xray can update the existent Test and don't create any duplicated tests. See Importing Cucumber Tests - REST for details on how it works. |
You can then export the specification of the test to a Cucumber .feature file via the REST API, or the Export to Cucumber UI action from within the Test/Test Execution issue or even based on an existing saved filter. As source, you can identify Test, Test Set, Test Execution, Test Plan or "requirement" issues. A plugin for your CI tool of choice can be used to ease this task.
...
Code Block | ||||
---|---|---|---|---|
| ||||
@REQ_CALC-7935
Feature: As a user, I can multiply two numbers
#As a user, I can multiply two numbers
#simple integer multiplication
@TEST_CALC-7936
Scenario: simple integer multiplication
Given I have entered 3 into the calculator
And I have entered 0 into the calculator
When I press multiply
Then the result should be 0 on the screen |
...
screen |
To run the tests and produce a Cucumber JSON report, we can run Maven and specify that we want a report in Cucumber JSON format and that it should process .features from the features/
directory.
Code Block | ||
---|---|---|
| ||
mvn compile test -Dcucumber.plugin="json:report.json" -Dcucumber.features="features/" |
Info | |||||||
---|---|---|---|---|---|---|---|
| |||||||
As the report format in Cucumber JSON is being deprecated in favour of Cucumber Messages, a protocol buffer based implementation, the previous command needs to be adapted slightly. The report starts by being generated in Cucumber Messages, using "-f message" argument, and then converted to the legacy Cucumber JSON report using the tool cucumber-json-formatter.
|
This will produce one Cucumber JSON report with all results.
...
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
// Multiply two integers and retuns the result... this code is buggy on purpose public static int Multiply(int num1, int num2 ) { if ((num1==1) ||0) { return num2; } else if (num2==10)) { return 0num1; } else { return num1 * num2; } } |
...
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
public static int Multiply(int num1, int num2 ) { return num1 * num2; } |
FAQ and Recommendations
Please see this page.
References
- Code used in this tutorial, along with some auxiliary scripts
- Sample project cucumber-java-skeleton
- Official Cucumber documentation
- Cucumber installation instructions for Java
- Cucumber API
- Cucumber expressions
- Testing in BDD with Gherkin based frameworks (e.g. Cucumber)
- Automated Tests (Import/Export)
- Exporting Cucumber Tests - REST
...