Page History

Versions Compared

Key

This line was added.
This line was removed.
Formatting was changed.

...

Your specification is made using Gherkin (i.e. Given, When, That) statements in Scenario(s) or Scenario Outline(s), eventually complemented with a Background. Implementation of each Gherkin statement (i.e. "step") is done in code; the Cucumber framework finds the code based on a regular or cucumber expressionexpressions.

Usage scenarios

Cucumber is used in diverse scenarios. Next you may find some usage patterns, even though Cucumber usage is mostly recommended only if you are adopting BDD.

Teams adopting BDD, start by defining a user story and clarify it using Cucumber Scenario(s); usualy, Cucumber Scenario(s)/Scenario Outline(s) are specified directly in Jira, using Xray
Teams adopting BDD but that favour a more Git based approach (e.g. GitOpGitOps). In this case, stories would be defined in Jira but Cucumber .feature files would be specified using some IDE and would be stored in Git, for example
Teams not adopting BDD but still using Cucumber, more as an automation framework. Sometimes focused on regression testing; sometimes, for non-regression testing. In this case, cucumber would be used...
1. With a user story or some sort of "requirement" described in Jira
2. Without any story/"requirement" described in Jira

...

Info

title	Learn more

Please see Testing in BDD with Gherkin based frameworks (e.g. Cucumber) for an overview of the possible workflows.

The place that you'll use to edit the Cucumber Scenarios will affect your workflow. There are teams that prefer to edit Cucumber Scenarios in Jira using Xray, while there others that prefer to edit them by writing the .feature files by hand using some IDE.

Requirements

Java
Add the dependency of cucumber-jvm (i.e. cucumber-java) to your maven "pom.xml" file

Examples

Example

For the purpose of this tutorial, we'll use a simple, dummy Calculator implemented in a Java class as our target for testing.

...

Code Block

language	java
title	src/main/java/com/xray/tutorials/Calculator.java
collapse	true

package com.xray.tutorials;

public class Calculator
{
// Square function
public static int Square(int num)
{
    return num*num;
}
// Add two integers and returns the sum
public static int Add(int num1, int num2 )
{
    return num1 + num2;
}
// Add two integers and returns the sum
public static double Add(double num1, double num2 )
{
    return num1 + num2;
}
// Multiply two integers and retuns the result... this code is buggy on purpose
public static int Multiply(int num1, int num2 )
{
    if ((num1==1) || (num2==1)) {
        return 0;
    } else {
        return num1 * num2;
    }
}



public static int Divide(int num1, int num2 )
{
    return num1 / num2;
}

// Subtracts small number from big number
public static int Subtract(int num1, int num2 )
{
    if ( num1 > num2 )
    {
    return num1 - num2;
    }
    return num2 - num1;
    }
}

This tutorial, has the following requirements:

Java
Add the dependency of cucumber-jvm (i.e. cucumber-java) to your maven "pom.xml" file

Using Using Jira and Xray as master

This section assumes using Xray as master, i.e. the place that you'll be using to edit the specifications (e.g. the scenarios that are part of .feature files).

...

To generate .feature file(s) based on Scenarios defined in Jira (i.e. Cucumber Tests and Preconditions), we can do it directly from Jira, by the REST API or using a CI tool; we'll see that ahead in more detail.

Example

All starts with a user story or some sort of “requirement” that you wish to validate. This is materialized as a Jira issue and identified by the corresponding issue key (e.g. CALC-7931).

...

After the Test is created, and since we have done it from the user story screen, it will impact the coverage of related "requirement", if any/story.

The coverage and the test results can be tracked in the "requirement" side (e.g. user story). In this case, you may see that coverage changed from being UNCOVERED to NOTRUN (i.e. covered and with at least one test not run).

...

use one of the available CI/CD plugins (e.g. see details of Integration with Jenkins)
or even

use the UI (e.g. from a Test issue)

Image Removed

REST API directly (more info here)

use the REST API directly (more info

here

)

Code Block

language	bash

#!/bin/bash
 
rm -f features/*.feature
curl -u admin:admin  "http://jiraserver.example.com/rest/raven/1.0/export/test?keys=CALC-7931;CALC-7935&fz=true" -o features.zip
unzip -o features.zip  -d features

... or even use the UI (e.g. from a Test issue)
- Image Added

We will export the features to a new directory named features/ on the root folder of your Java project (we'll need to tell Maven to use this folder).

...

After being exported, the created .feature(s) will contain references to the Test issue key, eventually prefixed (e.g. "TEST_") depending on an Xray global setting, and the covered "requirement" issue key, if that's the case. The naming of these files is detailed in Export Cucumber Features.

Code Block

title	features/1_CALC-7931.feature
collapse	true

@REQ_CALC-7931
Feature: As a user, I can calculate the sum of two numbers
	#As a user, I can calculate the sum of two numbers

	#Tests As a user, I can calculate the sum of two numbers
	@TEST_CALC-7934
	Scenario Outline: sum of two positive numbers
		Given I have entered <input_1> into the calculator
		And I have entered <input_2> into the calculator
		When I press <button>
		Then the result should be <output> on the screen
		
		  Examples:
		    | input_1 | input_2 | button | output |
		    | 20      | 30      | add    | 50     |
		    | 2       | 5       | add    | 7      |
		    | 0       | 40      | add    | 40     |
		    | 4       | 50      | add    | 54     |
		    | 5       | 50      | add    | 55     |	

	
	@TEST_CALC-7933
	Scenario: negative integer adition
		Given I have entered -1 into the calculator
		And I have entered 2 into the calculator
		When I press add
		Then the result should be 1 on the screen	

	#Tests As a user, I can calculate the sum of two numbers
	@TEST_CALC-7932
	Scenario: simple integer addition
		Given I have entered 1 into the calculator
		And I have entered 2 into the calculator
		When I press add
		Then the result should be 3 on the screen

...

To run the tests and produce a Cucumber JSON reports(s)report, we can run Maven and specify that we want a report in Cucumber JSON format and that it should process .features from the features/ directory.

...

After running the tests, results can be imported to Xray via the REST API, or the Import Execution Results action within an existing Test Execution, or by using one of the available CI/CD plugins (e.g. see an example of Integration with Jenkins).

...

Info

title	Which Cucumber endpoint to use?

To import results, you can use two different endpoints/"formats" (endpoints described in Import Execution Results - REST):

the "standard cucumber" endpoint
the "multipart cucumber" endpoint

The standard cucumber endpoint (i.e. /import/execution/cucumber) is simpler but more restrictive: you cannot specify values for custom fields on the Test Execution that will be created. This endpoint creates new Test Execution issues unless the Feature contains a tag having an issue key of an existing Test Execution.

The multipart cucumber endpoint will allow you to customize fields (e.g. Fix Version, Test Plan), if you wish to do so, on the Test Execution that will be created. Note that this endpoint always creates new Test Executions (as of Xray v4.2).

In sum, if you want to customize the Fix Version, Test Plan and/or Test Environment of the Test Execution issue that will be created, you'll have to use the "multipart cucumber" endpoint.

A new Test Execution will be created (unless you originally exported the Scenarios/Scenario Outlines from a Test Execution).

One of the tests fails (on purpose).

The execution screen details of the Test Run will provide overall status information and Gherkin statement-level results, therefore we can use it to analyze the failing test.

Image Modified

Results, including for each example on Scenario Outline, can be expanded to see all Gherkin statements.

Image Modified Image Modified

Note: in this case, the bug was added on purpose on the Calculator class.

Code Block

language	java
title	buggy Multiply() method in Calculator.java
collapse	true

public static int Multiply(int num1, int num2 )
{
    if ((num1==1) || (num2==1)) {
        return 0;
    } else {
        return num1 * num2;
    }
}

Info

title	Screenshots and other attachments

If available, it is possible to see also attached screenshot(s). For this, you'll need to use Cucumber's API and do it in a After hook, for example (using scenario.embed()).

The iconImage Added represents the evidences ("embeddings") for each Hook, Background and Steps.

Results are reflected on the covered items (e.g. Story issues) and can be seen in ther issue screen.

Coverage now shows that the addition related user story (e.g. CALC-7931) is OK based on the latest testing results; on the other hand, the multiplication related user story (CALC-7935) is NOK since it has one test currently failing.

Image Modified Image Modified

If we fix the code on the Calculator class, run the tests and import results, coverage for the multiplication related user story will be shown as OK.

Code Block

language	java
title	fix of Multiply() method in Calculator.java
collapse	true

public static int Multiply(int num1, int num2 )
{
	return num1 * num2;
}

Description

We will use the code from the Github repository "cucumber-java-skeleton", with slight changes in order to make the dummy Test pass.

The first step is to create a Cucumber Test, of Cucumber Type "Scenario", in Jira. The specification would be exactly the same as the one provided in the original repository.

After creating the Test in Jira and associating it with requirements, etc., you can export the specification of the test to a Cucumber .feature file via the REST API or the Export to Cucumber UI action from within the Test Execution issue.

The created file will be similar to the original, but will contain the references to the Test issue key and the covered requirement issue key.

Code Block

title	new feature after export /cucumber-java-skeleton/src/test/resources/skeleton/belly.feature

 @ABC-100
 Feature: Belly

  @ABC-122
  Scenario: a few cukes
    Given I have 42 cukes in my belly
    When I wait 1 hour
    Then my belly should growl

You can change the implementation of the steps in order to make them pass quickly.

Code Block

language	java
title	/cucumber-java-skeleton/src/test/java/skeleton/Stepdefs.java

package skeleton;
import cucumber.api.java.en.Given;
import cucumber.api.java.en.Then;
import cucumber.api.java.en.When;

public class Stepdefs {
    @Given("^I have (\\d+) cukes in my belly$")
    public void I_have_cukes_in_my_belly(int cukes) throws Throwable {
        Belly belly = new Belly();
        belly.eat(cukes);
    }
    
    
    @When("^I wait (\\d+) hour$")
    public void I_wait_hours(int hours) throws Throwable {
        Thread.sleep(hours*0);
    }
    
    @Then("^my belly should growl$")
    public boolean my_belly_should_growl() throws Throwable {
        return true;
    }
}

After running the tests and generating the Cucumber JSON report (e.g., data.json), it can be imported to Xray via the REST API or the Import Execution Results action within the Test Execution.

No Format
mvn compile test -Dcucumber.options="-p json:data.json"

The execution screen details will provide information on the test run result.

Image Removed

Info
The iconImage Removed represents the evidences ("embeddings") for each Hook, Background and Steps.

...

title	Learn more

...

References

...

Content

Space Tools

Versions Compared

Old Version 3

New Version 4

Key

Usage scenarios

Requirements

Examples

Example

Using Using Jira and Xray as master

Example

Description

References