Overview

In this tutorial, we will create some UI tests as Cucumber Scenario(s)/Scenario Outline(s) and use Cypress to implement the tests in JavaScript.

Requirements

• nodejs
• npm packages
  • cypress
  • cypress-cucumber-preprocessor
  • cucumber-json-merge

Description

For the purpose of this tutorial, we'll use a dummy website (source-code here) containing just a few pages to support login/logout kind of features; we aim to test precisely those features.

Before moving into the actual implementation, we need to decide is which workflow we'll use: do we want to use Xray/Jira as the master for writing the declarative specification (i.e. the Gherkin based Scenarios), or do we want to manage those outside using some editor and store them in Git, for example?

Using Jira and Xray as master

This section assumes using Xray as master, i.e. the place that you'll be using to edit the specifications (e.g. the scenarios that are part of .feature files).

The overall flow would be something like this:

1. create Scenario/Scenario Outline as a Test in Jira; usually, it would be linked to an existing "requirement"/Story (i.e. created from the respective issue screen)
2. implement the code related to Gherkin statements/steps and store it in Git, for example
3. generate .feature files based on the specification made in Jira
4. checkout the code from Git
5. run the tests in the CI
6. import the results back to Jira

Usually, you would start by having a Story, or similar (e.g. "requirement"), to describe the behavior of a certain feature and use that to drive your testing.

If you have it, then you can just use the "Create Test" on that issue to create the Scenario/Scenario Outline and have it automatically linked back to the Story/"requirement".

Otherwise, you can create the Test using the standard (issue) Create action from Jira's top menu. 

In this case, we'll create a Cucumber Test, of Cucumber Type "Scenario".

We can fill out the Gherkin statements immediately on the Jira issue create dialog or we can create the Test issue first and fill out the details on the next screen, from within the Test issue. In the latter case, we can take advantage of the built-in Gherkin editor which provides auto-complete of Gherkin steps.

After the Test is created it will impact the coverage of related "requirement", if any.

The coverage and the test results can be tracked in the "requirement" side (e.g. user story). In this case, you may see that coverage changed from being UNCOVERED to NOTRUN (i.e. covered and with at least one test not run).

The related statement's code is managed outside of Jira and stored in Git, for example.

In Cypress, tests related code is stored under cypress/integration directory, which itself contains several other directories. In this case, we've organized them as follows:

• cypress/integration/common: step implementation files, in JavaScript.
• cypress/integration/pages: abstraction of different pages, somehow based on the page-objects model
• cypress/integration/login: Cucumber .feature files, containing the tests as Gherkin Scenario(s)/Scenario Outline(s)

import { Given, When } from 'cypress-cucumber-preprocessor/steps';
import LoginPage from '../../pages/login-page';
import LoginResultsPage from '../../pages/login-results-page';

Given(/^browser is opened to login page$/, () => {
  LoginPage.visit();
});

When('user {string} logs in with password {string}', (username, password) => {
  LoginPage.enter_username(username);
  LoginPage.enter_password(password);
  LoginPage.pressLogin();
});

Then(/^welcome page should be open$/, () => {
    LoginResultsPage.expect().toBeSuccessful();
  });

Then(/^error page should be open$/, () => {
    LoginResultsPage.expect().toBeUnsuccessful();
  });

import { Given, When } from 'cypress-cucumber-preprocessor/steps';
import LoginPage from '../../pages/login-page';
import LoginResultsPage from '../../pages/login-results-page';

Given(/^browser is opened to login page$/, () => {
  LoginPage.visit();
});

When('user {string} logs in with password {string}', (username, password) => {
  LoginPage.enter_username(username);
  LoginPage.enter_password(password);
  LoginPage.pressLogin();
});

Then(/^welcome page should be open$/, () => {
    LoginResultsPage.expect().toBeSuccessful();
  });

Then(/^error page should be open$/, () => {
    LoginResultsPage.expect().toBeUnsuccessful();
  });

@REQ_CALC-7905
Feature: As a user, I can login the applicaiton

Scenario: Valid Login
    Given browser is opened to login page
    When user "demo" logs in with password "mode"
    Then welcome page should be open

Scenario: Invalid Login
    Given browser is opened to login page
    When user "dummy" logs in with password "password"
    Then error page should be open

Scenario Outline: Login With Invalid Credentials Should Fail
    Given browser is opened to login page
    When user "<username>" logs in with password "<password>"
    Then error page should be open

    Examples:
        | username  | password |
        | invalid   |   mode   |
        | demo      | invalid  |
        | invalid   | invalid  |

@REQ_CALC-7906
Feature: As a user, I can logout the application

Scenario: Valid Logout
    Given user is on the welcome page
    When user chooses to logout
    Then login page should be open

import LoginResultsPage from './login-results-page';

const USERNAME_FIELD = 'input[id=username_field]';
const PASSWORD_FIELD = 'input[id=password_field]';
const LOGIN_BUTTON = 'input[type=submit]';
const LOGIN_TEXT = 'LOGIN';


class LoginPage {
  static visit() {
    cy.visit('/');
  }


  static enter_username(username) {
    cy.get(USERNAME_FIELD)
      .type(username);
  }

  static enter_password(password) {
    cy.get(PASSWORD_FIELD)
      .type(password);
  }

  static pressLogin() {
    cy.get(LOGIN_BUTTON).contains(LOGIN_TEXT)
      .click();
    return new LoginResultsPage();
  }

}

export default LoginPage;

const RESULT_HEADER = 'h1';

class LoginResultsPage {
    static expect() {
      return {
        toBeSuccessful: () => {
          cy.get(RESULT_HEADER).should('have.text', 'Welcome Page')
        },

        toBeUnsuccessful: () => {
          cy.get(RESULT_HEADER).should('have.text', 'Error Page')
        },
      };
    }
  }
  
export default LoginResultsPage;

const RESULT_HEADER = 'h1';

class LogoutResultsPage {
    static expect() {
      return {
        toBeSuccessful: () => {
          cy.get(RESULT_HEADER).should('have.text', 'Login Page')
        },
      };
    }
  }
  
export default LogoutResultsPage;

import LoginPage from './login-page';

const LOGOUT_LINK = 'a';
const LOGOUT_TEXT = 'logout';


class WelcomePage {
  static visit() {
    cy.visit('/welcome.html');
  }


  static pressLogout() {
    cy.get(LOGOUT_LINK).contains(LOGOUT_TEXT)
      .click();
    return new LoginPage();
  }

}

export default WelcomePage;

You can then export the specification of the test to a Cucumber .feature file via the REST API, or the Export to Cucumber UI action from within the Test/Test Execution issue or even based on an existing saved filter. A plugin for your CI tool of choice can be used to ease this task.

So, you can either:

• use the UI
  • 
• use the REST API (more info here)

  • #!/bin/bash
    
    curl -u admin:admin  "http://192.168.56.102/rest/raven/1.0/export/test?keys=CALC-4763&fz=true" -o features.zip
    rm -f features/*.feature
    unzip -o features.zip  -d features

• use one of the available CI/CD plugins (e.g. see an example of Integration with Jenkins)

After being exported, the created .feature(s) will contain references to the Test issue key and the covered "requirement" issue key,  if that's the case. The naming of these files is detailed in Export Cucumber Features.

@REQ_CALC-7895
Feature: As a user, I can calculate the sum of 2 numbers

        #  Addition is great as a verification exercise to get the Cucumber-js infrastructure up and running
        @TEST_CALC-4763 @features/addition.feature
        Scenario: Add two number
                Given the numbers 2 and 3
                When they are added together
                Then should the result be 5

To run the tests and produce a Cucumber JSON report, we can use the cucumber-js binary.

./node_modules/cucumber/bin/cucumber-js -f json:report.json 

After running the tests, results can be imported to Xray via the REST API, or the Import Execution Results action within the Test Execution, or by using one of the available CI/CD plugins (e.g. see an example of Integration with Jenkins).

curl -H "Content-Type: application/json" -X POST -u admin:admin --data @"report.json" http://jiraserver.example.com/rest/raven/1.0/import/execution/cucumber

A new Test Execution will be created (unless you originally exported the Scenarios/Scenario Outlines from a Test Execution).

The execution screen details of the Test Run will provide overall status information and Gherkin statement-level results.

Results are reflected on the covered item (e.g. Story). On its issue screen, coverage now shows that the item is OK based on the latest testing results, that can also be tracked within the Test Coverage panel bellow. 

References

• Cypress
• Cypress documentation
• cypress-cucumber-example
• issue related to adding screenshots to the cucumber JSON report(s)