Versions Compared

Old Version 1

changes.mady.by.user Sergio Freire

Saved on

compared with

New Version 2

changes.mady.by.user Sergio Freire

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Note
iconfalse
titleSource-code for this tutorial
typeInfo

Code is available in GiHub; the repo contains some auxiliary scripts.

Requirements

  • Node.js
  • cucumber-json-formatternodejs
  • npm packages
    • cypress
    • @badeball/cypress-cucumber-preprocessor
    • cucumber-json-merge

Description

For the purpose of this tutorial, we will use a dummy website (source-code here) containing just a few pages to support login/logout kind of features; we aim to test precisely those features.

...

We need to configure Cypress to use the the the new @badeball/cypress-cucumber-preprocessor, which provides the ability to understand .feature files and also to produce Cucumber JSON reports.

This is done in Cypress' main configuration file, where you can also define the base URL of the website under test, the regex of the files that contain the test scenarios (i.e. <...>.feature files). Other options may be defined  (e.g for bypassing chromeWebSecurity, additional reporters, the ability to upload results to Cypress infrastructure in the cloud, etc).


Code Block
languagejs
titlecypress/plugins/index/cypress.config.js
collapsetrue
const cucumber { defineConfig } = require('cypress')
const createBundler = require("@bahmutov/cypress-esbuild-preprocessor");
const createEsbuildPlugin = require('@badeball/cypress-cucumber-preprocessor/esbuild').default

/**
 * @type {Cypress.PluginConfig}
 */
module.exports = createEsbuildPlugin;
const addCucumberPreprocessorPlugin = require('@badeball/cypress-cucumber-preprocessor').addCucumberPreprocessorPlugin;
 
async function setupNodeEvents(on, config) => {
  // `on` is used to hook into various events Cypress emits
  // `config` is the resolved Cypress config
  on('file:preprocessor', cucumber())
}

In Cypress' main configuration file, define the base URL of the website under test, the regex of the files that contain the test scenarios (i.e. <...>.feature files). Other options may be defined e.g for bypassing chromeWebSecurity, additional reporters, the ability to upload results to Cypress infrastructure in the cloud, etc).

Code Block
languagejs
title/cypress.json
collapsetrue
{
  "baseUrl": "https://robotwebdemo.herokuapp.com/",
  "testFiles": "**/*.feature",
  "ignoreTestFiles": [
    await addCucumberPreprocessorPlugin(on, config);
     
  on(
    "file:preprocessor",
         createBundler({
         plugins: [createEsbuildPlugin(config)],
        })
     
  );
 
  // Make sure to return the config object as it might have been modified by the plugin.
  return config;
}
 
module.exports = defineConfig({
  e2e: {
    baseUrl: "https://robotwebdemo.herokuapp.com/",
    specPattern: "**/*.feature",
    excludeSpecPattern: [
      "*.js",
      "*.md"
    ],
    "reporter"chromeWebSecurity: "junit"false,
  "reporterOptions"  projectId: {"bfi83g",
    "mochaFile": "test-results/test-output-[hash].xml"supportFile: false,
  },
  "chromeWebSecurity": false,setupNodeEvents
  "projectId": "bfi83g"
}}
 
})


The configuration of @badeball/cypress-cucumber-preprocessor can either be done on a JSON file .cypress-cucumber-preprocessorrc.json  or within package.json.

Next, here is an example of the contents of package.json.

Code Block
languagejs
titlepackage.json
collapsetrue
{
  "name": "tutorial-js-cypress-cucumber-robotdemo",
  "version": "1.0.0",
  "description": "An example for Cypress and Cucumber usage using Robot login demo website",
  "main": "index.js",
  "scripts": {
    "cypress:open:local": "CYPRESS_ENV=localhost npm run cypress:open",
    "cypress:open:prod": "CYPRESS_ENV=production npm run cypress:open",
    "cypress:open": "npx cypress open",
    "test:local": "CYPRESS_ENV=localhost npm run test  --spec 'cypress/integration/**/*.feature",
    "test:prod": "CYPRESS_ENV=production npm run test",
    "test": "npx cypress run --spec 'features/**/*.feature' --config integrationFolder=.",
    "test:debug:local": "CYPRESS_ENV=localhost npm run test:debug",
    "test:debug:prod": "CYPRESS_ENV=production npm run test:debug",
    "test:debug": "npx cypress run --headed --browser chrome --env TAGS='@e2e-test' --spec 'cypress/integration/**/*.feature'",
    "test:pull-features": "git submodule update --remote gherkin-features && cp -rf gherkin-features/* cypress/integration && node ./scripts/remove-old-features.js",
    "attach_screenshots": "node attach_screenshots.js"
  },
  "author": "Xblend",
  "license": "PrivateBSD-3-Clause",
  "dependencies": {
    "axios": "^0.18.0",
    "cucumberfs-json-mergeextra": "0^7.0.41",
    "fs-extraglob": "^7.1.7"
  },
  "devDependencies": {
    "@badeball/cypress-cucumber-preprocessor": "^7^13.0.12",
    "glob@bahmutov/cypress-esbuild-preprocessor": "^7^2.1.3",
  },
  "devDependencies@cypress/webpack-preprocessor": {"latest",
    "cypress": "^5^10.58.0",
    "cypress-cucumber-preprocessoresbuild": "^4^0.014.045",
    "eslint": "^5.13.0",
    "eslint-config-airbnb-base": "^12.1.0",
    "eslint-config-prettier": "^2.9.0",
    "eslint-plugin-importcypress": "^2.11.03",
    "eslint-plugin-prettierimport": "^2.623.04",
    "huskyeslint-plugin-prettier": "^1^2.36.10",
    "husky": "^1.3.1",
    "lint-staged": "^8.1.3"
  },
  "cypress-cucumber-preprocessor": {
    "nonGlobalStepDefinitionsjson": true,
    "cucumberJson": {
      "generateenabled": true,
      "outputFolderformatter": "cypress/usr/local/bin/cucumber-json-formatter",
      "filePrefixoutput": "",
      "fileSuffix": ".cucumbercucumber-report.json"
    }
  },
  "husky": {
    "hooks": {
      "pre-commit": "lint-staged"
    }
  },
  "lint-staged": {
    "*.js": [
      "eslint",
      "git add"
    ]
  }
}

Before moving into the actual implementation, we need to decide which workflow we'll use: do we want to use Xray/Jira as the master for writing the declarative specification (i.e. the Gherkin based Scenarios), or do we want to manage those outside using some editor and store them in Git, for example?


We need to have the cucumber-json-formatter tool, which can be downloaded from the respective GitHub repository. Make sure you pick the correct binary for your environment.
This tool is necessary to convert the Cucumber messages protobuf (*.ndjson file) report generated by  @badeball/cypress-cucumber-preprocessor.
 Code Block
languagebash
titleexample of Bash script to download hte cucumber-json-formatter tool for Linux
wget https://github.com/cucumber/json-formatter/releases/download/v19.0.0/cucumber-json-formatter-linux-amd64 -O /usr/local/bin/cucumber-json-formatter
chmod +x /usr/local/bin/cucumber-json-formatter

Before moving into the actual implementation, we need to decide which workflow we'll use: do we want to use Xray/Jira as the master for writing the declarative specification (i.e. the Gherkin based Scenarios), or do we want to manage those outside using some editor and store them in Git, for example?

 Info
titleLearn more
Please see Testing in BDD with Gherkin based frameworks (e.g. Cucumber) for an overview of the possible workflows.
The place that you'll use to edit the Cucumber 
 Info
titleLearn more
Please see Testing in BDD with Gherkin based frameworks (e.g. Cucumber) for an overview of the possible workflows.
The place that you'll use to edit the Cucumber Scenarios will affect your workflow. There are teams that prefer to edit Cucumber Scenarios in Jira using Xray, while others prefers to edit them by writing the .feature files by hand using some IDE.
...
In Cypress, the test code is mainly stored under cypress/integration directory, which itself contains several other directories. In this case, we've organized them the assets as follows:
  • cypress/integrationsupport/commonstep_definitions: step implementation files, in JavaScript.
    •  Code Block
      languagejs
      titlecypress/integrationsupport/commonstep_definitions/login.js
      collapsetrue
      import { Given, When, Then } from '"@badeball/cypress-cucumber-preprocessor/steps'";
import LoginPage from '../../pages/login-page';
import LoginResultsPage from '../../pages/login-results-page';

Given(/^browser is opened to login page$/, () => {
  LoginPage.visit();
});

When('user {string} logs in with password {string}', (username, password) => {
  LoginPage.enter_username(username);
  LoginPage.enter_password(password);
  LoginPage.pressLogin();
});

Then(/^welcome page should be open$/, () => {
    LoginResultsPage.expect().toBeSuccessful();
  });

Then(/^error page should be open$/, () => {
    LoginResultsPage.expect().toBeUnsuccessful();
  });
    •  Code Block
      languagejs
      titlecypress/integrationsupport/commonstep_definitions/logout.js
      collapsetrue
      import { Given, When, Then } from '"@badeball/cypress-cucumber-preprocessor/steps'";
 
import LoginPageWelcomePage from '../../pages/loginwelcome-page';
import LoginResultsPageLogoutResultsPage from '../../pages/loginlogout-results-page';
 
Given(/^browser^user is openedon tothe loginwelcome page$/, () => {
  LoginPageWelcomePage.visit();
});
 
When('user {string}chooses logs in with password {string}to logout', (username, password) => {
  LoginPageWelcomePage.enter_usernamepressLogout(username);
  LoginPage.enter_password(password});
  LoginPage.pressLogin();
});

Then(/^welcome^login page should be open$/, () => {
    LoginResultsPageLogoutResultsPage.expect().toBeSuccessful();
  });

Then(/^error page should be open$/, () => {
    LoginResultsPage.expect().toBeUnsuccessful();
  });
  • cypress/integration/cypress/integration/pages: abstraction of different pages, somehow based on the page-objects model
    •  Code Block
      languagejs
      titlecypress/integration/pages/login.js
      collapsetrue
      import LoginResultsPage from './login-results-page';

const USERNAME_FIELD = 'input[id=username_field]';
const PASSWORD_FIELD = 'input[id=password_field]';
const LOGIN_BUTTON = 'input[type=submit]';
const LOGIN_TEXT = 'LOGIN';


class LoginPage {
  static visit() {
    cy.visit('/');
  }


  static enter_username(username) {
    cy.get(USERNAME_FIELD)
      .type(username);
  }

  static enter_password(password) {
    cy.get(PASSWORD_FIELD)
      .type(password);
  }

  static pressLogin() {
    cy.get(LOGIN_BUTTON).contains(LOGIN_TEXT)
      .click();
    return new LoginResultsPage();
  }

}

export default LoginPage;
    •  Code Block
      languagejs
      titlecypress/integration/pages/login-results-page.js
      collapsetrue
      const RESULT_HEADER = 'h1';

class LoginResultsPage {
    static expect() {
      return {
        toBeSuccessful: () => {
          cy.get(RESULT_HEADER).should('have.text', 'Welcome Page')
        },

        toBeUnsuccessful: () => {
          cy.get(RESULT_HEADER).should('have.text', 'Error Page')
        },
      };
    }
  }
  
export default LoginResultsPage;
    •  Code Block
      languagejs
      titlecypress/integration/pages/logout-results-page.js
      collapsetrue
      const RESULT_HEADER = 'h1';

class LogoutResultsPage {
    static expect() {
      return {
        toBeSuccessful: () => {
          cy.get(RESULT_HEADER).should('have.text', 'Login Page')
        },
      };
    }
  }
  
export default LogoutResultsPage;
    •  Code Block
      languagejs
      titlecypress/integration/pages/welcome-page.js
      collapsetrue
      import LoginPage from './login-page';

const LOGOUT_LINK = 'a';
const LOGOUT_TEXT = 'logout';


class WelcomePage {
  static visit() {
    cy.visit('/welcome.html');
  }


  static pressLogout() {
    cy.get(LOGOUT_LINK).contains(LOGOUT_TEXT)
      .click();
    return new LoginPage();
  }

}

export default WelcomePage;
...
  • use the UI
  • use the REST API (more info here)
    •  Code Block
      languagebash
      #!/bin/bash

rm -f features/*.feature
curl -u admin:admin  "http://jiraserver.example.com/rest/raven/12.0/export/test?keys=CALC-7905;CALC-7906&fz=true" -o features.zip
unzip -o features.zip  -d features
  • use one of the available CI/CD plugins (e.g. see an example of Integration with Jenkins)
...
 No Format
npm run test

# or instead...

node_modules/cypress/bin/cypress run --spec 'features/**/*.feature' --config integrationFolder=.

This will produce one Cucumber JSON report in cypress/cucumber-json directory per each .feature file.
The cypress-cucumber-preprocessor package, as of  v4.0.0, does not produce reports containing the screenshots embedded. 
However, the following script (credits to the user that provided it on GitHub) can be used to update the previous JSON reports so that they contain the screenshots of the failed tests.
...
languagejs
titleattach_screenshots.js
collapsetrue
...

After running the tests, results can be imported to Xray via the REST API, or the Import Execution Results action within the Test Execution, or by using one of the available CI/CD plugins (e.g. see an example of Integration with Jenkins).

 No Format
curl -H "Content-Type: application/json" -X POST -u admin:admin --data @"cucumber-report.json" http://jiraserver.example.com/rest/raven/2.0/import/execution/cucumber
 Info
titleWhich Cucumber endpoint/"format" to use?
To import results, you can use two different endpoints/"formats" (endpoints described in Import Execution Results - REST):
  1. the "standard cucumber" endpoint
  2. the "multipart cucumber" endpoint
The standard cucumber endpoint (i.e. /import/execution/cucumber) is simpler but more restrictive: you cannot specify values for custom fields on the Test Execution that will be created. This endpoint creates new Test Execution issues unless the Feature contains a tag having an issue key of an existing Test Execution.
The multipart cucumber endpoint will allow you to customize fields (e.g. Fix Version, Test Plan) if you wish to do so, on the Test Execution that will be created. Note that this endpoint always creates new Test Executions (as of Xray v4.2).

In sum, if you want to customize the Fix Version, Test Plan and/or Test Environment of the Test Execution issue that will be created, you'll have to use the "multipart cucumber" endpoint.


A new Test Execution will be created (unless you originally exported the Scenarios/Scenario Outlines from a Test Execution).
Image Added

One of the tests fails (on purpose).
The execution screen details of the Test Run will provide overall status information and Gherkin statement-level results, therefore we can use it to analyze the failing test.

Image Added  

A given example can be expanded to see all Gherkin statements and, if available, it is possible to see also the attached screenshot(s).
Image Added
Image Added
Note: in this case, the bug was on the Scenario Outline example which was using a valid username/password combination.


Results are reflected on the covered item (e.g. Story). On the issue screen, coverage now shows that the item is OK based on the latest testing results which can also be tracked within the Test Coverage panel bellow. 
Image Added  
Using Git or other VCS as master
You can edit your .feature files using your IDE outside of Jira (eventually storing them in your VCS using Git, for example) alongside the remaining test code.
In any case, you'll need to synchronize your .feature files to Jira so that you can have visibility of them and report results against them.

The overall flow would be something like this:
  1. look at the existing "requirement"/Story issue keys to guide your testing; keep their issue keys
  2. specify Cucumber/Gherkin .feature files in your IDE supporting Cypress and store it in Git, for example
  3. implement the code related to Gherkin statements/steps and store it in Git, for example
  4. import/synchronize the .feature files to Xray to provision or update corresponding Test entities
  5. export/generate .feature files from Jira, so that they contain references to Tests and requirements in Jira
  6. checkout the Cypress related code from Git
  7. run the tests in the CI
  8. import the results back to Jira


Usually, you would start by having a Story, or similar (e.g. "requirement"), to describe the behavior of a certain feature and use that to drive your testing. 

Image Added  Image Added

Having those to guide testing, we could then move to Cypress to describe and implement the Cucumber test scenarios.
In Cypress, test related code is mainly stored inside the cypress/integration directory, which itself contains several other directories. In this case, we've organized the assets as follows:
  • cypress/support/step_definitions: step implementation files, in JavaScript.
    •  Code Block
      languagejs
      titlecypress/support/step_definitions/login.js
      collapsetrue
      import { Given, When, Then } from "@badeball/cypress-cucumber-preprocessor";
import LoginPage from '../../pages/login-page';
import LoginResultsPage from '../../pages/login-results-page';
 
Given(/^browser is opened to login page$/, () => {
  LoginPage.visit();
});
 
When('user {string} logs in with password {string}', (username, password) => {
  LoginPage.enter_username(username);
  LoginPage.enter_password(password);
  LoginPage.pressLogin();
});
 
Then(/^welcome page should be open$/, () => {
    LoginResultsPage.expect().toBeSuccessful();
});
 
Then(/^error page should be open$/, () => {
    LoginResultsPage.expect().toBeUnsuccessful();
});
    •  Code Block
      languagejs
      titlecypress/support/step_definitions/logout.js
      collapsetrue
      import { Given, When, Then } from "@badeball/cypress-cucumber-preprocessor";
 
import WelcomePage from '../../pages/welcome-page';
import LogoutResultsPage from '../../pages/logout-results-page';
 
Given(/^user is on the welcome page$/, () => {
  WelcomePage.visit();
});
 
When('user chooses to logout', () => {
  WelcomePage.pressLogout();
});
 
Then(/^login page should be open$/, () => {
    LogoutResultsPage.expect().toBeSuccessful();
});
  • cypress/integration/pages: abstraction of different pages, somehow based on the page-objects model
    •  Code Block
      languagejs
      titlecypress/integration/pages/login.js
      collapsetrue
      import LoginResultsPage from './login-results-page';

const USERNAME_FIELD = 'input[id=username_field]';
const PASSWORD_FIELD = 'input[id=password_field]';
const LOGIN_BUTTON = 'input[type=submit]';
const LOGIN_TEXT = 'LOGIN';


class LoginPage {
  static visit() {
    cy.visit('/');
  }


  static enter_username(username) {
    cy.get(USERNAME_FIELD)
      .type(username);
  }

  static enter_password(password) {
    cy.get(PASSWORD_FIELD)
      .type(password);
  }

  static pressLogin() {
    cy.get(LOGIN_BUTTON).contains(LOGIN_TEXT)
      .click();
    return new LoginResultsPage();
  }

}

export default LoginPage;
    •  Code Block
      languagejs
      titlecypress/integration/pages/login-results-page.js
      collapsetrue
      const RESULT_HEADER = 'h1';

class LoginResultsPage {
    static expect() {
      return {
        toBeSuccessful: () => {
          cy.get(RESULT_HEADER).should('have.text', 'Welcome Page')
        },

        toBeUnsuccessful: () => {
          cy.get(RESULT_HEADER).should('have.text', 'Error Page')
        },
      };
    }
  
...
    • }
  
export default LoginResultsPage;
    •  Code Block
      languagejs
      titlecypress/integration/pages/logout-results-page.js
      collapsetrue
      const RESULT_HEADER = 'h1';

class LogoutResultsPage {
    static expect() 
...
    • {
      return {
        
...
    • toBeSuccessful: (
...
    • ) => {
          cy.get(RESULT_HEADER).should('have.text', 'Login Page')
        
...
    • },
      };
    
...
    • }
  }
  
export default LogoutResultsPage;
    •  Code Block
      languagejs
      titlecypress/integration/pages/welcome-page.js
      collapsetrue
      import LoginPage from './login-page';

const LOGOUT_LINK 
...
    • = 'a';
const LOGOUT_TEXT = 'logout';


class WelcomePage {
  static visit() {
    cy.visit('/welcome.html');
  }


  static pressLogout() {
    cy.get(LOGOUT_LINK).contains(LOGOUT_TEXT)
      
...
    • .click();
    return 
...
    • new LoginPage();
  }

}

export 
...
    • default WelcomePage;
  • cypress/integration/login: Cucumber .feature files, containing the tests as Gherkin Scenario(s)/Scenario Outline(s). Please note that each "Feature: <..>" section should be tagged with the issue key of the corresponding "requirement"/story in Jira. You may need to add a prefix (e.g. "REQ_") before the issue key, depending on a global Xray setting.
    •  Code Block
      titlecypress/integration/login/login.feature
      collapsetrue
      @REQ_CALC-7905
Feature: As a user, I can login the applicaiton

Scenario: Valid Login
    Given browser is opened to login page
    When user "demo" logs in with password "mode"
    Then welcome page should be open

Scenario: Invalid Login
    Given browser is opened to login page
    When user "dummy" logs in with password "password"
    Then error page should be open

Scenario Outline: Login With Invalid Credentials Should Fail
    Given browser is opened to login page
    When user "<username>" logs in with password "<password>"
    Then error page should be open

    Examples:
        
...
    • | 
...
    • username 
...
    •  
...
    • | 
...
    • password |
        | invalid   
...
    • |   mode   
...
    • |
        | demo      
...
    • | invalid  |
        | invalid   
...
    • | 
...
    • invalid  |
    •  Code Block
      titlecypress/integration/login/logout.feature
      collapsetrue
      @REQ_CALC-7906
Feature: As a user, I can logout the application

Scenario: Valid Logout
    Given user is on the welcome page
    
...
    • When user chooses to logout
    Then login page should be open
Before running the tests in the CI environment, you need to import your .feature files to Xray/Jira; you can invoke the REST API directly or use one of the available plugins/tutorials for CI tools
The cucumber-json-merge utility may be handy to merge the results of each feature, so they can be then submitted to Xray as one single file.
Next, is an example of a shell script with all these steps.

 Code Block
languagebash
titleexample of a Bash script to run the tests and produce a unified Cucumber JSON report
collapsetrue
#!/bin/bash

rm -f cypress/cucumber-json/*
npm run test
npm run attach_screenshots
cucumber-json-merge  -d cypress/cucumber-json/
After running the tests, results can be imported to Xray via the REST API, or the Import Execution Results action within the Test Execution, or by using one of the available CI/CD plugins (e.g. see an example of Integration with Jenkins).
 No Format
zip -r features.zip cypress/integration/ -i \*.feature
curl -H "Content-Type: applicationmultipart/jsonform-data" -X POST -u admin:admin --dataF @"reportfile=@features.jsonzip" "http://jiraserver.example.com/rest/raven/12.0/import/execution/cucumberfeature?projectKey=CALC"


To import results, you can use two different endpoints/"formats" (endpoints described in Import Execution Results - REST):
  1. the "standard cucumber" endpoint
  2. the "multipart cucumber" endpoint
The standard cucumber endpoint (i.e. /import/execution/cucumber) is simpler but more restrictive: you cannot specify values for custom fields on the Test Execution that will be created. This endpoint creates new Test Execution issues unless the Feature contains a tag having an issue key of an existing Test Execution.
The multipart cucumber endpoint will allow you to customize fields (e.g. Fix Version, Test Plan) if you wish to do so, on the Test Execution that will be created. Note that this endpoint always creates new Test Executions (as of Xray v4.2).
In sum, if you want to customize the Fix Version, Test Plan and/or Test Environment of the Test Execution issue that will be created, you'll have to use the "multipart cucumber" endpoint.
 Info
titleWhich Cucumber endpoint/"format" to use?
A new Test Execution will be created (unless you originally exported the Scenarios/Scenario Outlines from a Test Execution).
Image Removed
One of the tests fails (on purpose).
The execution screen details of the Test Run will provide overall status information and Gherkin statement-level results, therefore we can use it to analyze the failing test.
Image Removed  
A given example can be expanded to see all Gherkin statements and, if available, it is possible to see also the attached screenshot(s).
Image Removed
Image Removed
Note: in this case, the bug was on the Scenario Outline example which was using a valid username/password combination.
Results are reflected on the covered item (e.g. Story). On the issue screen, coverage now shows that the item is OK based on the latest testing results which can also be tracked within the Test Coverage panel bellow. 
Image Removed  
Using Git or other VCS as master
You can edit your .feature files using your IDE outside of Jira (eventually storing them in your VCS using Git, for example) alongside the remaining test code.
In any case, you'll need to synchronize your .feature files to Jira so that you can have visibility of them and report results against them.
The overall flow would be something like this:
  1. look at the existing "requirement"/Story issue keys to guide your testing; keep their issue keys
  2. specify Cucumber/Gherkin .feature files in your IDE supporting Cypress and store it in Git, for example
  3. implement the code related to Gherkin statements/steps and store it in Git, for example
  4. import/synchronize the .feature files to Xray to provision or update corresponding Test entities
  5. export/generate .feature files from Jira, so that they contain references to Tests and requirements in Jira
  6. checkout the Cypress related code from Git
  7. run the tests in the CI
  8. import the results back to Jira
Usually, you would start by having a Story, or similar (e.g. "requirement"), to describe the behavior of a certain feature and use that to drive your testing. 
Image Removed  Image Removed
Having those to guide testing, we could then move to Cypress to describe and implement the Cucumber test scenarios.
In Cypress, test related code is stored inside the cypress/integration directory, which itself contains several other directories. In this case, we've organized them as follows:
  • cypress/integration/common: step implementation files, in JavaScript.
    •  Code Block
      languagejs
      titlecypress/integration/common/login.js
      collapsetrue
      import { Given, When } from 'cypress-cucumber-preprocessor/steps';
import LoginPage from '../../pages/login-page';
import LoginResultsPage from '../../pages/login-results-page';

Given(/^browser is opened to login page$/, () => {
  LoginPage.visit();
});

When('user {string} logs in with password {string}', (username, password) => {
  LoginPage.enter_username(username);
  LoginPage.enter_password(password);
  LoginPage.pressLogin();
});

Then(/^welcome page should be open$/, () => {
    LoginResultsPage.expect().toBeSuccessful();
  });

Then(/^error page should be open$/, () => {
    LoginResultsPage.expect().toBeUnsuccessful();
  });
    •  Code Block
      languagejs
      titlecypress/integration/common/logout.js
      collapsetrue
      import { Given, When } from 'cypress-cucumber-preprocessor/steps';
import LoginPage from '../../pages/login-page';
import LoginResultsPage from '../../pages/login-results-page';

Given(/^browser is opened to login page$/, () => {
  LoginPage.visit();
});

When('user {string} logs in with password {string}', (username, password) => {
  LoginPage.enter_username(username);
  LoginPage.enter_password(password);
  LoginPage.pressLogin();
});

Then(/^welcome page should be open$/, () => {
    LoginResultsPage.expect().toBeSuccessful();
  });

Then(/^error page should be open$/, () => {
    LoginResultsPage.expect().toBeUnsuccessful();
  });
  • cypress/integration/pages: abstraction of different pages, somehow based on the page-objects model
    •  Code Block
      languagejs
      titlecypress/integration/pages/login.js
      collapsetrue
      import LoginResultsPage from './login-results-page';

const USERNAME_FIELD = 'input[id=username_field]';
const PASSWORD_FIELD = 'input[id=password_field]';
const LOGIN_BUTTON = 'input[type=submit]';
const LOGIN_TEXT = 'LOGIN';


class LoginPage {
  static visit() {
    cy.visit('/');
  }


  static enter_username(username) {
    cy.get(USERNAME_FIELD)
      .type(username);
  }

  static enter_password(password) {
    cy.get(PASSWORD_FIELD)
      .type(password);
  }

  static pressLogin() {
    cy.get(LOGIN_BUTTON).contains(LOGIN_TEXT)
      .click();
    return new LoginResultsPage();
  }

}

export default LoginPage;
    •  Code Block
      languagejs
      titlecypress/integration/pages/login-results-page.js
      collapsetrue
      const RESULT_HEADER = 'h1';

class LoginResultsPage {
    static expect() {
      return {
        toBeSuccessful: () => {
          cy.get(RESULT_HEADER).should('have.text', 'Welcome Page')
        },

        toBeUnsuccessful: () => {
          cy.get(RESULT_HEADER).should('have.text', 'Error Page')
        },
      };
    }
  }
  
export default LoginResultsPage;
    •  Code Block
      languagejs
      titlecypress/integration/pages/logout-results-page.js
      collapsetrue
      const RESULT_HEADER = 'h1';

class LogoutResultsPage {
    static expect() {
      return {
        toBeSuccessful: () => {
          cy.get(RESULT_HEADER).should('have.text', 'Login Page')
        },
      };
    }
  }
  
export default LogoutResultsPage;
    •  Code Block
      languagejs
      titlecypress/integration/pages/welcome-page.js
      collapsetrue
      import LoginPage from './login-page';

const LOGOUT_LINK = 'a';
const LOGOUT_TEXT = 'logout';


class WelcomePage {
  static visit() {
    cy.visit('/welcome.html');
  }


  static pressLogout() {
    cy.get(LOGOUT_LINK).contains(LOGOUT_TEXT)
      .click();
    return new LoginPage();
  }

}

export default WelcomePage;
  • cypress/integration/login: Cucumber .feature files, containing the tests as Gherkin Scenario(s)/Scenario Outline(s). Please note that each "Feature: <..>" section should be tagged with the issue key of the corresponding "requirement"/story in Jira. You may need to add a prefix (e.g. "REQ_") before the issue key, depending on a global Xray setting.
    •  Code Block
      titlecypress/integration/login/login.feature
      collapsetrue
      @REQ_CALC-7905
Feature: As a user, I can login the applicaiton

Scenario: Valid Login
    Given browser is opened to login page
    When user "demo" logs in with password "mode"
    Then welcome page should be open

Scenario: Invalid Login
    Given browser is opened to login page
    When user "dummy" logs in with password "password"
    Then error page should be open

Scenario Outline: Login With Invalid Credentials Should Fail
    Given browser is opened to login page
    When user "<username>" logs in with password "<password>"
    Then error page should be open

    Examples:
        | username  | password |
        | invalid   |   mode   |
        | demo      | invalid  |
        | invalid   | invalid  |
    •  Code Block
      titlecypress/integration/login/logout.feature
      collapsetrue
      @REQ_CALC-7906
Feature: As a user, I can logout the application

Scenario: Valid Logout
    Given user is on the welcome page
    When user chooses to logout
    Then login page should be open
Before running the tests in the CI environment, you need to import your .feature files to Xray/Jira; you can invoke the REST API directly or use one of the available plugins/tutorials for CI tools.
 Code Block
languagebash
zip -r features.zip cypress/integration/ -i \*.feature
curl -H "Content-Type: multipart/form-data" -u admin:admin -F "file=@features.zip" "http://jiraserver.example.com/rest/raven/1.0/import/feature?projectKey=CALC"
 Info
titlePlease note
Each Scenario of each .feature will be created as a Test issue that contains unique identifiers, so that if you import once again then Xray can update the existent Test and don't create any duplicated tests.
Afterwards, you can export those features out of Jira, based on some criteria so they are properly tagged with corresponding issue keys; this is important because results need to contain these references.
You can then export the specification of the test to a Cucumber .feature file via the REST API, or the Export to Cucumber UI action from within the Test/Test Execution issue or even based on an existing saved filter. A plugin for your CI tool of choice can be used to ease this task.
So, you can either:
  • use the UI
    • Image Removed
  • use the REST API (more info here)
    •  Code Block
      languagebash
      #!/bin/bash

rm -f features/*.feature
curl -u admin:admin  "http://jiraserver.example.com/rest/raven/1.0/export/test?keys=CALC-7905;CALC-7906&fz=true" -o features.zip
unzip -o features.zip  -d features
  • use one of the available CI/CD plugins (e.g. see an example of Integration with Jenkins)
For CI only purpose, we will export the features to a new temporary directory named features/ on the root folder of your Cypress project (we'll need to tell Cypress to use this folder). Please note that while implementing the tests, .feature files should be edited inside the cypress/integration/login folder, in this case;  
After being exported, the created .feature(s) will contain references to the Test issue keys, eventually prefixed (e.g. "TEST_") depending on an Xray global setting, and the covered "requirement" issue key,  if that's the case. The naming of these files is detailed in Export Cucumber Features.
 Code Block
titlefeatures/1_CALC-7905.feature
@REQ_CALC-7905
Feature: As a user, I can login the application
	#As a user, I can login the application


	@TEST_CALC-7903 @cypress/integration/login/login.feature
	Scenario Outline: Login With Invalid Credentials Should Fail
		Given browser is opened to login page
		When user "<username>" logs in with password "<password>"
		Then error page should be open

			Examples:
				| username | password |
				| invalid  | mode     |
				| demo     | invalid  |
				| invalid  | invalid  |
				| demo     | mode     |


	@TEST_CALC-7902 @cypress/integration/login/login.feature
	Scenario: Invalid Login
		Given browser is opened to login page
		When user "dummy" logs in with password "password"
		Then error page should be open


	@TEST_CALC-7901 @cypress/integration/login/login.feature
	Scenario: Valid Login
		Given browser is opened to login page
		When user "demo" logs in with password "mode"
		Then welcome page should be open(base)
To run the tests and produce Cucumber JSON reports(s), we can either use npm or cypress command directly.
 No Format
npm run test

# or instead...

node_modules/cypress/bin/cypress run --spec 'features/**/*.feature' --config integrationFolder=.
This will produce one Cucumber JSON report in cypress/cucumber-json directory per each .feature file.
The cypress-cucumber-preprocessor package, as of  v4.0.0, does not produce reports containing the screenshots embedded. 
However, the following script (credits to the user that provided it on GitHub) can be used to update the previous JSON reports so that they contain the screenshots of the failed tests.
 Code Block
languagejs
titleattach_screenshots.js
collapsetrue
const fs = require('fs-extra')
const path = require('path')
const chalk = require('chalk')

const cucumberJsonDir = './cypress/cucumber-json'
const cucumberReportFileMap = {}
const cucumberReportMap = {}
const jsonIndentLevel = 2
const ReportDir = './cypress/reports/cucumber-report'
const screenshotsDir = './cypress/screenshots'

getCucumberReportMaps()
addScreenshots()

//Mapping cucumber json files from the cucumber-json directory to the features
function getCucumberReportMaps() {
    const files = fs.readdirSync(cucumberJsonDir).filter(file => {
        return file.indexOf('.json') > -1
    })
    files.forEach(file => {
        const json = JSON.parse(
            fs.readFileSync(path.join(cucumberJsonDir, file))
        )
        if (!json[0]) { return }
        const [feature] = json[0].uri.split('/').reverse()
        cucumberReportFileMap[feature] = file
        cucumberReportMap[feature] = json
    })
}

//Adding screenshots to the respective failed test steps in the feature files
function addScreenshots() {

    const prependPathSegment = pathSegment => location => path.join(pathSegment, location)

    const readdirPreserveRelativePath = location => fs.readdirSync(location).map(prependPathSegment(location))

    const readdirRecursive = location => readdirPreserveRelativePath(location)
        .reduce((result, currentValue) => fs.statSync(currentValue).isDirectory()
            ? result.concat(readdirRecursive(currentValue))
            : result.concat(currentValue), [])
    const screenshots = readdirRecursive(path.resolve(screenshotsDir)).filter(file => {
        return file.indexOf('.png') > -1
    })

    const featuresList = Array.from(new Set(screenshots.map(x => x.match(/[\w-_.]+\.feature/g)[0])))
    featuresList.forEach(feature => {
        screenshots.forEach(screenshot => {

            const regex = /(?<=\ --\ ).*?((?=\ \(example\ \#\d+\))|(?=\ \(failed\)))/g
            const [scenarioName] = screenshot.match(regex)
            console.info(chalk.blue('\n Adding screenshot to cucumber-json report for'))
            console.info(chalk.blue(scenarioName))

            console.log(featuresList)
            console.log(feature)
            console.log(cucumberReportMap)
            const myScenarios = cucumberReportMap[feature][0].elements.filter(
                e => scenarioName.includes(e.name)
            )
            if (!myScenarios) { return }
            let foundFailedStep = false
            myScenarios.forEach(myScenario => {
                if (foundFailedStep) {
                    return
                }
                let myStep
                if (screenshot.includes('(failed)')) {
                    myStep = myScenario.steps.find(
                        step => step.result.status === 'failed'
                    )
                } else {
                    myStep = myScenario.steps.find(
                        step => step.name.includes('screenshot')
                    )
                }
                if (!myStep) {
                    return
                }
                const data = fs.readFileSync(
                    path.resolve(screenshot)
                )
                if (data) {
                    const base64Image = Buffer.from(data, 'binary').toString('base64')
                    if (!myStep.embeddings) {
                        myStep.embeddings = []
                        myStep.embeddings.push({ data: base64Image, mime_type: 'image/png' })
                        foundFailedStep = true
                    }
                }
            })
            //Write JSON with screenshot back to report file.
            fs.writeFileSync(
                path.join(cucumberJsonDir, cucumberReportFileMap[feature]),
                JSON.stringify(cucumberReportMap[feature], null, jsonIndentLevel)
            )
        })
    })
}
The cucumber-json-merge utility may be handy to merge the results of each feature, so they can be then submitted to Xray as one single file.
Next, is an example of a shell script with all these steps.
 Code Block
languagebash
titleexample of a Bash script to run the tests and produce a unified Cucumber JSON report
collapsetrue
#!/bin/bash

rm -f cypress/cucumber-json/*
npm run test
npm run attach_screenshots
cucumber-json-merge  -d cypress/cucumber-json/
Please note
Each Scenario of each .feature will be created as a Test issue that contains unique identifiers, so that if you import once again then Xray can update the existent Test and don't create any duplicated tests.

Afterwards, you can export those features out of Jira, based on some criteria so they are properly tagged with corresponding issue keys; this is important because results need to contain these references.
You can then export the specification of the test to a Cucumber .feature file via the REST API, or the Export to Cucumber UI action from within the Test/Test Execution issue or even based on an existing saved filter. A plugin for your CI tool of choice can be used to ease this task.
So, you can either:
  • use the UI
    • Image Added
  • use the REST API (more info here)
    •  Code Block
      languagebash
      #!/bin/bash

rm -f features/*.feature
curl -u admin:admin  "http://jiraserver.example.com/rest/raven/2.0/export/test?keys=CALC-7905;CALC-7906&fz=true" -o features.zip
unzip -o features.zip  -d features
  • use one of the available CI/CD plugins (e.g. see an example of Integration with Jenkins)

For CI only purpose, we will export the features to a new temporary directory named features/ on the root folder of your Cypress project (we'll need to tell Cypress to use this folder). Please note that while implementing the tests, .feature files should be edited inside the cypress/integration/login folder, in this case;  
After being exported, the created .feature(s) will contain references to the Test issue keys, eventually prefixed (e.g. "TEST_") depending on an Xray global setting, and the covered "requirement" issue key,  if that's the case. The naming of these files is detailed in Export Cucumber Features.

 Code Block
titlefeatures/1_CALC-7905.feature
@REQ_CALC-7905
Feature: As a user, I can login the application
	#As a user, I can login the application


	@TEST_CALC-7903 @cypress/integration/login/login.feature
	Scenario Outline: Login With Invalid Credentials Should Fail
		Given browser is opened to login page
		When user "<username>" logs in with password "<password>"
		Then error page should be open

			Examples:
				| username | password |
				| invalid  | mode     |
				| demo     | invalid  |
				| invalid  | invalid  |
				| demo     | mode     |


	@TEST_CALC-7902 @cypress/integration/login/login.feature
	Scenario: Invalid Login
		Given browser is opened to login page
		When user "dummy" logs in with password "password"
		Then error page should be open


	@TEST_CALC-7901 @cypress/integration/login/login.feature
	Scenario: Valid Login
		Given browser is opened to login page
		When user "demo" logs in with password "mode"
		Then welcome page should be open(base)

To run the tests and produce Cucumber JSON reports(s), we can either use npm or cypress command directly.

 No Format
npm run test

# or instead...

node_modules/cypress/bin/cypress run --spec 'features/**/*.feature'

This will produce one Cucumber JSON report.

After running the tests, results can be imported to Xray via the REST API, or the Import Execution Results action within the Test Execution, or by using one of the available CI/CD plugins (e.g. see an example of Integration with Jenkins).
 Code Block
languagebash
titleexample of a Bash script to import results using the standard Cucumber endpoint
collapsetrue
#!/bin/bash

curl -H "Content-Type: application/json" -X POST -u admin:admin --data @"report.json" http://jiraserver.example.com/rest/raven/12.0/import/execution/cucumber
...