Versions Compared

Old Version 2

changes.mady.by.user Cristiano Cunha

Saved on

compared with

New Version Current

changes.mady.by.user Unknown User (tdda)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

In this tutorial, we will create some UI tests as Cucumber Scenario(s)/Scenario Outline(s) and use WebDriverIO to implement the tests in JavaScript.

...

For the purpose of this tutorial, we'll use one of the a dummy website website provided  provided by Heroku, in . In our case containing , it contains just a few pages to support login kind of features ; we aim to test precisely those featureswhich we will be testing.


To start using WebDriverIO please follow the Get Started documentation.

...

The test consists in validating the login feature (with valid and invalid credentials) of the demo site, for that we have created a feature file that will have the description of the test supported by a base page that contain contains all methods and functionality that is shared across all page objects, a login page, that will extend the base page, that will have all the methods for interacting with the login page and a result page that will have the methods to interact in the page that is loaded after the login operation.

We have followed the documentation and executed first the command to install the WebDriverIO test runner:

Code Block
languagebash
themeDJango
npm install @wdio/cli

Then we answer answered a series of questions that will define the code to be generated using:

Code Block
languagebash
themeDJango
npx wdio config


The output of the questionaire questionnaire will look like this:


This will automatically generate the following files:

...

In case you need to interact with the Xray REST API at low-level using scripts (e.g. Bash/shell scripts), this tutorial uses auxiliary files that will handle those interactions.

Code Block
languagejs
titleExample of cloud_auth.json used in this tutorial
collapsetrue
- export_features.sh
- import_features.sh
- import_results.sh
- run_all_git_workflow.sh
- run_all_standard_workflow.sh


We Now we need to decide is which workflow we'll to use: do we you want to use Xray/Jira as the master for writing the declarative specification (i.e. the Gherkin based Scenarios), or do we you want to manage those outside using some editor and store them in Git, for example?

...

Info
titleLearn more

Please see Testing in BDD with Gherkin based frameworks (e.g. Cucumber) for an overview of the possible workflows.

The place that you'll use to edit the Cucumber Scenarios will affect your workflow. There are teams that prefer to edit Cucumber Scenarios in Jira using Xray, while there are others that prefer to edit them by writing the .feature files by hand using some IDE.

...

If you have it, then you can just use the "Create Test" on that issue to create the Scenario/Scenario Outline and have it automatically linked back to the Story/"requirement.".

Otherwise, you can create the Test using the standard (issue) Create action from Jira's top menu. 

...

We need to create the Test issue first and fill out the Gherkin statements later on in the Test issue screen.

Image Modified  Image Modified  



After the Test is created it will impact the coverage of related "requirement," , if any.

The coverage and the test results can be tracked in on the "requirementrequirements" side (e.g. user story). In this case, you may see that coverage changed from being UNCOVERED to NOTRUN (i.e. covered and with at least one test not run).


Image Modified


Additional tests could be created , and eventually linked to the same Story or linked to another one (e.g. logout).

...

The related statement's code is managed outside of Jira and stored in Git, for example.

In our source code, tests related test code is stored under steps-definitions directory, which itself can contain several other directories or files. In this case, we've only one referring to the login feature:

...

Notice that we have added an After scenario that will be executed after each scenario and after . After validating that an error occurred it will take a screenshot and attach it to the report.

...

Code Block
titlefeatures/1_COMXT-19225.feature
collapsetrue
@REQ_XT-225
Feature: Login feature

	@TEST_XT-226
	Scenario: Test Login feature
		Scenario Outline: As a user, I can log into the secure area
				Given I am on the login page
				When I login with <username> and <password>
				Then I should see a flash message saying <message>
				
					Examples:
						| username | password             | message                        |
						| tomsmith | SuperSecretPassword! | You logged into a secure area! |
						| foobar   | barfoo               | Your username is invalid.      |

...

Info
titleWhich Cucumber endpoint/"format" to use?

To import results, you can use two different endpointsendpoint/"formats" (endpoints described in Import Execution Results - REST):

  1. the "standard cucumber" endpoint
  2. the "multipart cucumber" endpoint

The standard cucumber endpoint (i.e. /import/execution/cucumber) is simpler but more restrictive: you cannot specify values for custom fields on the Test Execution that will be created.   This endpoint creates new Test Execution issues unless the Feature contains a tag having an issue key of an existing Test Execution.

The multipart cucumber endpoint will allow you to customise customize fields (e.g. Fix Version, Test Plan) , if you wish to do so, on the Test Execution that will be created. Note that this endpoint always creates new Test Executions (as of Xray v4.2).


In sum, if you want to customise the Fix Version, Test Plan and/or Test Environment of the Test Execution issue that will be created, you'll have to use the "multipart cucumber" endpoint.

...

Results are reflected on the covered item (e.g. Story). On its issue screen, coverage now shows that the item is NOK based on the latest testing results, that this can also be tracked within the Test Coverage panel bellow. 

Image Modified
  

Using Git or other VCS as master

You can edit your .feature files using your IDE outside of Jira (eventually storing them in your VCS using Git, for example) alongside with the remaining test code.

In any case, you'll need to synchronise synchronize your .feature files to Jira so that you can have visibility of them and report results against them.

...

Having those to guide testing, we could then move to our code to describe and implement the Cucumber test scenarios.

Tests related Test code is stored inside the step-definitions directory. We also have other directories present, to hold for instance the page object definitions in the pageobjects directory.

In this case, we've organised organized them as follows:

  • step-definitions/steps.js: step implementation files, in JavaScript.
    • Code Block
      languagejs
      titlestep-definitions/steps.js
      collapsetrue
      const { Given, When, Then } = require('@cucumber/cucumber');

const LoginPage = require('../pageobjects/login.page');
const SecurePage = require('../pageobjects/secure.page');

const pages = {
    login: LoginPage
}

Given(/^I am on the (\w+) page$/, async (page) => {
    await pages[page].open()
});

When(/^I login with (\w+) and (.+)$/, async (username, password) => {
    await LoginPage.login(username, password)
});

Then(/^I should see a flash message saying (.*)$/, async (message) => {
    await expect(SecurePage.flashAlert).toBeExisting();
    await expect(SecurePage.flashAlert).toHaveTextContaining(message);
});
  • pageobjects: abstraction of different pages, somehow based on the page-objects model
    • Code Block
      languagejs
      titlepageobjects/page.js
      collapsetrue
      /**
* main page object containing all methods, selectors and functionality
* that is shared across all page objects
*/
module.exports = class Page {
    /**
    * Opens a sub page of the page
    * @param path path of the sub page (e.g. /path/to/page.html)
    */
    open (path) {
        return browser.url(`https://the-internet.herokuapp.com/${path}`)
    }
}
    • Code Block
      languagejs
      titlepageobjects/login-page.js
      collapsetrue
      const Page = require('./page');

/**
 * sub page containing specific selectors and methods for a specific page
 */
class LoginPage extends Page {
    /**
     * define selectors using getter methods
     */
    get inputUsername () { return $('#username') }
    get inputPassword () { return $('#password') }
    get btnSubmit () { return $('button[type="submit"]') }

    /**
     * a method to encapsule automation code to interact with the page
     * e.g. to login using username and password
     */
    async login (username, password) {
        await (await this.inputUsername).setValue(username);
        await (await this.inputPassword).setValue(password);
        await (await this.btnSubmit).click();
    }

    /**
     * overwrite specifc options to adapt it to page object
     */
    open () {
        return super.open('login');
    }
}

module.exports = new LoginPage();
    • Code Block
      languagejs
      titlepageobjects/secure-page.js
      collapsetrue
      const Page = require('./page');

/**
 * sub page containing specific selectors and methods for a specific page
 */
class SecurePage extends Page {
    /**
     * define selectors using getter methods
     */
    get flashAlert () { return $('#flash') }
}

module.exports = new SecurePage();
  • features/login.feature: Cucumber .feature files, containing the tests as Gherkin Scenario(s)/Scenario Outline(s). Please note that each "Feature: <..>" section should be tagged with the issue key of the corresponding "requirement"/story in Jira. You may need to add a prefix (e.g. "REQ_") before the issue key, depending on an Xray global setting.
    • Code Block
      titlefeatures/login.feature
      collapsetrue
      @REQ_XT-225
Feature: Login feature

	Scenario: Test Login feature
		Scenario Outline: As a user, I can log into the secure area
				Given I am on the login page
				When I login with <username> and <password>
				Then I should see a flash message saying <message>
				
					Examples:
						| username | password             | message                        |
						| tomsmith | SuperSecretPassword! | You logged into a secure area! |
						| foobar   | barfoo               | Your username is invalid.      |

...

  • use the UI
  • use the REST API (more info here)
    • Code Block
      languagebash
      titleexample of a shell script to export/generate .features from Xray
      collapsetrue
      #!/bin/bash

JIRA_BASEURL=https://192.168.2.168
JIRA_USERNAME=admin
JIRA_PASSWORD=admin
KEYS="XT-142"

rm -f features.zip
curl -u $JIRA_USERNAME:$JIRA_PASSWORD  "$JIRA_BASEURL/rest/raven/2.0/export/test?keys=$KEYS&fz=true" -o features.zip
unzip -o features.zip  -d features
  • use one of the available CI/CD plugins (e.g. see an example of Integration with Jenkins)


For CI only purposepurposes, we will export the features to a new temporary directory named features/ on the root folder of your project. Please note that while implementing the tests, .feature files should be edited inside their respective folder. 

...

Info
titleWhich Cucumber endpoint/"format" to use?

To import results, you can use two different endpoints/"formats" (endpoints described in Import Execution Results - REST):

  1. the "standard cucumber" endpoint
  2. the "multipart cucumber" endpoint

The standard cucumber endpoint (i.e. /import/execution/cucumber) is simpler but more restrictive: you cannot specify values for custom fields on the Test Execution that will be created.   This endpoint creates new Test Execution issues unless the Feature contains a tag having an issue key of an existing Test Execution.

The multipart cucumber endpoint will allow you to customize fields (e.g. Fix Version, Test Plan) , if you wish to do so, on the Test Execution that will be created. Note that this endpoint always creates new Test Executions (as of Xray v4.2).


In sum, if you want to customize the Fix Version, Test Plan and/or Test Environment of the Test Execution issue that will be created, you'll have to use the "multipart cucumber" endpoint.

...