Page History

Versions Compared

Old Version 1

changes.mady.by.user Cristiano Cunha

Saved on Jun 23, 2021

compared with

New Version 2

changes.mady.by.user Cristiano Cunha

Saved on Jun 23, 2021

Key

This line was added.
This line was removed.
Formatting was changed.

...

We need to create the Test issue first and fill out the Gherkin statements later on in the Test issue screen.

Image Modified Image Modified

After the Test is created it will impact the coverage of related "requirement", if any.

...

Code Block

title	features/1_COM-19.feature
collapse	true

@REQ_COMXT-19225
Feature: Login feature

	@TEST_COMXT-29226
	Scenario: Test Login feature
		Scenario Outline: As a user, I can log into the secure area
				Given I am on the login page
				When I login with <username> and <password>
				Then I should see a flash message saying <message>
				
					Examples:
						| username | password             | message                        |
						| tomsmith | SuperSecretPassword! | You logged into a secure area! |
						| foobar   | barfoo               | Your username is invalid.      |

...

Code Block

language	bash
title	import_results_cloud.sh

#!/bin/bash

BASEJIRA_URLBASEURL=https://xray192.cloud168.xpand-it.com
token=$(0.168
JIRA_USERNAME=admin
JIRA_PASSWORD=admin

curl -H "Content-Type: application/json" -X POST -u $JIRA_USERNAME:$JIRA_PASSWORD --data @"cloud_auth.tmp/json/login-feature.json" "$BASE_URL/api/v1/authenticate"| tr -d '"')
curl -H "Content-Type: application/json" -X POST -H "Authorization: Bearer $token"  --data @"login-feature.json" "$BASE_URL/api/v1/import/execution/cucumber"$JIRA_BASEURL/rest/raven/2.0/import/execution/cucumber

Info

title	Which Cucumber endpoint/"format" to use?

To import results, you can use two different endpoints/"formats" (endpoints described in Import Execution Results - REST):

the "standard cucumber" endpoint
the "multipart cucumber" endpoint

The standard cucumber endpoint (i.e. /import/execution/cucumber) is simpler but more restrictive: you cannot specify values for custom fields on the Test Execution that will be created. This endpoint creates new Test Execution issues unless the Feature contains a tag having an issue key of an existing Test Execution.

The multipart cucumber endpoint will allow you to customise fields (e.g. Fix Version, Test Plan), if you wish to do so, on the Test Execution that will be created. Note that this endpoint always creates new Test Executions (as of Xray v4.2).

In sum, if you want to customise the Fix Version, Test Plan and/or Test Environment of the Test Execution issue that will be created, you'll have to use the "multipart cucumber" endpoint.

...

A new Test Execution will be created (unless you originally exported the Scenarios/Scenario Outlines from a Test Execution).

Image RemovedImage Added

The tests have failed (on purpose).

The execution screen details of the Test Run will provide overall status information and Gherkin statement-level results, therefore we can use it to analyze the failing test.

Image RemovedImage RemovedImage AddedImage Added

A given example can be expanded to see all Gherkin statements and, if available, it is possible to see also the attached stack trace.

Image RemovedImage Added

Note: in this case, the bug was on the Scenario Outline example which was expecting an invalid message.

...

Results are reflected on the covered item (e.g. Story). On its issue screen, coverage now shows that the item is OK NOK based on the latest testing results, that can also be tracked within the Test Coverage panel bellow.

Image AddedImage Removed

Using Git or other VCS as master

...

In any case, you'll need to synchronize synchronise your .feature files to Jira so that you can have visibility of them and report results against them.

...

look at the existing "requirement"/Story issue keys to guide your testing; keep their issue keys
specify Cucumber/Gherkin .feature files in your IDE and store it in Git, for example
implement the code related to Gherkin statements/steps and store it in Git, for example
import/synchronize synchronise the .feature files to Xray to provision or update corresponding Test entities
export/generate .feature files from Jira, so that they contain references to Tests and requirements in Jira
checkout the WebDriverIO related code from Git
run the tests in the CI
import the results back to Jira

...

Usually, you would start by having a Story, or similar (e.g. "requirement"), to describe the behavior behaviour of a certain feature and use that to drive your testing.

Image RemovedImage Added

Having those to guide testing, we could then move to our code to describe and implement the Cucumber test scenarios.

...

step-definitions/steps.js: step implementation files, in JavaScript.

Code Block

language	js
title	step-definitions/steps.js
collapse	true

const { Given, When, Then } = require('@cucumber/cucumber');

const LoginPage = require('../pageobjects/login.page');
const SecurePage = require('../pageobjects/secure.page');

const pages = {
    login: LoginPage
}

Given(/^I am on the (\w+) page$/, async (page) => {
    await pages[page].open()
});

When(/^I login with (\w+) and (.+)$/, async (username, password) => {
    await LoginPage.login(username, password)
});

Then(/^I should see a flash message saying (.*)$/, async (message) => {
    await expect(SecurePage.flashAlert).toBeExisting();
    await expect(SecurePage.flashAlert).toHaveTextContaining(message);
});

pageobjects: abstraction of different pages, somehow based on the page-objects model

Code Block

language	js
title	pageobjects/page.js
collapse	true

/**
* main page object containing all methods, selectors and functionality
* that is shared across all page objects
*/
module.exports = class Page {
    /**
    * Opens a sub page of the page
    * @param path path of the sub page (e.g. /path/to/page.html)
    */
    open (path) {
        return browser.url(`https://the-internet.herokuapp.com/${path}`)
    }
}

Code Block

language	js
title	pageobjects/login-page.js
collapse	true

const Page = require('./page');

/**
 * sub page containing specific selectors and methods for a specific page
 */
class LoginPage extends Page {
    /**
     * define selectors using getter methods
     */
    get inputUsername () { return $('#username') }
    get inputPassword () { return $('#password') }
    get btnSubmit () { return $('button[type="submit"]') }

    /**
     * a method to encapsule automation code to interact with the page
     * e.g. to login using username and password
     */
    async login (username, password) {
        await (await this.inputUsername).setValue(username);
        await (await this.inputPassword).setValue(password);
        await (await this.btnSubmit).click();
    }

    /**
     * overwrite specifc options to adapt it to page object
     */
    open () {
        return super.open('login');
    }
}

module.exports = new LoginPage();

Code Block

language	js
title	pageobjects/secure-page.js
collapse	true

const Page = require('./page');

/**
 * sub page containing specific selectors and methods for a specific page
 */
class SecurePage extends Page {
    /**
     * define selectors using getter methods
     */
    get flashAlert () { return $('#flash') }
}

module.exports = new SecurePage();

features/login.feature: Cucumber .feature files, containing the tests as Gherkin Scenario(s)/Scenario Outline(s). Please note that each "Feature: <..>" section should be tagged with the issue key of the corresponding "requirement"/story in Jira. You may need to add a prefix (e.g. "REQ_") before the issue key, depending on an Xray global setting.

Code Block

title	features/login.feature
collapse	true

@REQ_COMXT-19225
Feature: Login feature

	Scenario: Test Login feature
		Scenario Outline: As a user, I can log into the secure area
				Given I am on the login page
				When I login with <username> and <password>
				Then I should see a flash message saying <message>
				
					Examples:
						| username | password             | message                        |
						| tomsmith | SuperSecretPassword! | You logged into a secure area! |
						| foobar   | barfoo               | Your username is invalid.      |

...

Code Block

language	bash
title	example of a shell script to import/synchronize Scenario(s)/Background(s) from .features to Jira and Xray
collapse	true

#!/bin/bash

BASEJIRA_URLBASEURL=https://xray192.cloud168.xpand-it.com0.168
JIRA_USERNAME=admin
JIRA_PASSWORD=admin
PROJECT=COMXT
FILE=features.zip

zip -r features.zip$FILE features/ -i \*.feature

token=$(curl -H "Content-Type: applicationmultipart/jsonform-data" -X POSTu $JIRA_USERNAME:$JIRA_PASSWORD --dataF @"cloud_auth.json"file=@$FILE" "$BASE$JIRA_URLBASEURL/rest/apiraven/v1/authenticate"| tr -d '"')
curl -H "Content-Type: multipart/form-data" -H "Authorization: Bearer $token"  -F "file=@features.zip" "$BASE_URL/api/v1/1.0/import/feature?projectKey=$PROJECT"

...

So, you can either:

use the UI
- Image RemovedImage Added

use the REST API (more info here)

Code Block

language	bash
title	example of a shell script to export/generate .features from Xray
collapse	true

#!/bin/bash

token=$(curl -H "Content-Type: application/json" -X POST --data @"cloud_auth.json" https://xray.cloud.xpand-it.com/api/v1/authenticate| tr -d '"')
curl -H "Content-Type: application/json" -X GET -H "Authorization: Bearer $token" "https://xray.cloud.xpand-it.com/api/v1/export/cucumber?keys=COM-19JIRA_BASEURL=https://192.168.2.168
JIRA_USERNAME=admin
JIRA_PASSWORD=admin
KEYS="XT-142"

rm -f features.zip
curl -u $JIRA_USERNAME:$JIRA_PASSWORD  "$JIRA_BASEURL/rest/raven/2.0/export/test?keys=$KEYS&fz=true" -o features.zip

rm -rf features/*.feature
unzip -o features.zip  -d features

use one of the available CI/CD plugins (e.g. see an example of Integration with Jenkins)

...

Code Block

title	features/1_COM_19.feature

@REQ_COMXT-19225
Feature: Login feature

	@TEST_COMXT-29226
	Scenario Outline: As a user, I can log into the secure area
		Given I am on the login page
		When I login with <username> and <password>
		Then I should see a flash message saying <message>
		
			Examples:
				| username | password             | message                        |
				| tomsmith | SuperSecretPassword! | You logged into a secure area! |
				| foobar   | barfoo               | Your username is invalid.      |

...

Code Block

language	bash
title	Example of shell script to import results (e.g. import_results_cloud.sh)
collapse	true

#!/bin/bash

BASE_URL=https://xray.cloud.xpand-it.com
token=$(curl -H "Content-Type: application/json" -X POST --data @"cloud_auth.json" "$BASE_URL/api/v1/authenticate"| tr -d '"')
JIRA_BASEURL=https://192.168.0.168
JIRA_USERNAME=admin
JIRA_PASSWORD=admin

curl -H "Content-Type: application/json" -X POST -H "Authorization: Bearer $token" u $JIRA_USERNAME:$JIRA_PASSWORD --data @".tmp/json/login-feature.json" "$BASE_URL/api/v1$JIRA_BASEURL/rest/raven/2.0/import/execution/cucumber"

Info

title	Which Cucumber endpoint/"format" to use?

To import results, you can use two different endpoints/"formats" (endpoints described in Import Execution Results - REST):

the "standard cucumber" endpoint
the "multipart cucumber" endpoint

The multipart cucumber endpoint will allow you to customize fields (e.g. Fix Version, Test Plan), if you wish to do so, on the Test Execution that will be created. Note that this endpoint always creates new Test Executions (as of Xray v4.2).

In sum, if you want to customize the Fix Version, Test Plan and/or Test Environment of the Test Execution issue that will be created, you'll have to use the "multipart cucumber" endpoint.

...

A new Test Execution will be created (unless you originally exported the Scenarios/Scenario Outlines from a Test Execution).

Image RemovedImage Added

One of the tests fails (on purpose).

The execution screen details of the Test Run will provide overall status information and Gherkin statement-level results, therefore we can use it to analyze analyse the failing test.

Image RemovedImage Removed Image AddedImage Added

Results are reflected on the covered item (e.g. Story). On its issue screen, coverage now shows that the item is OK based on the latest testing results, that can also be tracked within the Test Coverage panel bellow.

Image AddedImage Removed

If we change the specification (i.e. the Gherkin scenarios), we need to import the .feature(s) once again.

...

Content

Space Tools

Page History

Versions Compared

Old Version 1

New Version 2

Key

Using Git or other VCS as master