Page History
| Table of Contents |
|---|
Overview
Xray supports Gherkin based frameworks since its first version.
Gherkin is mainly used in BDD (behaviour-driven development) context. However, please note that Gherkin and BDD are two different thinksthings: you may use Gherkin to describe your test scenarios and still not following follow BDD.
BDD starts by having a clear understanding of the business value, or the ultimate benefit , of a user story, which is done by describing it properly (e.g. "As a ..., I want to ..., So that [benefit]").
Having it this understanding in mind will help depicting to depict "behaviours" that the system must address; these are the actual acceptance criteria, which are described in scenarios using an a “ubiquitous language”, as Dan North states. Gherkin can be used to provide the foundations of this language.
Key to BDD is having a shared/common understanding of the purpose of the feature: what is it supposed to do and why. This can be done collaboratively by the so-called "three amigos" (i.e. people from the business, development/coding and testing roles), which in the end need to agree on what composes a "feature" (i.e. the story itself along with the respective acceptance criteria as scenarios).
...
Xray provides the tooling to create and manage the specification inside Jira and to import/export it; however. However, how the implementation is managed is outside of Xray's scope.
...
Besides Cucumber, Xray also supports other frameworks, including Behave and SpecFlow.
| Table of Contents |
|---|
Supported features
Xray has a comprehensive support for Gherkin. First, it supports the creation of Scenarios and Scenario Outlines as Test cases, using the one of the possible values for the "Test Type" field (e.g. Gherkin, Cucumber).
...
The next sections detail in more depth all the features support supported by Xray.
Main features
- creation/edition of Scenario, Scenario Outline and Background entities
- validated validation with Cucumber (in different languages), Behave and SpecFlow
- centralized place to manage and do refactoring on Gherkin steps using the Automated Steps Library
- automatic provisioning of Gherkin based steps on the Automated Steps Library
- bulk import/update based on .feature files using the REST API and/or one of the available CI plugins
- export of .feature files containing Scenario, Scenario Outline and Background based on the respective Tests and Pre-Conditions through the UI, REST API and/or one of the available CI plugins
...
One key thing that you must decide is: where do you want to manage the Cucumber scenarios? Preferably you will only have a single source of truth. Do you want to use Jira as the master of information (recommended)? Or do you want to use Git, for example, as the master for your features and corresponding scenarios?
...
| Info | ||
|---|---|---|
| ||
Having both Jira and the VCS (e.g. Git, SVN) as master of information for Cucumber Scenarios/Scenario Outlines can lead to synchronization synchronisation issues because the scenarios edited in JIRA won't be commited committed automatically to the VCS to the original feature files used. Thus, you can use a mix of the Standard Jira and VCS workflows below but have in mind that the scenarios can be out of synch between what you have in Jira and what you have in the VCS. |
...
Jira based Cucumber workflow
In this workflow, Tests are created and managed in Jira, thus Jira will be the master for the Cucumber scenarios.
| Gliffy Diagram | ||||
|---|---|---|---|---|
|
- Specify Cucumber tests using natural language, in Jira.
- Export Cucumber features from Jira to the CI environment, using the REST API.
- Implement tests in code and commit them to the source code versioning system.
- Execute tests in the CI environment.
- Report results to Xray, using the REST API.
...
| Gliffy Diagram | ||||
|---|---|---|---|---|
|
- Create Cucumber tests (e.g. Scenario/Scenario Outlines) and their respective step code in some other tool.
- add an unique tag to each Scenario/Scenario outline, in the form of "@id:xxx", in which xxx is a number (e.g. 1, 2, 3, ...)
- Import existing Cucumber tests using the REST API (e.g., from an existing code base which may have tests, and their corresponding step implementation)
- Export Cucumber features from Jira to the CI environment, using the REST API or the Jenkins/Bamboo plugins. It is crucial to use these exported features instead of the ones from the VCS, since the features exported from JIRA will contain additional tags that will allow several things, including the ability to correctly report the results to back to the correct entities in Jira, afterwards.
- Execute tests in the CI environment.
- Report results to Xray, using the REST API or the Jenkins/Bamboo plugins.
- Repeat from step 1 onwards.
...
| Info | ||
|---|---|---|
| ||
This flow will work fine as long as:
Is any of the previous conditions is not satisfied then new and duplicated Tests, and Pre-Conditions, may be created. See an example in Importing Cucumber Tests - REST. |
Legacy Cucumber
...
migration workflow
The workflow below applies to legacy/existing projects that already have Cucumber tests in some version control system (e.g., GIT, SVN), possibly alongside the software's implementation.
...
| Gliffy Diagram | ||||
|---|---|---|---|---|
|
- Create Cucumber tests (e.g., Scenario/Scenario Outlines) and their respective step code in some other tool.
- Import existing Cucumber tests using the REST API (e.g., from an existing code base which may have tests, and their corresponding step implementation)
- Optionally, and in parallel,
- Specify additional or edit existing Cucumber tests in natural language, in Jira.
- Implement remaining tests in code and commit them to the source code versioning system.
- Export Cucumber features from Jira to the CI environment, using the REST API or the Jenkins/Bamboo plugins.
- Execute tests in the CI environment.
- Report results to Xray, using the REST API or the Jenkins/Bamboo plugins.
- Repeat from step 3 onwards.
...