Page History
Gliffy Diagram | ||||
---|---|---|---|---|
|
Cucumber is a framework for behavior-driven development (BDD), where the test specification is done in natural language, while the implementation of each specification's phrase (i.e., step) is done in code (e.g., Java, Ruby , C#, or some other language).
Xray provides the tool to create the specification inside Jira, although the implementation code itself is outside of Xray's scope.
One key thing that you must decide is: where do you want to manage the Cucumber scenarios? Do you want to use Jira as the master of information (recommended)? Or do you want to use Git, for example, as the master for your features and corresponding scenarios?
Info | ||
---|---|---|
| ||
Having both Jira and the VCS (e.g. Git, SVN) as master of information for Cucumber Scenarios/Scenario Outlines can lead to synchronization issues because the scenarios edited in JIRA won't be commited automatically to the VCS to the original feature files used. Thus, you can use a mix of the Standard and VCS workflows below but have in mind that the scenarios can be out of synch between what you have in Jira and what you have in the VCS. |
Table of Contents |
---|
Standard Cucumber workflow
In this workflow, Tests are created and managed in Jira. , thus Jira will be the master for the Cucumber scenarios.
Gliffy Diagram | ||||
---|---|---|---|---|
|
...
Info | ||
---|---|---|
| ||
See Automated Tests (Import/Export) for more information on how Cucumber feature files are generated and how to import back results to Xray. |
Pure VCS based workflow
Info | ||
---|---|---|
| ||
The support for this workflow has been added in v3.1, in experimental mode. |
In this workflow, the features are being "managed" in the source code VCS (versioning control system). This means that users will be editing features elsewhere, other than Jira, and they will want to synchronize the scenarios contained within those features to Jira. Therefore the master of information will be Git, SVN or whatever VCS is being used.
Gliffy Diagram | ||||||
---|---|---|---|---|---|---|
|
- Create Cucumber tests (e.g. Scenario/Scenario Outlines) and their respective step code in some other tool.
- add an unique tag to each Scenario/Scenario outline, in the form of "@id:xxx", in which xxx is a number (e.g. 1, 2, 3, ...)
- Import existing Cucumber tests using the REST API (e.g., from an existing code base which may have tests, and their corresponding step implementation)
- Export Cucumber features from Jira to the CI environment, using the REST API or the Jenkins/Bamboo plugins. It is crucial to use these exported features instead of the ones from the VCS, since the features exported from JIRA will contain additional tags that will allow several things, including the ability to correctly report the results to back to the correct entities in Jira, afterwards.
- Execute tests in the CI environment.
- Report results to Xray, using the REST API or the Jenkins/Bamboo plugins.
- Repeat from step 1 onwards.
Info | ||
---|---|---|
| ||
This flow will work fine as long as:
Is any of the previous conditions is not satisfied then new and duplicated Tests, and Pre-Conditions, may be created. See an example in Importing Cucumber Tests - REST. |
Legacy Cucumber migraton workflow
The workflow below applies to legacy/existing projects that already have Cucumber tests in some version control system (e.g., GIT, SVN), possibly alongside the software's implementation.
There are also cases where users prefer to edit Cucumber tests using another tool; even so, they can take advantage of Xray.
Existing Cucumber tests are initially imported to Jira, or they can be imported at a regular basis, if needed. Editing the tests can then proceed on the Jira side. While test editing can also occur simultaneously in Jira and in some other tool, it is highly recommended to keep it only on one side, preferably Jira.
Gliffy Diagram | ||||
---|---|---|---|---|
|
...
Note | ||
---|---|---|
| ||
As of Xray v2.3.x, step 2 is not idempotent, i.e., if you try to import Cucumber Tests multiple times using the endpoint Importing Cucumber Tests - REST, you may end up with repeated entities in Jira. This has been addressed in v3.1. |
VCS based workflow
...
. |
...
Common problems
Submission of Cucumber results says "No tests found in execution result"
...
Our recommendation is to manage it in Jira and always edit the Cucumber Scenarios/Scenario Outlines/Backgrounds there. You can use your VCS (e.g. Git, SVN) as the master but don't try to use it and at the same time also use Jira as master.