Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

One of the frameworks used in BDD context is Cucumber, which Xray has prime support for.  In this context, you may want to have a look at a BDD tutorial provided by the Cucumber teamat a tutorial for Java.

Similarly to what happens with other frameworks, with Cucumber "specification" and "implementation" are two distinct things, even though related: the test specification is done in natural language using Gherkin, while the implementation of each specification's phrase (i.e., step) is done in code (e.g., Java, Ruby or some other language).

...

...

  • """ (Doc Strings) - partial support (supported on creation/edition but not yet as reusable steps on the Automated Steps Library)
  • | (Data Tables) - partial support (supported on creation/edition but not yet as reusable steps on the Automated Steps Library)
  • @ (Tags), during the .feature export process based on the labels assigned to Test issues (e.g. Scenarios/Scenario Outlines) or during the import process
  • # (Comments)

...

Gliffy Diagram
namecucumber_tests_workflow
pagePin117


  1. Specify Cucumber tests using natural language, in Jira.
  2. Export Cucumber features from Jira to the CI environment, using the REST API.
  3. Implement tests in code and commit them to the source code versioning system.
  4. Execute tests in the CI environment.
  5. Report results to Xray, using the REST API.

...

Gliffy Diagram
namecucumber_tests_legacy_vcs_workflow2
pagePin117


  1. Create Cucumber tests (e.g. Scenario/Scenario Outlines) and their respective step code in some other tool.
    1. add an unique tag to each Scenario/Scenario outline, in the form of "@id:xxx", in which xxx is a number (e.g. 1, 2, 3, ...)
  2. Import existing Cucumber tests using the REST API (e.g., from an existing code base which may have tests, and their corresponding step implementation) 
  3. Export Cucumber features from Jira to the CI environment, using the REST API or the Jenkins/Bamboo plugins. It is crucial to use these exported features instead of the ones from the VCS, since the features exported from JIRA will contain additional tags that will allow several things, including the ability to correctly report the results to back to the correct entities in Jira, afterwards.
  4. Execute tests in the CI environment.
  5. Report results to Xray, using the REST API or the Jenkins/Bamboo plugins.
  6. Repeat from step 1 onwardsonward.


Info
titlePlease note

This flow will work fine as long as:

  • An unique tag "@id:xxx" is added to each Scenario/Scenario Outline within the file. The id just needs to be unique (e.g. sequential) within the feature file
  • The path of the feature file is not changed

Is any of the previous conditions is not satisfied then new and duplicated Tests, and Pre-Conditions, may be created.

See an example in Importing Cucumber Tests - REST.

...

Gliffy Diagram
namecucumber_tests_legacy_workflow
pagePin117


  1. Create Cucumber tests (e.g., Scenario/Scenario Outlines) and their respective step code in some other tool.
  2. Import existing Cucumber tests using the REST API (e.g., from an existing code base which may have tests, and their corresponding step implementation) 
  3. Optionally, and in parallel, 
    1. Specify additional or edit existing Cucumber tests in natural language, in Jira.
    2. Implement remaining tests in code and commit them to the source code versioning system.
  4. Export Cucumber features from Jira to the CI environment, using the REST API or the Jenkins/Bamboo plugins.
  5. Execute tests in the CI environment.
  6. Report results to Xray, using the REST API or the Jenkins/Bamboo plugins.
  7. Repeat from step 3 onwards.

...

titlePlease note

...

  1. .

Common problems

Submission of Cucumber results says "No tests found in execution result"

...