In this tutorial, we will perform some web/UI-based tests using Gwen.
Gwen uses the Given, When, Then syntax from Gherkin (thus, its name) to implement an interpretation engine that allows users to easily write "automated tests" (i.e. automated scripts), whose steps will be executed implicitly by their corresponding code implementation. Thus, users can focus on writing (executable) specifications without having to do all the implementation hard-work.
Gwen also separates declarative from imperative style Gherkin specifications. Declarative is done in standard .feature files that may include steps defined in while imperative specifications (i.e. "meta-features") are managed in .meta files.
Gwen uses Selenium under the hood, by providing a DSL that allows users to interact with the browser without having to write code.
From the many interesting features of Gwen we can highlight the auto-update capability and also the ability taking screenshots, which will be available for analysis after tests are run.
npm install -g cucumber-json-merge
Remember that we need to manage:
- features (declarative specifications, usually stored in .feature files)
- meta-features (imperative specifications, usually stored in .meta files)
Besides that, you need to decide is which workflow we'll use: do we want to use Xray/Jira as the master for writing the declarative specification or do we want to manage those in Git?
Please see Testing in BDD with Gherkin based frameworks (e.g. Cucumber) for an overview of the possible workflows.
Using Jira and Xray as master
This section assumes using Xray as master, i.e. the place that you'll be using to edit the specifications (e.g. the scenarios that are part of .feature files).
This tutorial explores using Xray for storing and managing the declarative scenarios and not the ones contained within the meta-features.
However, it should also be possible to manage them as Test issues with a "StepDef" label; it would require further evaluation though.
The first step is to create a Cucumber Test, of Cucumber Type "Scenario", in Jira. The specification would be exactly the same as the one provided in the original repository.
The test is quite self-explanatory, which is the ultimate purpose of using this approach: a browser is open, then we search by “Gwen automation” and then we look at the first Google result.
After creating the Test in Jira and associating it with requirements, etc., you can export the specification of the test to a Cucumber .feature file via the REST API, or the Export to Cucumber UI action from within the Test/Test Execution issue or even based on an existing saved filter. A plugin for your CI tool of choice can be used to ease this task.
The coverage and the test results can be tracked in the "requirement" side (e.g. user story).
After being exported, the created .feature file will be similar to the original but will contain the references to the Test issue key and the covered requirement issue key.
The steps correspond to reusable blocks, defined as @StepDef scenarios within meta-feature files like the following one. This is the automation glue.
In this example, we're assuming that this meta-feature is not imported to Xray nor managed there; thus, it will probably live in the VCS.
Gwen loads both standard and meta-features and finds the right code to execute.
After running the tests and generating the Cucumber JSON report (e.g., merged-test-results.json), it can be imported to Xray via the REST API or the Import Execution Results action within the Test Execution.
The cucumber-json-merge utility may be handy to merge the results of each feature.
The execution screen details will provide information on the test run result that includes step-level information including duration.
As shown above, besides a detailed error message, screenshots are also automatically available on failed steps.
On the “requirement”/user story side (i.e the “feature”) we can also see how this result impacting on the coverage.
If we wanted to correct the previous error, in this case, we would need to correct the meta-feature file containing the specification of the step “Then the first result page should open a Gwen page” and run the tests again.
Using Git or other VCS as master
You can edit your .feature and .meta files outside of Jira (eventually storing them in your VCS using Git, for example).
In any case, you'll need to synchronize your .feature files to Jira so that you can have visibility of them and report results against them.
Thus, you need to import your .feature files to Xray/Jira; you can invoke the REST API directly or use one of the available plugins/tutorials for CI tools.
Each Scenario of each .feature will be created as a Test issue that contains unique identifiers, so that if you import once again then Xray can update the existent Test and don't create any duplicated tests.
Afterward, you can export those features out of Jira based on some criteria, so they are properly tagged, run them and import back the results to correct entities in Xray.
If we change the specification (i.e. the Gherkin scenarios), we need to import the .feature(s) once again.
Therefore, in the CI we always need to start by importing the .feature file(s) to keep Jira/Xray on synch.
FAQ and Recommendations
Please see this page.
- Testing in BDD with Gherkin based frameworks (e.g. Cucumber)
- Automated Tests (Import/Export)
- Exporting Cucumber Tests - REST