Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  1. Teams adopting BDD, start by defining a user story and clarify it using gherkin Gherkin Scenario(s); usualy, gherkin Gherkin Scenario(s)/Scenario Outline(s) are specified directly in Jira, using Xray
  2. Teams adopting BDD but that favour a more Git based approach (e.g. GitOps). In this case, stories would be defined in Jira but gherkin Gherkin .feature files would be specified using some IDE and would be stored in Git, for example
  3. Teams not adopting BDD but still using SpecFlow, more as an automation framework. Sometimes focused on regression testing; sometimes, for non-regression testing. In this case, SpecFlow would be used...
    1. With a user story or some sort of "requirement" described in Jira
    2. Without any story/"requirement" described in Jira

...

For the purpose of this tutorial, we'll use a simple , dummy Calculator implemented in a C# class as our target for testing.

The code is structure in two projects: one that we will be targeted by testing and one with the test code and SpecFlow.


Info
titleTry it yourself!

The code on this tutorial is available in the tutorial-csharp-specflow GitHub repository.

You can fork it, adapt, and try it for youself.

Code Block
languagec#
titleCalculator.cs
collapsetrue
namespace SpecFlowExamples
{
    public class Calculator
    {
        public int FirstNumber { get; set; }

        public int SecondNumber { get; set; }

        public int Add()
        {
            return FirstNumber + SecondNumber;
        }

        public int Subtract()
        {
            return FirstNumber - SecondNumber;
        }

        public int Divide()
        {
            if (FirstNumber == 0 || SecondNumber == 0)
            {
                return 0;
            }
            else
            {
                return FirstNumber / SecondNumber;
            }
        }

        public int Multiply()
        {
            return FirstNumber * SecondNumber;
        }
    }
}

This tutorial, has the following requirements:

...


Requirements

  • .NET 5.0 (.NET 6.0 is not supported by SpecFlow together with the SpecFlow+ Runner) 
  • mono (SpecFlow reporting utility utility requires mono)
  • Packages
    • SpecFlow<SpecFlow <= 3.9.40
    • SpecRun.SpecFlow <= 3.9.31
    • FluentAssertions

...

  1. define the story (skip if you already have it)
  2. create Scenario/Scenario Outline as a Test in Jira; usually, it would be linked to an existing "requirement"/Story (i.e. created from the respective issue screen)
  3. implement the code related to Gherkin statements/steps and store it in Git, for example. To start, and during development, you may need to generate/export the .feature file to your local environment
  4. commit previous code to Git
  5. checkout the code from Git
  6. generate .feature files based on the specification made in Jira
  7. run the tests in the CI
  8. obtain the report in Cucumber JSON format
  9. import the results back to Jira


Image RemovedImage Added

Note that steps (5-9) performed by the CI tool are all automated, obviously.

...

The coverage and the test results can be tracked in the "requirement" side (e.g. user story). In this case, you may see that coverage changed from being UNCOVERED to NOTRUN (i.e. covered and with at least one test not run).

Image RemovedImage Added

Additional tests could be created, eventually linked to the same Story.

...

You can then export the specification of the test to a gherkin Gherkin .feature file via the REST API, or the Export to Cucumber UI action from within the Test/Test Execution issue or even based on an existing saved filter. As source, you can identify Test, Test Set, Test Execution, Test Plan or "requirement" issues. A plugin for your CI tool of choice can be used to ease this task.

...

Code Block
languagexml
titleDefault.srprofile
collapsetrue
<?xml version="1.0" encoding="utf-8"?>
<TestProfile  xmlns="http://www.specrun.com/schemas/2011/09/TestProfile">
  <Settings name="dummy" projectName="Calculator.Specs" />
  <Execution stopAfterFailures="3" testThreadCount="1" testSchedulingMode="Sequential" />

  <TestAssemblyPaths>
    <TestAssemblyPath>Calculator.Specs.dll</TestAssemblyPath>
  </TestAssemblyPaths>

  <Report copyAlsoToBaseFolder="false">
    <Template name="../../../CucumberJson.cshtml" outputName="cucumber.json" existingFileHandlingStrategy="Overwrite" />
  </Report>

</TestProfile>


"SpecFlow+ Runner" supports customizable reports and provides a report template tailored for the generation of Cucumber JSON reports. You can can copy it from your local packages directory (e.g., .nuget/packages/specrun.runner/3.9.31/templates/CucumberJson.cshtml) to your test project directory. This template has changed slightly with SpecFlow 3, so please make sure you use the proper one for your scenario.

...

  1. look at the existing "requirement"/Story issue keys to guide your testing; keep their issue keys
  2. specify SpecFlow/Gherkin .feature files in your IDE supporting SpecFlow/Gherkin and store it in Git, for example. Meanwhile, you may decide to import/synchronize them Xray to provision or update corresponding Test and/or Precondition entities
  3. implement the code related to Gherkin statements/steps and store it in Git, for example. 
  4. commit code and .feature file(s) to Git
  5. checkout the code from Git
  6. import/synchronize the .feature files to Xray to provision or update corresponding Test and/or Precondition entities
  7. export/generate .feature files from Jira, so that they contain references to Tests and requirements in Jira
  8. run the tests in the CI
  9. obtain the report in Cucumber JSON format
  10. import the results back to Jira


Image Added


Note that steps (5-10) performed by the CI tool are all automated, obviously.

To import .features to Jira we can either use the REST API or a CI tool. To export tagged .features from Jira, we can do it directly from Jira, by the REST API or using a CI tool.


Info
titleLearn more

Please check the tutorial Testing using Cucumber in Java which showcases this flow for Cucumber+Java and adapt it accordingly to the SpecFlow use case.


FAQ and Recommendations

Please see this page.

...

...