Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Besides this, Xray also provides specific integration for Cucumber and related variants.


Gliffy Diagram
displayNameintegration_with_automation_frameworks_aug2019
nameintegration_with_automation_frameworks_aug2019
pagePin12

Summary of features per framework

...


JUnit 4

TestNG

NUnit 2.6/3.x

xUnit 2

Cucumber/Behave

Robot Framework

Xray JSON

Abstract automated test as a Test

(map an automated test to a Test issue in Jira)

Yes

(as a Generic Test)

Yes

(as a Generic Test)

Yes

(as a Generic Test)

Yes

(as a Generic Test)

Yes

(as a Cucumber Test)

Yes

(as a Generic Test)

No

Tests must exist beforehand

Make Test specification in Jira

(specify the Test itself in Jira)


NoNoNoNoYesNoYes

Create Tests from results

(create Tests whenever importing results)

Yes

Yes

Yes

Yes

No*

(but there is and endpoint for importing Cucumber features, which will create Tests for the respective Scenarios/Scenario Outlines)

YesNo

Uniquely identify Tests

(identify existing Tests whenever importing results, avoiding duplication of Test issues)

Yes

Yes

Yes

YesYesYes

Yes*

(based on the provided Test issue keys)

Import results

(importing results by REST API or UI)

Yes

Yes

Yes

YesYesYesYes

Import “labels”

(create labels in the Test issues)

No

Yes

(labels may be specified in the test's source code)

Yes

(labels may be specified in the test's source code)

Yes

(labels may be specified in the test's source code)

Yes*

(this is only available when using the endpoint for importing Cucumber features; it's not possible when importing results)

Yes

(labels may be specified in the test's source code)

No

Automatic linking to requirements

(create links to requirements)

Yes*

(requirement's issue key may be specified in the JUnit report)

Yes

(requirement's issue key may be specified in the test's source code)

Yes

(requirement's issue key may be specified in the test's source code)

Yes

(requirement's issue key may be specified in the test's source code)

Yes*

(this is only available when using the endpoint for importing Cucumber features; it's not possible when importing results)

Yes

(requirement's issue key may be specified in the test's source code)

No

Semantic on the results

(present execution details in the execution screen)

-

Yes

(for every "test" section in the XML config file and parameterized Tests)

Yes

(Test Suites and parameterized Tests)

Yes

(parameterized Tests)

Yes

(steps for Scenario/Scenario Outline and Background)

Yes

(keyword “steps”)

Yes*

(semantic is implicit, because the Test must be created beforehand)