Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

In this scenario, common if you're using JUnit, TestNG, NUnit, xUnit, Robot framework, developers/testers write the automated tests as code accordingly with the automation framework being used.

After that, they just need to submit the results to Xray and it will automaticaly automatically create Test entities for each of them (or update existing ones if they already exist), so you have visibility of the automation results in Xray. 

...

Gliffy Diagram
namegeneric_tests_automatic_provisioning_workflow Copy
pagePin119


  1. Implement the automated test code, store it in the source control system, and put the reference to the Test in Jira (i.e., the issue key).
    1. This depends on the testing framework: it can be as a "tag" or as the test name, if the framework supports that.
  2. Execute tests in the CI environment.
  3. Report execution results using the format specific to the automation framework (either using the REST API directly or through Bamboo/Jenkins add-ons).
  4. Create our update existing Tests in Xray
    1. In the first iteration, Tests will be automaticaly automatically created in Xray and the Generic Test Definition field will act as the test identifier (e.g. classname class name plus classmethod class method corresponding to automated test). Xray is able to identify uniquely from the report file
    2. In second and onward iterations, existing Tests will be updated; Xray will use the Generic Test Definition field as the test identifier in order to find the existing Test in JIRA
    3. Depending on the test automation framework being used, links can also be automaticaly automatically created between the Tests and the requirements

...

Info
titleLearn more

You can see many examples in the page Integrating with Testing Frameworks, including:

...

Gliffy Diagram
namegeneric_tests_workflow
pagePin119


  1. Create a Generic Test in Jira.
    1. The test will be uniquely identified by the issue key; however, the Generic Test Definition custom field may be used as a more friendly way to identify the test (e.g., with the name of the test class/method, name of the script or executable implementing the test).
    2. The test can be automatically created in Jira when importing test results; as mentioned in the previous point, the Generic Test Definition field acts as the test identifier.
  2. Implement the automated test code, store it in the source control system, and put the reference to the Test in Jira (i.e., the issue key).
    1. This depends on the testing framework: it can be as a "tag" or as the test name, if the framework supports that.
  3. Execute tests in the CI environment.
  4. Report execution results using Xray JSON format (and optionally, the REST API).
    1. In order to build the JSON file containing the test results, you will need to reference the Tests in Jira by their issue key.

...

Info
titleLearn more

See Automated Tests (Import/Export) for more information on Xray's JSON format and Import Execution Results - REST to know how to import those test results using the REST API.

...

Info
titleExploratory testing

...

"Generic" (i.e. unstructured) tests can also be used as an abstraction of exploratory tests.

Learn more on  Performing exploratory testing with Xray

You can use Generic Tests as means to abstract some exploratory session. You can use the Generic Test Definition field to identify the session charter (e.g. goals of that session/mission).

During the execution of your session, you can attach evidence, report if all went fine and if not you may create defects. You may also decide to create a Manual Test, with the reproducible steps, for validating that the problem does not arise in the future.

Info
titlePlease note

If you have multiple testers involved in your exploratory session, then you can create different Test Executions each one assigned to a different tester; you may use the Test Execution planning dates to define the fixed timeframe allocated to your session. The Test Execution description can be used to specify the tester charter, which complements and focuses the session charter for the tester(s) involved in that Test Execution. Please note that a Test Execution can only contain the same Test once.