In this tutorial, we will create a Java Test class with multiple Test Cases, implemented in Java.
The tutorial shows two different approaches for providing additional meta-information for each Test method, one of them will require developing specific code for extending TestNG.
Note that providing additional information on the Test results is optional.
The automated tests validate a Calculator class and exploits some TestNG features such as the ability of validating the same Test against multiple input values, and also the possibility of linking Tests with requirements in Jira using attributes on the test results, amongst other things.
Example 1: Using TestNG standard capabilities
This example uses the standard built-in TestNG capabilities, without making use of TestNG's advanced features such as annotations (which in turn would require specific code to handle them).
The test class provides some methods marked with the @Test annotation and in some of them additional meta-information is added to the results using the ITestResult object.
Xray is able to process TestNG's ITestResult custom attributes named "test", "requirement" and "labels". Whenever specified, they will allow Xray to identify an existing Xray Test issue to report results to, the requirement to link to and additional labels to add to the Test issue, respectively.
Example 2: Using TestNG annotation capabilities
This examples uses some of the advanced TestNG capabilities, namely the annotation mechanism.
We'll use a specific "Xray" annotation in order to quickly, and in a more elegant way, provide additional meta-information (i.e. "test", "requirement" and "labels") to the Test result, without having to use the ITestResult object in the Test method's code. Whenever specified, they will allow Xray to identify an existing Xray Test issue to report results to, the requirement to link to and additional labels to add to the Test issue, respectively.
This requires some additional side code, that uses TestNG extension mechanisms, in order to process the custom "Xray" annotation shown above. This code is not part of Xray and is only provided for reference; feel free to adapt and customize it to your needs.
We'll define an interface that will be used by a custom TestNG listener (i.e. implementing ITestListener).
The XrayListener must be defined as a service so TestNG is able to load it at runtime.
XrayListener class is responsible for processing "Xray" annotation, get the values of some specific attributes and them as attributes to the ITestResult object.
pom.xml file, we need to make sure test result related attributes are added to the generated XML report. This can be done by setting
generateTestResultAttributes to "true".
After successfully running the tests and generating the TestNG XML report (e.g. testng-results.xml), it can be imported to Xray (either by the REST API or through the Import Execution Results action within the Test Execution).
TestNG's tests are mapped to Generic Tests in Jira, and the Generic Test Definition field contains the namespace, the name of the class, and the method name that implements the Test case.
The Execution Details of the Generic Test contains information about the context, which in this case corresponds to the Test case method, along with the different input values that were validated.
It can also be seen that the Test "CanAddNumbersFromGivenData" was automatically linked to the sum requirement (i.e., the user story "CALC-1234").