Page History
| Table of Contents |
|---|
Overview
...
Then we need to decide which Newman reporter to use. Newman provides a built-in JUnit reporter; however, better alternatives exist such as junitxray or junitfull.; this will produce a XML based report (e.g. output.xml).
| Info | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||||
The standard Newman junit reporter produces <testcase> entries in the JUnit XML report that can be misleading as tests will be identified on the Postman test description, which can be similar between different tests (e.g. "response is ok"). Therefore, two alternative reporters arise: newman-reporter-junitxray and newman-reporter-junitfull "newman-reporter-junitxray" (simply known as "junitxray"), will create <testcases> per each request, which in the end will lead to corresponding Test issues in Xray. This means that there won't be explicit visibility for each Postman test on that request, as they will be treated just as one. "newman-reporter-junitfull" (simply known as "junitfull"), on the other hand, will produce one <testcase> per each Postman test, which will lead to the same number of corresponding Test issues in Xray. If you aim just to have high-level overview of the request, then "junitxray" reporter will be preferable; otherwise, "junitfull" may be a better option.
|
| Code Block | ||||
|---|---|---|---|---|
| ||||
npm install -g newman
# install one of the following ones
npm install -g newman-reporter-junitxray
npm install -g newman-reporter-junitfull |
Whenever running Newman, we can specify one or more reporters (if we want to), including a CLI friendly one.
| Code Block | ||
|---|---|---|
| ||
newman run https://www.getpostman.com/collections/c7334a5cf52a90639a48 -r 'cli,junit,junitfull,junitxray' --reporter-junit-export postman_echo_junit.xml --reporter-junitfull-export postman_echo_junitfull.xml --reporter-junitxray-export postman_echo_junitxray.xml -n 1 |
If using Jenkins, we need to configure a build step to execute "newman" command.
Importing results is as easy as submitting them to the REST API with a POST request (e.g. curl), or by using one of the CI plugins available for free (e.g. Xray Jenkins plugin).
The following screenshots show the Jenkins configuration.
An unstructured (i.e. "Generic") Test issue will be auto-provisioned the first time you import the results, based on the name identification of the test case and of the corresponding test suites(see notes for possible Newman reporters above).
If you maintain the test case name and the respective test suites, the Test will be reused on subsequent result imports. You may always enforce the results to be reported against an existing Test, if you wish so: just specify its issue key as a tag.
...
A Test Execution will be created containing results for all test cases executed. In this case, you can see that it is also linked back to an existing Test Plan where you can track the consolidated results from multiple "iterations" (i.e. Test Executions).
Within the execution screen details, accessible from each row, you can look at the Test Run details which include the overall result and also specifics about each keyword, including duration and status.