Page History
| Table of Contents |
|---|
Overview
Postman, more than and a utility, is a collaboration platform for developing APIs.
...
With Postman, comes also a built-in (test) collection runner; it is also possible to execute tests from the outside, using a CLI tool named "newman" Newman.
Concepts
- request: an API request (e.g. HTTP POST on some URL, with some values)
- authentication: authentication for the API request (e.g. HTTP basic auth, etc); can be defined at multiple levels and inherited
- collection: a way of grouping multiple requests
- folders within the collection: a way to better organize requests within the collection
- variables: can be defined at multiple levels (e.g. global, collection, environment, local, ...)
- test: a test; can be defined at request, folder or collection level
- pre-request script: some code execute before each test; can also be defined at request, folder or collection level
- environment: an abstraction of some test environment that describes a context for running the requests; it consists of one description plus a set of variables with their corresponding values
Implementing tests
Tests can be implemented using Javascript and making use of Postman APIs/objects supported by Chai assertions.
One or more tests can be defined at the request level, or even at the whole collection level.
Pre-request scripts may be useful as a means to initialize some data before the test or to implement some test setup code.
Variables
AuthenticationOne thing that may be useful, is pre-r
Requirements
- Postman
- Jenkins plugin (optional)´
Example
Postman Echo API
In this example, we're going to use Postman' sample Echo API as a way to showcase some tests and their integration with Xray.
...
The collection (or a subset of its tests) can be run using the Collection Runner.
The runner shows the overall count for the number of passed and failed tests. We can also see the assertion error on failed tests; in this case, saving the response (setting the proper flag above) can help us better understand what is happening.
Running the tests can also be done from the command line or from within Jenkins (or any other CI/CD tool). This can be achieved using Newman.
In order to run Newman, we need to provide a path or a URL to our collection.
In this case, we'll obtain a public link to it.
Then we need to decide which reporter to use. Newman provides a built-in JUnit reporter; however, better alternatives exist such as junitxray or junitfull.
; this will produce a XML based report (e.g. output.xml).
| Info | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||||
The standard Newman junit reporter produces <testcase> entries in the JUnit XML report that can be misleading as tests will be identified on the Postman test description, which can be similar between different tests (e.g. "response is ok"). Therefore, two alternative reporters arise: newman-reporter-junitxray and newman-reporter-junitfull "newman-reporter-junitxray" (simply known as "junitxray"), will create <testcases> per each request, which in the end will lead to corresponding Test issues in Xray. This means that there won't be explicit visibility for each Postman test on that request, as they will be treated just as one. "newman-reporter-junitfull" (simply known as "junitfull"), on the other hand, will produce one <testcase> per each Postman test, which will lead to the same number of corresponding Test issues in Xray. If you aim just to have high-level overview of the request, then "junitxray" reporter will be preferable; otherwise, "junitfull" may be a better option.
|
| Code Block | ||
|---|---|---|
| ||
newman run https://www.getpostman.com/collections/c7334a5cf52a90639a48 -r 'cli,junit,junitfull,junitxray' --reporter-junit-export postman_echo_junit.xml --reporter-junitfull-export postman_echo_junitfull.xml --reporter-junitxray-export postman_echo_junitxray.xml -n 1 |
Importing results is as easy as submitting them to the REST API with a POST request (e.g. curl), or by using one of the CI plugins available for free (e.g. Xray Jenkins plugin).
...
Within the execution screen details, accessible from each row, you can look at the Test Run details which include the overall result and also specifics about each keyword, including duration and status.
