What you'll learn
- How to integrate with Travis CI
- Run the Travis CI pipeline push the test report to Xray
- Validate that the test results are available in Jira
Source-code for this tutorial
- code is available in GitHub
Travis CI is a continuous integration service used to build and test projects hosted at GitHub, Bitbucket, GitLab and Assembla.
It will automatically detects when a commit has been made and pushed to a repository that is configured with Travis CI, and each time this happens, it will try to execute what is defined in the Travis CI yaml file (build, pack, tests, etc).
For this example we will use a simple calculator application that is part of the code available and tests defined in JUnit. We will use that repository to start a pipeline in Travis CI, where we will execute the compilation and tests, and report back to Xray.
What you'll need:
- Java and Maven installed
- Travis CI account linked to GitHub (in this case, you can also use Assembla, Bitbucket or GitLab)
In the repository you will find a calculator application and also tests that are validating those abilities.
The test consists of validating the supported operations (addition, subtraction, division and multiplication) of calculator application.
Once the code is implemented, we can compile and execute the tests with the following command:
The results are immediately available in the terminal
In this example, all test have succeed, the output generated to the terminal is the above one and the corresponding JUnit report is below:
- For more information regarding the execution and integration of JUnit tests with Xray please check this article.
Integrating with Travis CI
As we saw in the previous example , where we are producing a JUnit report locally with the test results; it is now a matter of executing those tests in Travis CI and importing their results to your Jira instance. You can do this by simply submitting automation results to Xray through the REST API or using the Jira interface to do so. In this case we will show how to integrate with Travis CI using API calls.
Please follow the Travis CI documentation to configure your GitHub repo.
In order to integrate Travis CI with Xray, we define the CLIENT_ID and CLIENT_SECRET as environment variables so they can be later used in the Xray API calls mentioned in the .travis.yml file.
For that please go to the configuration of your repo in Travis CI and proceed as follows.
We have also added the necessary .travis.yml file to our project in GitHub in order to configure our pipeline.
On the file we can see one job definition that is compiling the code (like we did locally):
Besides, we can also see two API calls: one for authentication with Xray (with escaping characters) and another to submit the results.
Here is where we are using the CLIENT_ID and CLIENT_SECRET (based on an API key pair from your Xray instance) to authenticate with Xray and obtain the token used in the following requests (for more details about this process please check this documentation).
The response of this request will return the token to be used in the subsequent requests for authentication purposes.
With this command, you will create a new Test Execution for our tests, linked to the referred Test Plan "CAL-18". Tests will be autoprovisioned at the moment of the first import of results and will be reused afterward.
Once everything is configured you can start the pipeline by committing to your code repository (in our case pushing code in GitHub).
On Travis CI side it will look like this:
The log that will appear in the bottom of the page in Travis CI will output all steps we are taking:
Let's take a look on what it looks like on the Xray side once the test report is ingested back in, namely on the TestPlan "CAL-18":
As we can see the test plan have 4 Tests, all currently marked as being passed.
If we drill down to the latest Test Run, we can see the details of that specific execution:
Triggering automation from Xray side
Please have a look at Taking advantage of Jira Cloud built-in automation capabilities to see some examples of how automation can be triggered from Xray side.
- Taking advantage of Jira Cloud built-in automation capabilities