Page History
| Table of Contents |
|---|
Overview
This add-on Xray enables an easy integration between with Jenkins and through the "Xray for JIRA Jenkins Plugin", providing the means for successful Continuous Integration by allowing users to report automated testing results.
...
...
Please take a look at the general guidelines for Working with automated Tests and also the specifics for Testing with Cucumber.
Release Notes
| Children Display |
|---|
Installation
The installation is made manually. For more information on how to install add-ons, please refer to how to install add-ons.
| Info | ||
|---|---|---|
| ||
This add-on This app was tested against Jenkins v2.32.x and it may not work properly with previous versions versions. |
Manual Installation
| Info | ||
|---|---|---|
| ||
You may download the latest version of the Jenkins plugin from the latest Release Notes. |
If you have the add-on file (a file with extension .hpi), in order to manually install Xray for JIRA Jenkins Plugin, you need to perform the following steps:actual xray-for-jira-connector.hpi file,
- Go to the Update Center of Jenkins ("in Manage Jenkins " > " Manage Plugins");.
- Select the advanced tab;
- On In the Upload Plugin section, click upload and select the file
xray-for-jira-connector.hpifile.
...
| title | Download the latest version of the Jenkins Plugin |
|---|
...
- .
Configuration
Xray for JIRA Jenkins Plugin is configured in the global settings configuration page Manage Jenkins > Configure System > Xray for JIRA Jira configuration.
...
Jira servers
The JIRA Jira servers configuration defines connections with JIRA Jira instances.
To add a new JIRA Jira instance connection, you need to specify some properties need to be specified:
- Configuration alias;
- Server Address: The address of the JIRA Jira Server where Xray is running;
- Authentication:
- User: Username;username
- Password.
Creating a new Project
The project is where the work that should be performed by Jenkis Jenkins is configured.
For this add-on app, you should configure a "Freestyle project". In the home page, start by clicking on the option " New Item ", then select "> Freestyle project", provide it a name, and then click " OK".
Build Steps
Build steps are the building blocks of the build process. These need to be defined in the project configuration.
The add-on app provides one build step for exporting Cucumber Scenario/Scenario Outlines from JIRA Jira as .feature files, and one post-build action which publishes the execution results back to JIRAJira, regardless of the build process status.
| Info | ||
|---|---|---|
| ||
| The fields of the tasks may take advantage of the Jenkins Environment variables, which can be used to populate fields such as the "Revision" for specifying the source code's revision. For more information, please see : Jenkins set environment variables. |
Xray: Cucumber Features Export Task
By using this This build step , it will export the Cucumber Tests (i.e., Scenario/Scenario Outlines) in .feature or bundled in a .zip file. The rules for exporting are defined here.
This build step corresponds to invoke It invokes Xray's Export Cucumber Tests REST API endpoint (see more info information here).
Configuration
...
Some fields need to be configured in order to export the Cucumber Tests. As input, you can either specify issue keys (see the endpoint documention here) or the ID of the saved filter in Jira.
| field | description |
|---|---|
| JIRA Jira instance | The JIRA Jira instance where Xray is running |
| Issue keys | Set of issue keys separated by ";" |
Filter ID | A number that indicates the filter idID |
| File path | The relative path of the directory where the features should be exported to; normally, this corresponds to the "features" folder of the Cucumber project that has the implementation steps. Note: The directory will be created if it does not exist. |
Xray: Results Import Task
The add-on app provides easy access to Xray's Import Execution Results REST API endpoints (see more info information here). Therefore, and therefore it mimics the endpoints input parameters.
It supports importing results in Xray's own JSON format, Cucumber, Behave, JUnit, Nunitand NUnit, among others.
...
Configuration
| field | description |
|---|---|
| JIRA Jira instance | The JIRA Jira instance where Xray is running |
| Format | A list of test result formats and it's its specific endpoint |
| Execution Report File | The results relative file path Note: regex is not supported. |
Additional fields
Depending on the choosen chose test result format and endpoint, some additional fields may need to be configured.
| format and specific endpoint | field | description | |||||
|---|---|---|---|---|---|---|---|
Behave JSON multipart Cucumber JSON multipart NUnit XML multipart JUnit XML multipart Robot XML multipart | Test execution fields | An object (JSON) specifying the fields for the issue. You may either specify the object , either directly in the field , or in the file path.
| |||||
NUnit XML JUnit XML Robot XML | Project key | Key of the project where the Test Execution (if the Test Execution Key field wasn't provided) and the Tests (if they aren't created yet) are going to be created | |||||
| Test execution key | Key of the Test Execution | ||||||
| Test plan key | Key of the Test Plan | ||||||
| Test environments | List of Test Environments separated by ";" | ||||||
| Revision | Source code's revision being target by the Test Execution | ||||||
| Fix version | The Fix Version associated with the test execution (it supports only one value) |
Examples
Cucumber
In a typical Cucumber Workflow, after having created a Cucumber project and the Cucumber tests specified in JIRA Jira, you may want to have a project that exports the features from JIRAJira, executes the automated tests on a CI environment and then imports back its results.
...
- Pulling the Cucumber project;
- Exporting Cucumber features from JIRA Jira to your Cucumber project;
- Executing the tests in the CI environment;
- Importing the execution results back to JIRA.Jira
Exporting Cucumber features
To start the configuration, add the build step Xray: Cucumber Features Export Task.
After that, configure it.
In this example, we configured the task to extract the features from a set of issues (PROJ-78 and PROJ-79) , to the folder that holds the Cucumber project.
Importing the execution results
To start the configuration, add the post-build action Xray: Results Import Task.
After that, configure it.
In this example, we configured the task to import the Cucumber JSON results back to JIRAJira.
Finished Once all configurations , hit the save button in are done, click Save at the bottom of the page.
After running the job, the expected result is a new Test Execution issue created in the JIRA Jira instance.
Importing the execution results with user-defined field values
For Cucumber, Behave, JUnit, Nunit and Robot, Xray for JIRA Jenkins Plugin provides an option for users who want Jenkins allows you to create new Test Executions and have control over newly-created Test Execution fields. The way it provides this is by allowing the user to You can send two files, the normal execution result file and a JSON file similar to the one JIRA one Jira uses to create new issues. More details regarding how JIRA Jira creates new issues here.
For this scenario and example, the import task needs to be configured with the Cucumber JSON Multipart format. When selecting this option, you can , additionally , configure the Test Execution fields in one of two ways:
- Inserting Insert the relative path to the JSON file containing the information;, or
- Insert Or by inserting the JSON content directly in the field.
...
| Code Block | ||
|---|---|---|
| ||
{
"fields": {
"project": {
"key": "PROJ"
},
"summary": "Test Execution for Cucumber results (Generated by job: ${BUILD_TAG})",
"issuetype": {
"id": "10102"
}
}
} |
...
And configured the task to import the Cucumber JSON Multipart results back to JIRAJira.
Finished Once all configurations are done, hit the save button in the click Save at the bottom of the page.
After running the job, the expected result is a new Test Execution issue created in the JIRA Jira instance, with the Test Execution fields as specified in the Jenkins build step configuration.
JUnit
Apart from supporting Cucumber natively Cucumber, Xray for JIRA enables you to take advantage of many other testing frameworks , for instance like JUnit. In this sense, Xray for JIRA Jenkins Plugin lets you to import other results in other formats besides Cucumber JSON.
In a scenario where the user wants If you want to import JUnit XML reports, a typical Job outline would be:
- Pulling the JUnit project;
- Executing the tests in the CI environment;
- Importing the execution results, including Tests, to JIRA.
Importing the execution results
To start the configuration, add the post-build action Xray: Results Import Task.
After that, configure it.
In this example, we have a configuration where the format JUnit XML is format is chosen.
After running the plan, the expected result is a new Test Execution issue created in the JIRA instance.
Troubleshooting
The build process is failing with status code 403
Your build process is failing and when When you check its the log, it presents you has the following:
By default, when the user you successively tries try to log in to JIRA into Jira with the wrong credentials, the next time you try to log in, JIRA Jira instance will prompt you to provide a CAPTCHA the next time you try to log in. It is not possible to provide this information via the build process, so it will fail with status code 403 Forbidden.
You will need to log in to JIRA into Jira via the browser and provide the CAPTCHA.
In case If you are a JIRA Jira administrator, you can go to JIRA Jira administration > User Management and reset the failed login for the user who failed to log in.
