Overview
This add-on enables an easy integration between Bamboo and Xray for JIRA, providing the means for successful Continuous Integration by allowing users to report automated testing results.
Learn more
Please take a look at the general guidelines for Working with automated Tests and also the specifics for Testing with Cucumber.
Release Notes
Installation
The installation can be made either automatically or manually. For more information on how to install add-ons, please refer to how to install add-ons.
Requirements
This add-on requires Java 1.8 to be installed in the Bamboo server.
It was tested against Bamboo v5.9.x and it may not work properly with previous versions.
Automatic Installation
You can install Xray for JIRA add-on for Bamboo through the UPM (Universal Plugin Manager), by going to Bamboo's administration, in "Add-ons" > "Find new add-ons" section.
Manual Installation
If you have the add-on file (a file with extension .jar), in order to manually install Xray for JIRA add-on for Bamboo, you need to perform the following steps:
- Go to the administration section of Bamboo, look for the Add-Ons -> Manage Add-Ons menu
- Select option upload Add-On and upload the xray-bamboo-X.X.X.jar file. (X.X.X is the file version).
Configuration
Xray for JIRA add-on for Bamboo is configured in Bamboo Administration, page Add-Ons > Xray for JIRA configuration.
JIRA servers
The JIRA servers configuration defines connections with JIRA instances.
To add a new JIRA instance connection, some properties need to be specified:
- Name: Configuration alias;
- Server Address: The address of the JIRA Server where Xray is running;
- Authentication information:
- User: Username;
- Password.
Tasks
The add-on provides one task for exporting Cucumber Scenario/Scenario Outlines from JIRA as .feature files, and another task for importing execution results.
Please note
Xray: Cucumber Features Export Task
By using this task, it will export the Cucumber Tests (i.e. Scenario/Scenario Outlines) in .feature or bundled in a .zip file. The rules for exporting are defined here.
This task corresponds to invoke Xray's Export Cucumber Tests REST API endpoint (more info here).
Configuration fields
Some fields need to be configured in order to export the Cucumber Tests.
field | description |
---|---|
Task description | A short task description |
JIRA instance | The JIRA instance where Xray is running |
Issue keys | Set of issue keys separated by ";" |
Filter ID | A number that indicates the filter id |
File path | The relative path of the directory where the features should be exported to; normaly this corresponds to the "features" folder of the cucumber project that has the implement steps. Note: the directory will be created if it does not exist. |
Xray: Results Import Task
The add-on provides easy access to Xray's Import Execution Results REST API endpoints (more info here), and therefore it mimics the endpoints input parameters.
It supports importing results in Xray's own JSON format, Cucumber, Behave, JUnit, Nunit, among others.
Configuration fields
field | description |
---|---|
Task description | A short task description |
JIRA instance | The JIRA instance where Xray is running |
Format | A list of test result formats and it's specific endpoint |
Execution Report File | The relative path of results file, including file name. Note: regex is not supported. |
Additional fields
Depending on the choosen test result format, some additional fields may need to be configured.
format and specific endpoint | field | description |
---|---|---|
Behave JSON multipart Cucumber JSON multipart NUnit XML multipart JUnit XML multipart Robot XML multipart | Test execution fields | An object (JSON) specifying the fields for the issue. You may either specify the object, directly in the field, or the file path. Learn more The custom field IDs can be discovered using the JIRA REST API Browser tool included in JIRA. Each ID is of the form "customfield_ID". Another option, which does not require JIRA administration rights, can be invoking the "Get edit issue meta" in an existing issue (e.g. in a Test issue) as mentioned here. |
NUnit XML JUnit XML Robot XML | Project key | Key of the project where the Test Execution (if the Test Execution Key field wasn't provided) and the Tests (if they aren't created yet) are going to be created |
Test execution key | Key of the Test Execution | |
Test plan key | Key of the Test Plan | |
Test environments | List of Test Environments separated by ";" | |
Revision | Source code's revision being target by the Test Execution | |
Fix version | The Fix Version associated with the test execution (it supports only one value) |
Please note
The "Xray Import Execution Results" task should be configured as a final task in order to ensure that results are imported to Xray even when there are failures during test execution.
Examples
Cucumber
In a typical Cucumber Workflow, after having created a Cucumber project and the Cucumber tests specified in JIRA you may want to have a plan that exports the features from JIRA, executes the automated tests on a CI environment and then imports back its results.
For this scenario, the Bamboo plan would be configured with a set of tasks responsible for:
- Pulling the Cucumber project;
- Exporting Cucumber features from JIRA to your Cucumber project;
- Executing the tests in the CI environment;
- Importing the execution results back to JIRA.
Exporting Cucumber features
To start the configuration, add the task Xray: Cucumber Features Export Task. After that, configure it and hit the Save button.
In this example we configured the task to extract the features from a set of issues (PROJ-78 and PROJ-79), to the folder that holds the Cucumber project.
Importing the execution results
To start the configuration, add the task Xray: Results Import Task. After that, configure it and hit the Save button.
In this example, we configured the task to import the Cucumber JSON results back to JIRA.
After running the plan, the expected result is a new Test Execution issue created in the JIRA instance.
Importing the execution results with user-defined field values
For Cucumber, Behave, JUnit, Nunit and Robot, Xray for JIRA add-on for Bamboo provides an option for users that want to create new Test Executions and have control over newly created Test Execution fields. The way it provides this is by allowing the user to send two files, the normal execution result file and a JSON similar to the one JIRA uses to create new issues. More details regarding how JIRA creates new issues here.
For this scenario and example, the import task needs to be configured with Cucumber JSON Multipart format. When selecting this option you can, additionally, configure the Test Execution fields in two ways:
- Inserting the relative path to the JSON file containing the information;
- Or by inserting the JSON content directly in the field.
In this example, we configured the following object:
{ "fields": { "project": { "key": "PROJ" }, "summary": "Test Execution for Cucumber results (Generated by job: ${bamboo.buildKey})", "issuetype": { "id": "10102" } } }
And configured the task to import the Cucumber JSON Multipart results back to JIRA
After running the plan, the expected result is a new Test Execution issue created in the JIRA instance, with the Test Execution fields as specified in the Bamboo task configuration.
JUnit
Apart from supporting natively Cucumber, Xray for JIRA enables you to take advantage of many other testing frameworks, for instance JUnit. In this sense, Xray for JIRA add-on for Bamboo lets you to import other results in other formats besides Cucumber JSON.
In a scenario where the user wants to import JUnit XML reports, a typical Job outline would be:
- Pulling the JUnit project;
- Executing the tests in the CI environment;
- Importing the execution results, including Tests, to JIRA.
Importing the execution results
To start the configuration, add the task Xray: Results Import Task. After that, configure it and hit the Save button.
In this example we have a configuration where the format JUnit XML is chosen.
After running the plan, the expected result is a new Test Execution issue created in the JIRA instance.
Troubleshooting
The build process is failing with status code 403
Your build process is failing and when you check its log, it presents you the following:
By default, when the user successively tries to log in to JIRA with wrong credentials, the next time you try to log in, JIRA instance will prompt you to provide a CAPTCHA. It is not possible to provide this information via the build process, so it will fail with status code 403 Forbidden.
You will need to log in to JIRA via browser and provide the CAPTCHA.
In case you are a JIRA administrator, you can go to JIRA administration > User Management and reset the failed login for the user who failed to log in.