Functionize is an intelligent software testing platform for teams of all sizes and skill sets. By combining the power of human insights with machine learning and other AI-based technologies, it's possible to reduce the time and cost spent in testing while accelerating product releases.
Tests can run across multiple browsers and use AI to self-heal upon UI visual or structural changes.
More details about Functionize here.
This integration provides:
- integration with Xray cloud or Server/DC
- ability to link tests to an existing "Test" in Jira/Xray
- ability to track automation results in Xray from testing performed using Orchestrations in Functionize
- Creating a Test Execution linked to the Test Case
- Creating a Test Execution associated to a Test Plan
Mapping of concepts
|Test Cases (Folder organization)||Test Repository|
Test Plan/Test Execution
|Test Execution||Test Execution|
A specific integration for Xray is natively provided by Functionize; in order to configure it, the user must follow the below steps to be able to push the results back to Xray.
First open the Team Page, then the Integrations Tab and on the left, select Xray from the Test Case Management section.
The necessary configuration must be filled:
- In the main panel, provide credentials for the connection. Select Cloud or Server/Data Center, accordingly with your Jira deployment type
- Enter the URL for your Jira project, your Jira username, and your Jira API token (not your password). For more info on Jira API tokens, please check Manage API tokens for your Atlassian account.
- Enter your Xray Client ID and Client Secret (obtain/create these on Xray settings; please check Global Settings: API Keys).
- Click Test Sign In and the status next to the button should update to Connection established with a green check mark when ready.
Finally scroll down to the Project Mappings section.
Here you will select the Functionize and Xray projects you wish to link together and select the appropriate testing environment to be used (only for that environment, in the Functionize side, we will see the integration button appear).
Then proceed to define the mapping between status in each side (map the Functionize test statuses to the Xray test statuses as you would wish them to be reported).
As you can see, multiple projects may be added; however, each one must be configured individually.
To finish the setup click Submit to save your connection details.
Linking Test Cases to Tests
Once you have the setup finished you will need to link the Test Cases (from Functionize) with Tests (in Xray). In order to do that, open the Functionize project you have linked with Xray.
On the left panel, confirm you have selected the testing environment previously mapped with your Xray project (the list of tests in the main panel will display the Xray logo + Link Test).
In each Test that you want to link with a Test in Xray, in the Project view, click the "Link Test" link to open the Xray Linking window.
Select the corresponding Xray test entry > Click Link Test.
Once the mapping is done, the Test name (from Xray) will be listed in the place of Link Test along with a pencil icon to edit the connection at any time.
Import results to Xray
In order to import the results back to Xray, go to the Orchestrations page and create a new Orchestration.
Fill in all the details that are relevant to your execution.
Finally go to the "Advanced" tab to select how you'd like the results of each Orchestration run to be sent to Xray.
The last configuration allows you to associate the orchestration to a Test Plan and/or Fix Version, for that you will find two drop down options, like below, where you can choose the Test Plan (fetched from Xray) and/or the Fix Version to be associated to the Execution (also fetched from Xray, i.e., from the configured project releases).
If you choose to configure those they will be pre-filled on the orchestration view, this is not a mandatory configuration so you can also leave it blank (you will be able to define these per orchestration execution), next press "Add New".
We are taken to a confirmation panel where we can select which of the Test Cases we want to include in this Orchestration.
Once you are happy with the selection, press "Submit" and you will see a new Orchestration appearing in the list.
Now let's execute the Orchestration, for that press the play icon and click Run Orchestration in the confirmation modal.
Once the execution have finished you can check the details by choosing the "Orchestration Results".
On the Orchestration Results panel, if you selected On Demand when setting up your Orchestration, you will see two drop down options, one for choosing a Test Plan and another for choosing a Fix Version (if you already have defined these in the orchestration configuration they will be pre-filled) and a "Submit to Xray" button on the upper right corner.
These combinations will allow you to:
- Define a Test Plan - this option will associate the Test Executions to a Test Plan in Xray (you can use this option without having a Fix Version defined)
- Define the Fix Version - you can choose the Fix Version that you are testing to be associated in the Test Execution (you can use this option without having a Test Plan defined)
If none of the above options are filled, that is ok too, it will mean that you will just submit the Orchestration results directly to a Test Execution (without Fix Version or Test Plan associated with it).
For this example, we will define a Test Plan and a Fix Version and then press the "Submit" button to send those results to Xray.
A modal will open showing the name of the Xray Execution and a link to the execution will be visible under the Submit to Xray button.
The result will be added in the Xray side as a new Test Execution. In Functionize, a Xray link will be provided; when you click it, you will be taken to the Test Execution in the Xray side.
On the Test Execution we can see:
- The Summary, that is a concatenation of the title given to the Orchestration plus the date hour
- In the Description, we can find the link back to Functionize application and a table with the details of the execution.
- The Fix Version we have define in Functionize side, if any
- The Test Plan we have chosen to associate in the Functionize side, if any
If you scroll down you will see two Test Executions (associated with the Tests we linked in the Functionize side).
If we press the Test Execution details icon, we are taken to the Execution Details where we can see the details of that specific execution, namely the two entries that correspond to the execution results in the Functionize side.