Versions Compared

Old Version 3

changes.mady.by.user Claudio Sousa

Saved on

compared with

New Version Current

changes.mady.by.user Nuno Santos

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

When importing execution results using Xray JSON result format in a Continuous Integration environment, you can specify which Test Execution issue to import the results on using the "testExecutionKey" property. Alternatively, you can create a new Test Execution for the execution results and specify the Test Execution issue fields in the "info" object.

Updating an existing Test Run using Xray format REST API will reset all dataset related fields. This means that all current iteration data and dataset present in the Test Run will be replaced with the new information given in the REST API request.


Expand
titlePOST /rest/raven/1.0/import/execution
Panel
borderColor#ccc
borderStylesolid

Import the execution results present in query variable "executionResults".

Request

Example 1: new Test Execution
Code Block
titleExample Input
{
    "info" : {
        "project" : "DEMO",
        "summary" : "Execution of automated tests for release v1.3",
		"description" : "This execution is automatically created when importing execution results from an external source",
		"version" : "v1.3",
		"user" : "admin",
		"revision" : "1.0.42134",
		"startDate" : "2014-08-30T11:47:35+01:00",
		"finishDate" : "2014-08-30T11:53:00+01:00",
		"testPlanKey" : "DEMO-100",
		"testEnvironments": ["iOS", "Android"]
    },
    "tests" : [
        {
            "testKey" : "DEMO-6",
            "start" : "2014-08-30T11:47:35+01:00",
            "finish" : "2014-08-30T11:50:56+01:00",
            "comment" : "Successful execution",
            "status" : "PASS"
        },
        {
            "testKey" : "DEMO-7",
            "start" : "2014-08-30T11:51:00+01:00",
            "finish" : "2014-08-30T11:52:30+01:00",
            "comment" : "Execution failed. Example #5 FAIL.",
            "status" : "FAIL",
			"evidences" : [
            	{
                	"data": "iVBORw0KGgoAAAANSUhEUgAABkIAAAO9CAYAAADezXv6AAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAAEn(...base64 file enconding)",
					"filename": "image21.jpg",
                    "contentType": "image/jpeg"
                }
			],
            "examples" : [
                "PASS",
                "PASS",
                "PASS",
                "PASS",
                "FAIL"
            ],
			"steps": [
                {
                    "status": "PASS",
                    "comment": "Coment on Test Step Result 1",
                    "evidences" : [
                        {
                            "data": "iVBORw0KGgoAAAANSUhEUgAABkIAAAO9CAYAAADezXv6AAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAAEn(...base64 file enconding)",
							"filename": "image22.jpg",
                            "contentType": "image/jpeg"
                        }
					],
					"actualResult": "Actual result on Test Step 1"
				}
			],
		    "defects" : [
					"DEMO-10",
					"DEMO-11"
		   ]
        }
    ]
}


Example 2: update existing Test Execution
Code Block
titleExample Input
{
    "testExecutionKey": "DEMO-1206",
    "info" : {
        "summary" : "Execution of automated tests for release v1.3",
		"description" : "This execution is automatically created when importing execution results from an external source",
		"version" : "v1.3",
		"user" : "admin",
		"revision" : "1.0.42134",
		"startDate" : "2014-08-30T11:47:35+01:00",
		"finishDate" : "2014-08-30T11:53:00+01:00",
		"testPlanKey" : "DEMO-100",
		"testEnvironments": ["iOS", "Android"]
    },
    "tests" : [
        {
            "testKey" : "DEMO-6",
            "start" : "2014-08-30T11:47:35+01:00+01:00",
		"testPlanKey" : "DEMO-100",
		"testEnvironments": ["iOS", "Android"]
    },
    "tests" : [
",
            "finish" : "2014-08-30T11:50:56+01:00",
            "comment" : "Successful execution",
            "status" : "PASS"
        }
     ]
}
Example 3: create new Test
Code Block
languagejava
titleExample Input
{
    "testExecutionKey" : "DEMO-1206",
    "tests" : [
        {
            "status": "FAIL",
            "steps": [
                {
                    "status": "PASS"
                },
                {
                    "status": "FAIL"
                }
            ],
            "testInfo": {
                "summary": "Create new test",
                "testType": "Manual",
                "projectKey": "DEMO",
                "steps": [
                    {
                        "action": "Step action 1",
                        "data": "Data 1",
                        "result": "Step result 1"
                    },
                    {
                        "action": "Step action 2",
                        "data": "Data 2",
                        "result": "Step result 2"
                    }
                ]
            }
        }
    ]
}
Example 4: update existing Test
Code Block
languagejava
titleExample Input
{
    "testExecutionKey" : "DEMO-1206",
    "tests" : [
        {
            "status": "FAIL",
            "steps": [
                {
                    "status": "PASS"
                },
                {
                    "status": "FAIL"
                }
            ],
            "testKey": "DEMO-1207",
            "testInfo": {
                "summary": "Update existing test",
                "testType": "Manual",
                "projectKey": "DEMO",
                "steps": [
                    {
                        "action": "Step action 1",
                        "data": "Data 1",
                        "result": "Step result 1"
                    },
                    {
                        "testKeyaction" : "DEMO-6Step action 2",
                        "startdata" : "2014-08-30T11:47:35+01:00Data 2",
                        "finishresult" : "2014-08-30T11:50:56+01:00",Step result 2"
            "comment" : "Successful execution",
        }
              "status" : "PASS"]
            }
        }
    ]
}
Tip
titleExample Request
curl -H "Content-Type: application/json" -X POST -u admin:admin --data @data.json http://yourserver/rest/raven/1.0/import/execution

Responses

200 OK : application/json : Successful. The results where successfully imported to Jira.

Code Block
titleExample Output
{
    "testExecIssue": {
           "id": "10000",
           "key": "DEMO-123",
           "self": "http://www.example.com/jira/rest/api/2/issue/10000"
     }
}


400 BAD_REQUEST : application/json : No execution results where provided.

401 UNAUTHORIZED : application/json : The Xray license is not valid.

500  INTERNAL SERVER ERROR : application/json : An internal error occurred when importing execution results.

...

Xray provides another endpoint if you want to create or update a Test Executions and have control over all Test Execution fields. It allows you to send two JSON files, the normal Xray JSON result and a JSON similar to the one Jira uses to create/update issues. For more information about that second format, check the Jira documentation here. Note that in this endpoint the info property in the Xray Json result part will be ignored.

Updating an existing Test Run using Xray format REST API will reset all dataset related fields. This means that all current iteration data and dataset present in the Test Run will be replaced with the new information given in the REST API request.


Expand
titlePOST /rest/raven/1.0/import/execution/multipart
Panel
borderColor#ccc
borderStylesolid

Import the execution results present in query variable "executionResults".

Request

Example 1: new Test Execution
Code Block
titleResult Json
{
    "tests" : [
        {
            "testKey" : "DEMO-6",
            "start" : "2014-08-30T11:47:35+01:00",
            "finish" : "2014-08-30T11:50:56+01:00",
            "comment" : "Successful execution",
            "status" : "PASS"
        },
        {
            "testKey" : "DEMO-7",
            "start" : "2014-08-30T11:51:00+01:00",
            "finish" : "2014-08-30T11:52:30+01:00",
            "comment" : "Execution failed. Example #5 FAIL.",
            "status" : "FAIL",
			"evidences" : [
            	{
                	"data": "iVBORw0KGgoAAAANSUhEUgAABkIAAAO9CAYAAADezXv6AAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAAEn(...base64 file enconding)",
					"filename": "image21.jpg",
                    "contentType": "image/jpeg"
                }
			],
            "examples" : [
                "PASS",
                "PASS",
                "PASS",
                "PASS",
                "FAIL"
            ],
			"steps": [
                {
                    "status": "PASS",
                    "comment": "Comment on Test Step Result 1",
                    "evidences" : [
                        {
                            "data": "iVBORw0KGgoAAAANSUhEUgAABkIAAAO9CAYAAADezXv6AAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAAEn(...base64 file enconding)",
							"filename": "image22.jpg",
                            "contentType": "image/jpeg"
                        }
					],
					"actualResult": "Actual Result on Test Step 1"
				}
			],
		    "defects" : [
					"DEMO-10",
					"DEMO-11"
		   ]
        }
    ]
}
Code Block
titleInfo JSON (Test Execution)
{
	"fields": {
		"project": {
			"id": "10402"
		},
		"summary": "Brand new Test execution",
		"issuetype": {
			"id": "10007"
		},
		"components" : [
			{
			"name":"Interface"
			},
			{
			"name":"Core"
			}
		],
		"customfield_10032" : [
			"TES-38"
		]
	}
}
Example 2: update existing Test Execution
Code Block
titleResult Json
{
    "testExecutionKey": "DEMO-1206",
    "tests" : [
        {
            "testKey" : "DEMO-6",
            "start" : "2014-08-30T11:47:35+01:00",
            "finish" : "2014-08-30T11:50:56+01:00",
            "comment" : "Successful execution",
            "status" : "PASS"
        }
     ]
}
Code Block
titleInfo JSON (Test Execution)
{
	"fields": {
		"customfield_10032" : [
			"a_label"
		],
		"description": "update the issue description"
	}
}
Tip
titleExample Request

curl -u admin:admin -F info=@issueFields.json -F result=@results.json http://yourserver/rest/raven/1.0/import/execution/multipart

Responses

200 OK : application/json : Successful. The results where successfully imported to Jira.

Code Block
titleExample Output
{
    "testExecIssue": {
           "id": "10000",
           "key": "DEMO-123",
           "self": "http://www.example.com/jira/rest/api/2/issue/10000"
     }
}


400 BAD_REQUEST : application/json : No execution results where provided.

401 UNAUTHORIZED : application/json : The Xray license is not valid.

500  INTERNAL SERVER ERROR : application/json : An internal error occurred when importing execution results.

...

How results are mapped to Test entities

Whenever importing results from some frameworks (i.e. JUnit, TestNG, NUnit, Robot framework), Xray can identify the automated test from the report/results file, based on an hardcoded criteria (such as the class name plus the class method corresponding to the automated test).

For example, in JUnit the classname and name properties of a testcase are concatenated. If no classname is provided then it is replaced by the the testsuite name and the result will be the concatenation of the testsuite name and testcase name

...

. This value becomes the Definition of the Generic Test to where the results are mapped.

Depending on the test automation framework, it's possible to specify the Test issue key to which report the results in the test's code.

Independently of whether the test is identified implicitly (based on some attributes present in the test result file) or explicitly (based on the provided Test key), related Test Runs are always reported against the correct Test issue. As a consequence, if you report results multiple times there won't be duplicated Test entities.

...