Versions Compared

Old Version 1

changes.mady.by.user Nuno Santos

Saved on

compared with

New Version Current

changes.mady.by.user Nuno Santos

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Xray JSON format

/rest/raven/1.0/import/execution

Xray JSON format multipart/rest/raven/1.0/import/execution/multipart
Cucumber JSON output format

/rest/raven/1.0/import/execution/cucumber

Cucumber JSON output format multipart

/rest/raven/1.0/import/execution/cucumber/multipart

Behave JSON output format

/rest/raven/1.0/import/execution/behave

Behave JSON output format multipart

/rest/raven/1.0/import/execution/behave/multipart

JUnit XML output format/rest/raven/1.0/import/execution/junit
JUnit XML output format multipart/rest/raven/1.0/import/execution/junit/multipart
TestNG XML output format/rest/raven/1.0/import/execution/testng
TestNG XML output format multipart/rest/raven/1.0/import/execution/testng/multipart
NUnit XML output format/rest/raven/1.0/import/execution/nunit
NUnit XML output format multipart/rest/raven/1.0/import/execution/nunit/multipart
XUnit xUnit XML output format/rest/raven/1.0/import/execution/xunit
XUnit xUnit XML output format multipart/rest/raven/1.0/import/execution/xunit/multipart
Robot Framework XML output format/rest/raven/1.0/import/execution/robot
Robot Framework XML output format multipart/rest/raven/1.0/import/execution/robot/multipart
Compressed .zip file (e.g., Calabash execution results)

/rest/raven/1.0/import/execution/bundle

...

When importing execution results using Xray JSON result format in a Continuous Integration environment, you can specify which Test Execution issue to import the results on using the "testExecutionKey" property. Alternatively, you can create a new Test Execution for the execution results and specify the Test Execution issue fields in the "info" object.

Updating an existing Test Run using Xray format REST API will reset all dataset related fields. This means that all current iteration data and dataset present in the Test Run will be replaced with the new information given in the REST API request.


Expand
titlePOST /rest/raven/1.0/import/execution
Panel
borderColor#ccc
borderStylesolid

Import the execution results present in query variable "executionResults".

Request

Example 1: new Test Execution
Code Block
titleExample Input
{
    "info" : {
        "project" : "DEMO",
        "summary" : "Execution of automated tests for release v1.3",
		"description" : "This execution is automatically created when importing execution results from an external source",
		"version" : "v1.3",
		"user" : "admin",
		"revision" : "1.0.42134",
		"startDate" : "2014-08-30T11:47:35+01:00",
		"finishDate" : "2014-08-30T11:53:00+01:00",
		"testPlanKey" : "DEMO-100",
		"testEnvironments": ["iOS", "Android"]
    },
    "tests" : [
        {
            "testKey" : "DEMO-6",
            "start" : "2014-08-30T11:47:35+01:00",
            "finish" : "2014-08-30T11:50:56+01:00",
            "comment" : "Successful execution",
            "status" : "PASS"
        },
        {
            "testKey" : "DEMO-7",
            "start" : "2014-08-30T11:51:00+01:00",
            "finish" : "2014-08-30T11:52:30+01:00",
            "comment" : "Execution failed. Example #5 FAIL.",
            "status" : "FAIL",
			"evidences" : [
            	{
                	"data": "iVBORw0KGgoAAAANSUhEUgAABkIAAAO9CAYAAADezXv6AAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAAEn(...base64 file enconding)",
					"filename": "image21.jpg",
                    "contentType": "image/jpeg"
                }
			],
            "examples" : [
                "PASS",
                "PASS",
                "PASS",
                "PASS",
                "FAIL"
            ],
			"steps": [
                {
                    "status": "PASS",
                    "comment": "Coment on Test Step Result 1",
                    "evidences" : [
                        {
                            "data": "iVBORw0KGgoAAAANSUhEUgAABkIAAAO9CAYAAADezXv6AAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAAEn(...base64 file enconding)",
							"filename": "image22.jpg",
                            "contentType": "image/jpeg"
                        }
					],
					"actualResult": "Actual result on Test Step 1"
				}
			],
		    "defects" : [
					"DEMO-10",
					"DEMO-11"
		   ]
        }
    ]
}


Example 2: update existing Test Execution
Code Block
titleExample Input
{
    "testExecutionKey": "DEMO-1206",
    "info" : {
        "summary" : "Execution of automated tests for release v1.3",
		"description" : "This execution is automatically created when importing execution results from an external source",
		"version" : "v1.3",
		"user" : "admin",
		"revision" : "1.0.42134",
		"startDate" : "2014-08-30T11:47:35+01:00",
		"finishDate" : "2014-08-30T11:53:00+01:00",
		"testPlanKey" : "DEMO-100",
		"testEnvironments": ["iOS", "Android"]
    },
    "tests" : [
        {
            "testKey" : "DEMO-6",
            "start" : "2014-08-30T11:47:35+01:00",
            "finish" : "2014-08-30T11:50:56+01:00",
            "comment" : " : "DEMO-100",
		"testEnvironments": ["iOS", "Android"]
    },
    "tests" : [
Successful execution",
            "status" : "PASS"
        }
     ]
}
Example 3: create new Test
Code Block
languagejava
titleExample Input
{
    "testExecutionKey" : "DEMO-1206",
    "tests" : [
        {
            "status": "FAIL",
            "steps": [
                {
                    "status": "PASS"
                },
                {
                    "status": "FAIL"
                }
            ],
            "testInfo": {
                "summary": "Create new test",
                "testType": "Manual",
                "projectKey": "DEMO",
                "steps": [
                    {
                        "action": "Step action 1",
                        "data": "Data 1",
                        "result": "Step result 1"
                    },
                    {
                        "action": "Step action 2",
                        "data": "Data 2",
                        "result": "Step result 2"
                    }
                ]
            }
        }
    ]
}
Example 4: update existing Test
Code Block
languagejava
titleExample Input
{
    "testExecutionKey" : "DEMO-1206",
    "tests" : [
        {
            "status": "FAIL",
            "steps": [
                {
                    "status": "PASS"
                },
                {
                    "status": "FAIL"
                }
            ],
            "testKey": "DEMO-1207",
            "testInfo": {
                "summary": "Update existing test",
                "testType": "Manual",
                "projectKey": "DEMO",
                "steps": [
                    {
                        "action": "Step action 1",
                        "data": "Data 1",
                        "result": "Step result 1"
                    },
                    {
              "testKey"           "action": "DEMO-6Step action 2",
            "start" : "2014-08-30T11:47:35+01:00",
            "data": "Data 2",
                        "finishresult" : "2014-08-30T11:50:56+01:00",Step result 2"
            "comment" : "Successful execution",
        }
              "status" : "PASS"]
            }
        }
    ]
}
Tip
titleExample Request
curl -H "Content-Type: application/json" -X POST -u admin:admin --data @data.json http://yourserver/rest/raven/1.0/import/execution

Responses

200 OK : application/json : Successful. The results where successfully imported to Jira.

Code Block
titleExample Output
{
    "testExecIssue": {
           "id": "10000",
           "key": "DEMO-123",
           "self": "http://www.example.com/jira/rest/api/2/issue/10000"
     }
}


400 BAD_REQUEST : application/json : No execution results where provided.

401 UNAUTHORIZED : application/json : The Xray license is not valid.

500  INTERNAL SERVER ERROR : application/json : An internal error occurred when importing execution results.

...

Xray provides another endpoint if you want to create or update a Test Executions and have control over all Test Execution fields. It allows you to send two JSON files, the normal Xray JSON result and a JSON similar to the one Jira uses to create/update issues. For more information about that second format, check the Jira documentation here. Note that in this endpoint the info property in the Xray Json result part will be ignored.

Updating an existing Test Run using Xray format REST API will reset all dataset related fields. This means that all current iteration data and dataset present in the Test Run will be replaced with the new information given in the REST API request.


Expand
titlePOST /rest/raven/1.0/import/execution/multipart
Panel
borderColor#ccc
borderStylesolid

Import the execution results present in query variable "executionResults".

Request

Example 1: new Test Execution
Code Block
titleResult Json
{
    "tests" : [
        {
            "testKey" : "DEMO-6",
            "start" : "2014-08-30T11:47:35+01:00",
            "finish" : "2014-08-30T11:50:56+01:00",
            "comment" : "Successful execution",
            "status" : "PASS"
        },
        {
            "testKey" : "DEMO-7",
            "start" : "2014-08-30T11:51:00+01:00",
            "finish" : "2014-08-30T11:52:30+01:00",
            "comment" : "Execution failed. Example #5 FAIL.",
            "status" : "FAIL",
			"evidences" : [
            	{
                	"data": "iVBORw0KGgoAAAANSUhEUgAABkIAAAO9CAYAAADezXv6AAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAAEn(...base64 file enconding)",
					"filename": "image21.jpg",
                    "contentType": "image/jpeg"
                }
			],
            "examples" : [
                "PASS",
                "PASS",
                "PASS",
                "PASS",
                "FAIL"
            ],
			"steps": [
                {
                    "status": "PASS",
                    "comment": "Comment on Test Step Result 1",
                    "evidences" : [
                        {
                            "data": "iVBORw0KGgoAAAANSUhEUgAABkIAAAO9CAYAAADezXv6AAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAAEn(...base64 file enconding)",
							"filename": "image22.jpg",
                            "contentType": "image/jpeg"
                        }
					],
					"actualResult": "Actual Result on Test Step 1"
				}
			],
		    "defects" : [
					"DEMO-10",
					"DEMO-11"
		   ]
        }
    ]
}
Code Block
titleInfo JSON (Test Execution)
{
	"fields": {
		"project": {
			"id": "10402"
		},
		"summary": "Brand new Test execution",
		"issuetype": {
			"id": "10007"
		},
		"components" : [
			{
			"name":"Interface"
			},
			{
			"name":"Core"
			}
		],
		"customfield_10032" : [
			"TES-38"
		]
	}
}
Example 2: update existing Test Execution
Code Block
titleResult Json
{
    "testExecutionKey": "DEMO-1206",
    "tests" : [
        {
            "testKey" : "DEMO-6",
            "start" : "2014-08-30T11:47:35+01:00",
            "finish" : "2014-08-30T11:50:56+01:00",
            "comment" : "Successful execution",
            "status" : "PASS"
        }
     ]
}
Code Block
titleInfo JSON (Test Execution)
{
	"fields": {
		"customfield_10032" : [
			"a_label"
		],
		"description": "update the issue description"
	}
}
Tip
titleExample Request

curl -u admin:admin -F info=@issueFields.json -F result=@results.json http://yourserver/rest/raven/1.0/import/execution/multipart

Responses

200 OK : application/json : Successful. The results where successfully imported to Jira.

Code Block
titleExample Output
{
    "testExecIssue": {
           "id": "10000",
           "key": "DEMO-123",
           "self": "http://www.example.com/jira/rest/api/2/issue/10000"
     }
}


400 BAD_REQUEST : application/json : No execution results where provided.

401 UNAUTHORIZED : application/json : The Xray license is not valid.

500  INTERNAL SERVER ERROR : application/json : An internal error occurred when importing execution results.

...

Whenever importing results from some frameworks (i.e. JUnit, TestNG, NUnit, Robot framework), Xray can identify the automated test from the report/results file, based on an hardcoded criteria (such as the class name plus the class method corresponding to the automated test). However, depending

For example, in JUnit the classname and name properties of a testcase are concatenated. If no classname is provided then it is replaced by the the testsuite name and the result will be the concatenation of the testsuite name and testcase name. This value becomes the Definition of the Generic Test to where the results are mapped.

Depending on the test automation framework, it's possible to specify the Test issue key to which report the results in the test's code.

...