Importing Execution Results
Execution results can be imported to Jira through JSON/XML representation formats specified in Import Execution Results.Â
For each import file format, Xray provides a specific REST endpoint:
Xray JSON format | /api/v1/import/execution |
Cucumber JSON output format | /api/v1/import/execution/cucumber |
JUnit XML output format | /api/v1/import/execution/junit |
NUnit XML output format | /api/v1/import/execution/nunit |
TestNG XML output format | /api/v1/import/execution/testng |
Xray JSON results
When importing execution results using Xray JSON result format in a Continuous Integration environment, you can specify which Test Execution issue to import the results on using the testExecutionKey property. Alternatively, you can create a new Test Execution for the execution results and specify the Test Execution issue fields in the info object.
Import the execution results present in query variable "executionResults".
Request
Example 1: new Test Execution
{ "info" : { "summary" : "Execution of automated tests for release v1.3", "description" : "This execution is automatically created when importing execution results from an external source", "version" : "v1.3", "user" : "admin", "revision" : "1.0.42134", "startDate" : "2014-08-30T11:47:35+01:00", "finishDate" : "2014-08-30T11:53:00+01:00", "testPlanKey" : "DEMO-100", "testEnvironments": ["iOS", "Android"] }, "tests" : [ { "testKey" : "DEMO-6", "start" : "2014-08-30T11:47:35+01:00", "finish" : "2014-08-30T11:50:56+01:00", "comment" : "Successful execution", "status" : "PASSED" }, { "testKey" : "DEMO-7", "start" : "2014-08-30T11:51:00+01:00", "finish" : "2014-08-30T11:52:30+01:00", "comment" : "Execution failed. Example #5 FAIL.", "status" : "FAILED", "evidences" : [ { "data": "iVBORw0KGgoAAAANSUhEUgAABkIAAAO9CAYAAADezXv6AAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAAEn(...base64 file enconding)", "filename": "image21.jpg", "contentType": "image/jpeg" } ], "examples" : [ "PASSED", "PASSED", "PASSED", "PASSED", "FAILED" ], "steps": [ { "status": "PASSED", "comment": "Coment on Test Step Result 1", "evidences" : [ { "data": "iVBORw0KGgoAAAANSUhEUgAABkIAAAO9CAYAAADezXv6AAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAAEn(...base64 file enconding)", "filename": "image22.jpg", "contentType": "image/jpeg" } ] } ], "defects" : [ "DEMO-10", "DEMO-11" ] } ] }
Example 2: update existing Test Execution
{ "testExecutionKey": "DEMO-1206", "info" : { "summary" : "Execution of automated tests for release v1.3", "description" : "This execution is automatically created when importing execution results from an external source", "version" : "v1.3", "user" : "admin", "revision" : "1.0.42134", "startDate" : "2014-08-30T11:47:35+01:00", "finishDate" : "2014-08-30T11:53:00+01:00", "testPlanKey" : "DEMO-100", "testEnvironments": ["iOS", "Android"] }, "tests" : [ { "testKey" : "DEMO-6", "start" : "2014-08-30T11:47:35+01:00", "finish" : "2014-08-30T11:50:56+01:00", "comment" : "Successful execution", "status" : "PASSED" } ] }
Example Request
curl -H "Content-Type: application/json" -X POST -H "Authorization: Bearer $token" --data @"data.json" https://xray.cloud.xpand-it.com/api/v1/import/execution
Responses
200 OK : application/json : Successful. The results where successfully imported to Jira.
{ "testExecIssue": { "id": "10000", "key": "DEMO-123", "self": "http://www.example.com/jira/rest/api/2/issue/10000" } }
400 BAD_REQUEST : application/json : No execution results where provided.
401 UNAUTHORIZED : application/json : The Xray license is not valid.
500 INTERNAL SERVER ERROR : application/json : An internal error occurred when importing execution results.
Cucumber JSON results
After executing Cucumber features, you must import the outputted JSON execution results to Jira using the following endpoint:
Â
Import the execution results created with the Cucumber JSON output formatter. For more information please check the Cucumber reports documentation (example here).
Request
Example
[ { "keyword": "Feature", "name": "Arithmetic Operations", "line": 3, "description": "", "tags": [ { "name": "@DEMO-48", "line": 1 }, { "name": "@REQ_DEMO-45", "line": 2 } ], "id": "arithmetic-operations", "uri": "features/1_DEMO-45.feature", "elements": [ { "comments": [ { "value": "#In order to avoid silly mistakes", "line": 4 }, { "value": "#As a math idiot ", "line": 5 }, { "value": "#I want to be told the result of basic arithmetic operations between two numbers", "line": 6 } ], "keyword": "Scenario Outline", "name": "Add two Numbers", "line": 18, "description": "", "tags": [ { "name": "@TEST_DEMO-47", "line": 9 } ], "id": "arithmetic-operations;add-two-numbers;;2", "type": "scenario", "steps": [ { "embeddings": [ { "mime_type": "text/plain", "data": "{data base64}" },{ "mime_type": "text/plain", "data": "{data base64}" } "keyword": "Given ", "name": "I have entered 20 into the calculator", "line": 11, "match": { "arguments": [ { "offset": 15, "val": "20" } ], "location": "features/step_definitions/calculator_steps.rb:14" }, "result": { "status": "passed", "duration": 487000 } }, { "keyword": "And ", "name": "I have entered 30 into the calculator", "line": 12, "match": { "arguments": [ { "offset": 15, "val": "30" } ], "location": "features/step_definitions/calculator_steps.rb:14" }, "result": { "status": "passed", "duration": 340000 } }, { "keyword": "When ", "name": "I press add", "line": 13, "match": { "arguments": [ { "offset": 8, "val": "add" } ], "location": "features/step_definitions/calculator_steps.rb:18" }, "result": { "status": "passed", "duration": 327000 } }, { "keyword": "Then ", "name": "the result should be 50 on the screen", "line": 14, "match": { "arguments": [ { "offset": 21, "val": "50" } ], "location": "features/step_definitions/calculator_steps.rb:22" }, "result": { "status": "passed", "duration": 11723000 } } ] }, { "comments": [ { "value": "#In order to avoid silly mistakes", "line": 4 }, { "value": "#As a math idiot ", "line": 5 }, { "value": "#I want to be told the result of basic arithmetic operations between two numbers", "line": 6 } ], "keyword": "Scenario Outline", "name": "Add two Numbers", "line": 19, "description": "", "tags": [ { "name": "@TEST_DEMO-47", "line": 9 } ], "id": "arithmetic-operations;add-two-numbers;;3", "type": "scenario", "steps": [ { "keyword": "Given ", "name": "I have entered 2 into the calculator", "line": 11, "match": { "arguments": [ { "offset": 15, "val": "2" } ], "location": "features/step_definitions/calculator_steps.rb:14" }, "result": { "status": "passed", "duration": 992000 } }, { "keyword": "And ", "name": "I have entered 5 into the calculator", "line": 12, "match": { "arguments": [ { "offset": 15, "val": "5" } ], "location": "features/step_definitions/calculator_steps.rb:14" }, "result": { "status": "passed", "duration": 775000 } }, { "keyword": "When ", "name": "I press add", "line": 13, "match": { "arguments": [ { "offset": 8, "val": "add" } ], "location": "features/step_definitions/calculator_steps.rb:18" }, "result": { "status": "passed", "duration": 322000 } }, { "keyword": "Then ", "name": "the result should be 7 on the screen", "line": 14, "match": { "arguments": [ { "offset": 21, "val": "7" } ], "location": "features/step_definitions/calculator_steps.rb:22" }, "result": { "status": "passed", "duration": 423000 } } ] }, { "comments": [ { "value": "#In order to avoid silly mistakes", "line": 4 }, { "value": "#As a math idiot ", "line": 5 }, { "value": "#I want to be told the result of basic arithmetic operations between two numbers", "line": 6 } ], "keyword": "Scenario Outline", "name": "Add two Numbers", "line": 20, "description": "", "tags": [ { "name": "@TEST_DEMO-47", "line": 9 } ], "id": "arithmetic-operations;add-two-numbers;;4", "type": "scenario", "steps": [ { "keyword": "Given ", "name": "I have entered 0 into the calculator", "line": 11, "match": { "arguments": [ { "offset": 15, "val": "0" } ], "location": "features/step_definitions/calculator_steps.rb:14" }, "result": { "status": "passed", "duration": 384000 } }, { "keyword": "And ", "name": "I have entered 40 into the calculator", "line": 12, "match": { "arguments": [ { "offset": 15, "val": "40" } ], "location": "features/step_definitions/calculator_steps.rb:14" }, "result": { "status": "passed", "duration": 313000 } }, { "keyword": "When ", "name": "I press add", "line": 13, "match": { "arguments": [ { "offset": 8, "val": "add" } ], "location": "features/step_definitions/calculator_steps.rb:18" }, "result": { "status": "passed", "duration": 280000 } }, { "keyword": "Then ", "name": "the result should be 40 on the screen", "line": 14, "match": { "arguments": [ { "offset": 21, "val": "40" } ], "location": "features/step_definitions/calculator_steps.rb:22" }, "result": { "status": "passed", "duration": 350000 } } ] }, { "keyword": "Scenario Outline", "name": "Divide Two Numbers", "line": 32, "description": "", "tags": [ { "name": "@TEST_DEMO-46", "line": 23 } ], "id": "arithmetic-operations;divide-two-numbers;;2", "type": "scenario", "steps": [ { "keyword": "Given ", "name": "I have entered 8 into the calculator", "line": 25, "match": { "arguments": [ { "offset": 15, "val": "8" } ], "location": "features/step_definitions/calculator_steps.rb:14" }, "result": { "status": "passed", "duration": 344000 } }, { "keyword": "And ", "name": "I have entered 4 into the calculator", "line": 26, "match": { "arguments": [ { "offset": 15, "val": "4" } ], "location": "features/step_definitions/calculator_steps.rb:14" }, "result": { "status": "passed", "duration": 292000 } }, { "keyword": "When ", "name": "I press divide", "line": 27, "match": { "arguments": [ { "offset": 8, "val": "divide" } ], "location": "features/step_definitions/calculator_steps.rb:18" }, "result": { "status": "passed", "duration": 291000 } }, { "keyword": "Then ", "name": "the result should be 2 on the screen", "line": 28, "match": { "arguments": [ { "offset": 21, "val": "2" } ], "location": "features/step_definitions/calculator_steps.rb:22" }, "result": { "status": "passed", "duration": 320000 } } ] }, { "keyword": "Scenario Outline", "name": "Divide Two Numbers", "line": 33, "description": "", "tags": [ { "name": "@TEST_DEMO-46", "line": 23 } ], "id": "arithmetic-operations;divide-two-numbers;;3", "type": "scenario", "steps": [ { "keyword": "Given ", "name": "I have entered 12 into the calculator", "line": 25, "match": { "arguments": [ { "offset": 15, "val": "12" } ], "location": "features/step_definitions/calculator_steps.rb:14" }, "result": { "status": "passed", "duration": 1102000 } }, { "keyword": "And ", "name": "I have entered 3 into the calculator", "line": 26, "match": { "arguments": [ { "offset": 15, "val": "3" } ], "location": "features/step_definitions/calculator_steps.rb:14" }, "result": { "status": "passed", "duration": 891000 } }, { "keyword": "When ", "name": "I press divide", "line": 27, "match": { "arguments": [ { "offset": 8, "val": "divide" } ], "location": "features/step_definitions/calculator_steps.rb:18" }, "result": { "status": "passed", "duration": 291000 } }, { "keyword": "Then ", "name": "the result should be 4 on the screen", "line": 28, "match": { "arguments": [ { "offset": 21, "val": "4" } ], "location": "features/step_definitions/calculator_steps.rb:22" }, "result": { "status": "passed", "duration": 339000 } } ] }, { "keyword": "Scenario Outline", "name": "Divide Two Numbers", "line": 34, "description": "", "tags": [ { "name": "@TEST_DEMO-46", "line": 23 } ], "id": "arithmetic-operations;divide-two-numbers;;4", "type": "scenario", "steps": [ { "keyword": "Given ", "name": "I have entered 3 into the calculator", "line": 25, "match": { "arguments": [ { "offset": 15, "val": "3" } ], "location": "features/step_definitions/calculator_steps.rb:14" }, "result": { "status": "passed", "duration": 304000 } }, { "keyword": "And ", "name": "I have entered 1 into the calculator", "line": 26, "match": { "arguments": [ { "offset": 15, "val": "1" } ], "location": "features/step_definitions/calculator_steps.rb:14" }, "result": { "status": "passed", "duration": 309000 } }, { "keyword": "When ", "name": "I press divide", "line": 27, "match": { "arguments": [ { "offset": 8, "val": "divide" } ], "location": "features/step_definitions/calculator_steps.rb:18" }, "result": { "status": "passed", "duration": 257000 } }, { "keyword": "Then ", "name": "the result should be 5 on the screen", "line": 28, "match": { "arguments": [ { "offset": 21, "val": "5" } ], "location": "features/step_definitions/calculator_steps.rb:22" }, "result": { "status": "passed", "duration": 840000 } } ] } ] } ]
Example Request
curl -H "Content-Type: application/json" -X POST -H "Authorization: Bearer $token" --data @"data.json" https://xray.cloud.xpand-it.com/api/v1/import/execution/cucumber
Responses
200 OK : application/json : Successful. The results where successfully imported to Jira.
{ "testExecIssue": { "id": "10000", "key": "DEMO-123", "self": "http://www.example.com/jira/rest/api/2/issue/10000" } }
400 BAD_REQUEST : application/json : No execution results where provided.
401 UNAUTHORIZED : application/json : The Xray license is not valid.
500 INTERNAL SERVER ERROR : application/json : An internal error occurred when importing execution results.
JUnit XML results
After executing JUnit tests, you must import the outputted XML execution results to Jira using the following endpoint:
Import the execution results created with the JUnit XML output formatter. For more information, please check the documentation about JUnit integration.
Request
PATH PARAMETERS
parameter | type | description |
---|---|---|
projectKey | String | - key of the project where the test execution (if the testExecKey parameter wasn't provided) and the tests (if they aren't created yet) are going to be created. |
testExecKey | String | - key of the Test Execution. |
testPlanKey | String | - key of the Test Plan; if you specify the Test Plan, the Tests will be added automatically to the Test Plan if they're not part of it. |
testEnvironments | String | - a string containing a list of test environments separated by ";" |
revision | String | - source code and documentation version used in the test execution. |
fixVersion | String | - the Fix Version associated with the test execution (it supports only one value). |
Example
<?xml version="1.0" encoding="UTF-8" ?> <testsuite tests="15" failures="0" name="ut.com.xpandit.raven.service.impl.IssueDataSetTest" time="0.163" errors="0" skipped="0"> <properties> ... </properties> <testcase classname="ut.com.xpandit.raven.service.impl.IssueDataSetTest" name="testApplyOptions_withValidIssueAndValidLimitOverflowOption_returnsExpectedSubset" time="0.114"/> <testcase classname="ut.com.xpandit.raven.service.impl.IssueDataSetTest" name="testApplyOptions_withNullOptionsAndValidIssue_throwsIllegalArgumentException" time="0.002"/> <testcase classname="ut.com.xpandit.raven.service.impl.IssueDataSetTest" name="testApplyOptions_withValidIssueAndValidEmptyOptions_returnsAllIssues" time="0.002"/> <testcase classname="ut.com.xpandit.raven.service.impl.IssueDataSetTest" name="testApplyOptions_withValidIssueAndValidGlobalSearchOptions_returnsExpectedTests" time="0.016"/> <testcase classname="ut.com.xpandit.raven.service.impl.IssueDataSetTest" name="testApplyOptions_withValidIssueAndInvalidColumnSearchOption_returnsAllTests" time="0.007"/> <testcase classname="ut.com.xpandit.raven.service.impl.IssueDataSetTest" name="testApplyOptions_withValidIssueAndValidLimitUnderOption_returnsExpectedSubset" time="0.001"/> <testcase classname="ut.com.xpandit.raven.service.impl.IssueDataSetTest" name="testApplyOptions_withValidIssueAndValidGlobalSearchOptionThatMachesIssueKey_returnsExpectedTestWithMatchedKey" time="0.006"/> <testcase classname="ut.com.xpandit.raven.service.impl.IssueDataSetTest" name="testApplyOptions_withValidIssueAndValidSummaryColumnAscSortOption_returnsExpectedIssuesInAscOrder" time="0.006"/> <testcase classname="ut.com.xpandit.raven.service.impl.IssueDataSetTest" name="testApplyOptions_withValidIssueAndValidSummaryColumnDescSortOption_returnsExpectedIssuesInDescOrder" time="0.002"/> <testcase classname="ut.com.xpandit.raven.service.impl.IssueDataSetTest" name="testApplyOptions_withValidIssueAndValidGlobalSearchOptionThatMatchesAllElements_returnsAllTests" time="0.001"/> <testcase classname="ut.com.xpandit.raven.service.impl.IssueDataSetTest" name="testApplyOptions_withValidIssueAndValidColumnSearchOptionThatMatchesOneElement_returnsOneTest" time="0.002"/> <testcase classname="ut.com.xpandit.raven.service.impl.IssueDataSetTest" name="testApplyOptions_withValidIssueAndValidColumnSearchOptionThatMatchesNoIssue_returnsEmptyList" time="0.001"/> <testcase classname="ut.com.xpandit.raven.service.impl.IssueDataSetTest" name="testApplyOptions_withValidIssueAndValidGlobalSearchOptionThatMachesNoIssue_returnsEmptyList" time="0.001"/> <testcase classname="ut.com.xpandit.raven.service.impl.IssueDataSetTest" name="testApplyOptions_withValidIssueAndValidKeyColumnDescSortOption_returnsExpectedIssuesInDescOrder" time="0.001"/> <testcase classname="ut.com.xpandit.raven.service.impl.IssueDataSetTest" name="testApplyOptions_withValidIssueAndValidKeyColumnAscSortOption_returnsExpectedIssuesInAscOrder" time="0.001"/> </testsuite>
Example Request
curl -H "Content-Type: text/xml" -X POST -H "Authorization: Bearer $token" --data @"data.xml" https://xray.cloud.xpand-it.com/api/v1/import/execution/junit?projectKey=XTP
curl -H "Content-Type: text/xml" -X POST -H "Authorization: Bearer $token" --data @"data.xml" https://xray.cloud.xpand-it.com/api/v1/import/execution/junit?testExecKey=XNP-23
curl -H "Content-Type: text/xml" -X POST -H "Authorization: Bearer $token" --data @"data.xml" https://xray.cloud.xpand-it.com/api/v1/import/execution/junit?projectKey=XTP&testExecKey=XNP-23
curl -H "Content-Type: text/xml" -X POST -H "Authorization: Bearer $token" --data @"data.xml" https://xray.cloud.xpand-it.com/api/v1/import/execution/junit?projectKey=XTP&testPlanKey=XTP-12&revision=v2.1.0
Responses
200 OK : application/json : Successful. The results where successfully imported to Jira.
{ "testExecIssue": { "id": "10200", "key": "XNP-24", "self": "http://www.example.com/jira/rest/api/2/issue/10200" } }
400 BAD_REQUEST : application/json : Returns the error.
401 UNAUTHORIZED : application/json : The token is invalid.
500 INTERNAL SERVER ERROR : application/json : An internal error occurred when importing execution results.
NUnit XML results
After executing NUnit tests, you must import the outputted XML execution results to Jira using the following endpoint:
Import the execution results created with the NUnit XML output formatter. For more information please check the documentation about NUnit integration.
Request
PATH PARAMETERS
parameter | type | description |
---|---|---|
projectKey | String | - key of the project where the Test Execution (if the testExecKey parameter wasn't provided) and the tests (if they aren't created yet) are going to be created. |
testExecKey | String | - key of the Test Execution. |
testPlanKey | String | - key of the Test Plan; if you specify the Test Plan, the Tests will be added automatically to the Test Plan if they're not part of it. |
testEnvironments | String | - a string containing a list of test environments separated by ";" |
revision | String | - source code and documentation version used in the test execution. |
fixVersion | String | - the Fix Version associated with the test execution (it supports only one value). |
Example
<?xml version="1.0" encoding="utf-8" standalone="no"?> <test-run id="0" testcasecount="14" total="14" passed="13" failed="1" inconclusive="0" skipped="0" asserts="14" result="Failed" portable-engine-version="3.3.0.0" start-time="2016-12-26 14:36:03Z" end-time="2016-12-26 14:36:03Z" duration="0.140400"> <test-suite type="Assembly" id="1021" name="x, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" fullname="x, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" runstate="Runnable" testcasecount="14" result="Failed" site="Child" start-time="2016-12-26 14:36:03Z" end-time="2016-12-26 14:36:03Z" duration="0.110549" total="14" passed="13" failed="1" inconclusive="0" skipped="0" asserts="14"> <settings> <setting name="WorkDirectory" value="C:\Users\Sergio\x" /> </settings> <failure> <message><![CDATA[One or more child tests had errors]]></message> </failure> <test-suite type="TestFixture" id="1000" name="TestClass" fullname="TestClass" classname="TestClass" runstate="Runnable" testcasecount="2" result="Failed" site="Child" start-time="2016-12-26 14:36:03Z" end-time="2016-12-26 14:36:03Z" duration="0.084668" total="2" passed="1" failed="1" inconclusive="0" skipped="0" asserts="2"> <failure> <message><![CDATA[One or more child tests had errors]]></message> </failure> <test-suite type="ParameterizedMethod" id="1003" name="SubtractTest" fullname="TestClass.SubtractTest" classname="TestClass" runstate="Runnable" testcasecount="2" result="Failed" site="Child" start-time="2016-12-26 14:36:03Z" end-time="2016-12-26 14:36:03Z" duration="0.080887" total="2" passed="1" failed="1" inconclusive="0" skipped="0" asserts="2"> <properties> <property name="Requirement" value="DEV-771" /> </properties><failure> <message><![CDATA[One or more child tests had errors]]></message> </failure> <test-case id="1001" name="SubtractTest(1)" fullname="TestClass.SubtractTest(1)" methodname="SubtractTest" classname="TestClass" runstate="Runnable" seed="1166833138" result="Failed" start-time="2016-12-26 14:36:03Z" end-time="2016-12-26 14:36:03Z" duration="0.043525" asserts="1"> <failure> <message><![CDATA[ Expected: 10 But was: 1 ]]></message> <stack-trace><![CDATA[at TestClass.SubtractTest(Int32 x) in C:\Users\Sergio\x\TestClass.cs:line 13 ]]></stack-trace> </failure> </test-case> <test-case id="1002" name="SubtractTest(10)" fullname="TestClass.SubtractTest(10)" methodname="SubtractTest" classname="TestClass" runstate="Runnable" seed="1003146807" result="Failed" start-time="2016-12-26 14:36:03Z" end-time="2016-12-26 14:36:03Z" duration="12.000098" asserts="1" /> </test-suite> </test-suite> <test-suite type="TestSuite" id="1022" name="x" fullname="x" runstate="Runnable" testcasecount="12" result="Passed" start-time="2016-12-26 14:36:03Z" end-time="2016-12-26 14:36:03Z" duration="0.015218" total="12" passed="12" failed="0" inconclusive="0" skipped="0" asserts="12"> <test-suite type="TestFixture" id="1004" name="CalculatorTests" fullname="x.CalculatorTests" classname="x.CalculatorTests" runstate="Runnable" testcasecount="12" result="Passed" start-time="2016-12-26 14:36:03Z" end-time="2016-12-26 14:36:03Z" duration="0.014979" total="12" passed="12" failed="0" inconclusive="0" skipped="0" asserts="12"> <test-suite type="ParameterizedMethod" id="1008" name="CanAddNumbers" fullname="x.CalculatorTests.CanAddNumbers" classname="x.CalculatorTests" runstate="Runnable" testcasecount="3" result="Passed" start-time="2016-12-26 14:36:03Z" end-time="2016-12-26 14:36:03Z" duration="0.004228" total="3" passed="3" failed="0" inconclusive="0" skipped="0" asserts="3"> <properties> <property name="Requirement" value="DEV-771" /> </properties> <test-case id="1005" name="CanAddNumbers(1,1,2)" fullname="x.CalculatorTests.CanAddNumbers(1,1,2)" methodname="CanAddNumbers" classname="x.CalculatorTests" runstate="Runnable" seed="1846389584" result="Passed" start-time="2016-12-26 14:36:03Z" end-time="2016-12-26 14:36:03Z" duration="0.001194" asserts="1" /> <test-case id="1006" name="CanAddNumbers(-1,-1,-2)" fullname="x.CalculatorTests.CanAddNumbers(-1,-1,-2)" methodname="CanAddNumbers" classname="x.CalculatorTests" runstate="Runnable" seed="1113780989" result="Passed" start-time="2016-12-26 14:36:03Z" end-time="2016-12-26 14:36:03Z" duration="0.000067" asserts="1" /> <test-case id="1007" name="CanAddNumbers(100,5,105)" fullname="x.CalculatorTests.CanAddNumbers(100,5,105)" methodname="CanAddNumbers" classname="x.CalculatorTests" runstate="Runnable" seed="1585332966" result="Passed" start-time="2016-12-26 14:36:03Z" end-time="2016-12-26 14:36:03Z" duration="0.000103" asserts="1" /> </test-suite> <test-suite type="ParameterizedMethod" id="1020" name="CanDivide" fullname="x.CalculatorTests.CanDivide" classname="x.CalculatorTests" runstate="Runnable" testcasecount="3" result="Passed" start-time="2016-12-26 14:36:03Z" end-time="2016-12-26 14:36:03Z" duration="0.004041" total="3" passed="3" failed="0" inconclusive="0" skipped="0" asserts="3"> <properties> <property name="Requirement" value="DEV-771" /> </properties> <test-case id="1017" name="CanDivide(1,1,1)" fullname="x.CalculatorTests.CanDivide(1,1,1)" methodname="CanDivide" classname="x.CalculatorTests" runstate="Runnable" seed="1285501252" result="Passed" start-time="2016-12-26 14:36:03Z" end-time="2016-12-26 14:36:03Z" duration="0.000354" asserts="1" /> <test-case id="1018" name="CanDivide(-1,-1,1)" fullname="x.CalculatorTests.CanDivide(-1,-1,1)" methodname="CanDivide" classname="x.CalculatorTests" runstate="Runnable" seed="1436436719" result="Passed" start-time="2016-12-26 14:36:03Z" end-time="2016-12-26 14:36:03Z" duration="0.000073" asserts="1" /> <test-case id="1019" name="CanDivide(100,5,20)" fullname="x.CalculatorTests.CanDivide(100,5,20)" methodname="CanDivide" classname="x.CalculatorTests" runstate="Runnable" seed="213310888" result="Passed" start-time="2016-12-26 14:36:03Z" end-time="2016-12-26 14:36:03Z" duration="0.000060" asserts="1" /> </test-suite> <test-suite type="ParameterizedMethod" id="1016" name="CanMultiply" fullname="x.CalculatorTests.CanMultiply" classname="x.CalculatorTests" runstate="Runnable" testcasecount="3" result="Passed" start-time="2016-12-26 14:36:03Z" end-time="2016-12-26 14:36:03Z" duration="0.002759" total="3" passed="3" failed="0" inconclusive="0" skipped="0" asserts="3"> <test-case id="1013" name="CanMultiply(1,1,1)" fullname="x.CalculatorTests.CanMultiply(1,1,1)" methodname="CanMultiply" classname="x.CalculatorTests" runstate="Runnable" seed="1192735127" result="Passed" start-time="2016-12-26 14:36:03Z" end-time="2016-12-26 14:36:03Z" duration="0.000331" asserts="1"> <properties> <property name="label" value="multiplication" /> </properties> </test-case> <test-case id="1014" name="CanMultiply(-1,-1,1)" fullname="x.CalculatorTests.CanMultiply(-1,-1,1)" methodname="CanMultiply" classname="x.CalculatorTests" runstate="Runnable" seed="39988064" result="Passed" start-time="2016-12-26 14:36:03Z" end-time="2016-12-26 14:36:03Z" duration="0.000059" asserts="1"> <properties> <property name="label" value="multiplication" /> </properties> </test-case> <test-case id="1015" name="CanMultiply(100,5,500)" fullname="x.CalculatorTests.CanMultiply(100,5,500)" methodname="CanMultiplyAgain" classname="x.CalculatorTests" runstate="Runnable" seed="1462346243" result="Passed" start-time="2016-12-26 14:36:03Z" end-time="2016-12-26 14:36:03Z" duration="0.000052" asserts="1"> <properties> <property name="requirement" value="DEV-34" /> </properties> </test-case> </test-suite> <test-suite type="ParameterizedMethod" id="1012" name="CanSubtract" fullname="x.CalculatorTests.CanSubtract" classname="x.CalculatorTests" runstate="Runnable" testcasecount="3" result="Passed" start-time="2016-12-26 14:36:03Z" end-time="2016-12-26 14:36:03Z" duration="0.002827" total="3" passed="3" failed="0" inconclusive="0" skipped="0" asserts="3"> <properties> <property name="requirement" value="DEV-328" /> </properties> <test-case id="1009" name="CanSubtract(1,1,0)" fullname="x.CalculatorTests.CanSubtract(1,1,0)" methodname="CanSubtract" classname="x.CalculatorTests" runstate="Runnable" seed="1019357734" result="Failed" start-time="2016-12-26 14:36:03Z" end-time="2016-12-26 14:36:03Z" duration="0.000303" asserts="1"> <failure> <message><![CDATA[Error subtracting]]></message> </failure> </test-case> <test-case id="1010" name="CanSubtract(-1,-1,0)" fullname="x.CalculatorTests.CanSubtract(-1,-1,0)" methodname="CanSubtract" classname="x.CalculatorTests" runstate="Runnable" seed="1322022615" result="Failed" start-time="2016-12-26 14:36:03Z" end-time="2016-12-26 14:36:03Z" duration="0.000056" asserts="1" > <failure> <message><![CDATA[Error subtracting]]></message> </failure> </test-case> <test-case id="1011" name="CanSubtract(100,5,95)" fullname="x.CalculatorTests.CanSubtract(100,5,95)" methodname="CanSubtract" classname="x.CalculatorTests" runstate="Runnable" seed="4493553" result="Passed" start-time="2016-12-26 14:36:03Z" end-time="2016-12-26 14:36:03Z" duration="0.000053" asserts="1" /> </test-suite> </test-suite> </test-suite> </test-suite> </test-run>
Example Request
curl -H "Content-Type: text/xml" -X POST -H "Authorization: Bearer $token" --data @"data.xml" https://xray.cloud.xpand-it.com/api/v1/import/execution/nunit?projectKey=XTP
curl -H "Content-Type: text/xml" -X POST -H "Authorization: Bearer $token" --data @"data.xml" https://xray.cloud.xpand-it.com/api/v1/import/execution/nunit?testExecKey=XNP-23
curl -H "Content-Type: text/xml" -X POST -H "Authorization: Bearer $token" --data @"data.xml" https://xray.cloud.xpand-it.com/api/v1/import/execution/nunit?projectKey=XTP&testExecKey=XNP-23
curl -H "Content-Type: text/xml" -X POST -H "Authorization: Bearer $token" --data @"data.xml" https://xray.cloud.xpand-it.com/api/v1/import/execution/nunit?projectKey=XTP&testPlanKey=XTP-12&revision=v2.1.0
Responses
200 OK : application/json : Successful. The results where successfully imported to Jira.
{ "testExecIssue": { "id": "10200", "key": "XNP-24", "self": "http://www.example.com/jira/rest/api/2/issue/10200" } }
400 BAD_REQUEST : application/json : Returns the error.
401 UNAUTHORIZED : application/json : The token is invalid.
500 INTERNAL SERVER ERROR : application/json : An internal error occurred when importing execution results.
TestNG XML results
After executing TestNG tests, you must import the outputted XML execution results to Jira using the following endpoint:
Import the execution results created with the TestNG XML output formatter. For more information please check the documentation about TestNG integration.
Request
QUERY PARAMETERS
parameter | type | description |
---|---|---|
projectKey | String | - key of the project where the Test Execution (if the testExecKey parameter wasn't provided) and the tests (if they aren't created yet) are going to be created. |
testExecKey | String | - key of the Test Execution. |
testPlanKey | String | - key of the Test Plan; if you specify the Test Plan, the Tests will be added automatically to the Test Plan if they're not part of it. |
testEnvironments | String | - a string containing a list of test environments separated by ";" |
revision | String | - source code and documentation version used in the test execution. |
fixVersion | String | - the Fix Version associated with the test execution (it supports only one value). |
multipart/form-data:
"file" : a MultipartFormParam containing a XML file to import.
Example
<?xml version="1.0" encoding="UTF-8"?> <testng-results skipped="0" failed="2" ignored="0" total="8" passed="6"> <reporter-output> </reporter-output> <suite name="TestAll" duration-ms="33" started-at="2018-03-06T11:53:00Z" finished-at="2018-03-06T11:53:00Z"> <groups> </groups> <test name="calculator" duration-ms="33" started-at="2018-03-06T11:53:00Z" finished-at="2018-03-06T11:53:00Z"> <class name="com.xpand.java.CalcTest"> <test-method status="PASS" signature="setUp()[pri:0, instance:com.xpand.java.CalcTest@36d4b5c]" name="setUp" is-config="true" duration-ms="9" started-at="2018-03-06T11:53:00Z" finished-at="2018-03-06T11:53:00Z"> <reporter-output> </reporter-output> </test-method> <!-- setUp --> <test-method status="PASS" signature="CanAddNumbers()[pri:0, instance:com.xpand.java.CalcTest@36d4b5c]" name="CanAddNumbers" duration-ms="2" started-at="2018-03-06T11:53:00Z" finished-at="2018-03-06T11:53:00Z"> <reporter-output> </reporter-output> <attributes> <attribute name="test"> <![CDATA[]]> </attribute> <!-- test --> <attribute name="requirement"> <![CDATA[CALC-1235]]> </attribute> <!-- requirement --> <attribute name="labels"> <![CDATA[core]]> </attribute> <!-- labels --> </attributes> </test-method> <!-- CanAddNumbers --> <test-method status="PASS" signature="CanAddNumbersFromGivenData(int, int, int)[pri:0, instance:com.xpand.java.CalcTest@36d4b5c]" name="CanAddNumbersFromGivenData" duration-ms="0" started-at="2018-03-06T11:53:00Z" data-provider="ValidDataProvider" finished-at="2018-03-06T11:53:00Z"> <params> <param index="0"> <value> <![CDATA[1]]> </value> </param> <param index="1"> <value> <![CDATA[2]]> </value> </param> <param index="2"> <value> <![CDATA[3]]> </value> </param> </params> <reporter-output> </reporter-output> <attributes> <attribute name="test"> <![CDATA[]]> </attribute> <!-- test --> <attribute name="requirement"> <![CDATA[CALC-1235]]> </attribute> <!-- requirement --> <attribute name="labels"> <![CDATA[core]]> </attribute> <!-- labels --> </attributes> </test-method> <!-- CanAddNumbersFromGivenData --> <test-method status="FAIL" signature="CanAddNumbersFromGivenData(int, int, int)[pri:0, instance:com.xpand.java.CalcTest@36d4b5c]" name="CanAddNumbersFromGivenData" duration-ms="1" started-at="2018-03-06T11:53:00Z" data-provider="ValidDataProvider" finished-at="2018-03-06T11:53:00Z"> <params> <param index="0"> <value> <![CDATA[2]]> </value> </param> <param index="1"> <value> <![CDATA[3]]> </value> </param> <param index="2"> <value> <![CDATA[4]]> </value> </param> </params> <exception class="java.lang.AssertionError"> <message> <![CDATA[expected [4] but found [5]]]> </message> <full-stacktrace> <![CDATA[java.lang.AssertionError: expected [4] but found [5] at org.testng.Assert.fail(Assert.java:93) at org.testng.Assert.failNotEquals(Assert.java:512) at org.testng.Assert.assertEqualsImpl(Assert.java:134) at org.testng.Assert.assertEquals(Assert.java:115) at org.testng.Assert.assertEquals(Assert.java:388) at org.testng.Assert.assertEquals(Assert.java:398) at com.xpand.java.CalcTest.CanAddNumbersFromGivenData(CalcTest.java:40) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:108) at org.testng.internal.Invoker.invokeMethod(Invoker.java:661) at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:869) at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1193) at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:126) at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109) at org.testng.TestRunner.privateRun(TestRunner.java:744) at org.testng.TestRunner.run(TestRunner.java:602) at org.testng.SuiteRunner.runTest(SuiteRunner.java:380) at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:375) at org.testng.SuiteRunner.privateRun(SuiteRunner.java:340) at org.testng.SuiteRunner.run(SuiteRunner.java:289) at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52) at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86) at org.testng.TestNG.runSuitesSequentially(TestNG.java:1301) at org.testng.TestNG.runSuitesLocally(TestNG.java:1226) at org.testng.TestNG.runSuites(TestNG.java:1144) at org.testng.TestNG.run(TestNG.java:1115) at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:283) at org.apache.maven.surefire.testng.TestNGXmlTestSuite.execute(TestNGXmlTestSuite.java:75) at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:120) at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:373) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:334) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:119) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:407) ]]> </full-stacktrace> </exception> <!-- java.lang.AssertionError --> <reporter-output> </reporter-output> <attributes> <attribute name="test"> <![CDATA[]]> </attribute> <!-- test --> <attribute name="requirement"> <![CDATA[CALC-1235]]> </attribute> <!-- requirement --> <attribute name="labels"> <![CDATA[core]]> </attribute> <!-- labels --> </attributes> </test-method> <!-- CanAddNumbersFromGivenData --> <test-method status="PASS" signature="CanAddNumbersFromGivenData(int, int, int)[pri:0, instance:com.xpand.java.CalcTest@36d4b5c]" name="CanAddNumbersFromGivenData" duration-ms="0" started-at="2018-03-06T11:53:00Z" data-provider="ValidDataProvider" finished-at="2018-03-06T11:53:00Z"> <params> <param index="0"> <value> <![CDATA[-1]]> </value> </param> <param index="1"> <value> <![CDATA[1]]> </value> </param> <param index="2"> <value> <![CDATA[0]]> </value> </param> </params> <reporter-output> </reporter-output> <attributes> <attribute name="test"> <![CDATA[]]> </attribute> <!-- test --> <attribute name="requirement"> <![CDATA[CALC-1235]]> </attribute> <!-- requirement --> <attribute name="labels"> <![CDATA[core]]> </attribute> <!-- labels --> </attributes> </test-method> <!-- CanAddNumbersFromGivenData --> <test-method status="FAIL" signature="CanDoStuff()[pri:0, instance:com.xpand.java.CalcTest@36d4b5c]" name="CanDoStuff" duration-ms="0" started-at="2018-03-06T11:53:00Z" finished-at="2018-03-06T11:53:00Z"> <exception class="java.lang.AssertionError"> <message> <![CDATA[null]]> </message> <full-stacktrace> <![CDATA[java.lang.AssertionError: null at org.testng.Assert.fail(Assert.java:93) at org.testng.Assert.assertNotEquals(Assert.java:897) at org.testng.Assert.assertNotEquals(Assert.java:902) at com.xpand.java.CalcTest.CanDoStuff(CalcTest.java:86) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:108) at org.testng.internal.Invoker.invokeMethod(Invoker.java:661) at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:869) at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1193) at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:126) at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109) at org.testng.TestRunner.privateRun(TestRunner.java:744) at org.testng.TestRunner.run(TestRunner.java:602) at org.testng.SuiteRunner.runTest(SuiteRunner.java:380) at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:375) at org.testng.SuiteRunner.privateRun(SuiteRunner.java:340) at org.testng.SuiteRunner.run(SuiteRunner.java:289) at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52) at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86) at org.testng.TestNG.runSuitesSequentially(TestNG.java:1301) at org.testng.TestNG.runSuitesLocally(TestNG.java:1226) at org.testng.TestNG.runSuites(TestNG.java:1144) at org.testng.TestNG.run(TestNG.java:1115) at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:283) at org.apache.maven.surefire.testng.TestNGXmlTestSuite.execute(TestNGXmlTestSuite.java:75) at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:120) at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:373) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:334) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:119) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:407) ]]> </full-stacktrace> </exception> <!-- java.lang.AssertionError --> <reporter-output> </reporter-output> </test-method> <!-- CanDoStuff --> <test-method status="PASS" signature="CanDivide()[pri:0, instance:com.xpand.java.CalcTest@36d4b5c]" name="CanDivide" duration-ms="0" started-at="2018-03-06T11:53:00Z" finished-at="2018-03-06T11:53:00Z"> <reporter-output> </reporter-output> <attributes> <attribute name="test"> <![CDATA[]]> </attribute> <!-- test --> <attribute name="requirement"> <![CDATA[CALC-1235]]> </attribute> <!-- requirement --> <attribute name="labels"> <![CDATA[core]]> </attribute> <!-- labels --> </attributes> </test-method> <!-- CanDivide --> <test-method status="PASS" signature="CanMultiplyX()[pri:0, instance:com.xpand.java.CalcTest@36d4b5c]" name="CanMultiplyX" duration-ms="0" started-at="2018-03-06T11:53:00Z" finished-at="2018-03-06T11:53:00Z"> <reporter-output> </reporter-output> <attributes> <attribute name="test"> <![CDATA[]]> </attribute> <!-- test --> <attribute name="requirement"> <![CDATA[CALC-1235]]> </attribute> <!-- requirement --> <attribute name="labels"> <![CDATA[core]]> </attribute> <!-- labels --> </attributes> </test-method> <!-- CanMultiplyX --> <test-method status="PASS" signature="CanSubtract()[pri:0, instance:com.xpand.java.CalcTest@36d4b5c]" name="CanSubtract" duration-ms="0" started-at="2018-03-06T11:53:00Z" finished-at="2018-03-06T11:53:00Z"> <reporter-output> </reporter-output> <attributes> <attribute name="test"> <![CDATA[]]> </attribute> <!-- test --> <attribute name="requirement"> <![CDATA[CALC-1235]]> </attribute> <!-- requirement --> <attribute name="labels"> <![CDATA[core]]> </attribute> <!-- labels --> </attributes> </test-method> <!-- CanSubtract --> <test-method status="PASS" signature="tearDown()[pri:0, instance:com.xpand.java.CalcTest@36d4b5c]" name="tearDown" is-config="true" duration-ms="0" started-at="2018-03-06T11:53:00Z" finished-at="2018-03-06T11:53:00Z"> <reporter-output> </reporter-output> </test-method> <!-- tearDown --> </class> <!-- com.xpand.java.CalcTest --> </test> <!-- calculator --> </suite> <!-- TestAll --> </testng-results>
Example Request
curl -H "Content-Type: text/xml" -X POST -H "Authorization: Bearer $token" --data @"data.xml" https://xray.cloud.xpand-it.com/api/v1/import/execution/testng?projectKey=XTP
curl -H "Content-Type: text/xml" -X POST -H "Authorization: Bearer $token" --data @"data.xml" https://xray.cloud.xpand-it.com/api/v1/import/execution/testng?testExecKey=XNP-23
curl -H "Content-Type: text/xml" -X POST -H "Authorization: Bearer $token" --data @"data.xml" https://xray.cloud.xpand-it.com/api/v1/import/execution/testng?projectKey=XTP&testExecKey=XNP-23
curl -H "Content-Type: text/xml" -X POST -H "Authorization: Bearer $token" --data @"data.xml" https://xray.cloud.xpand-it.com/api/v1/import/execution/testng?projectKey=XTP&testPlanKey=XTP-12&revision=v2.1.0
Responses
200 OK : application/json : Successful. The results where successfully imported to Jira.
{ "testExecIssue": { "id": "10200", "key": "XNP-24", "self": "http://www.example.com/jira/rest/api/2/issue/10200" } }
400 BAD_REQUEST : application/json : Returns the error.
401 UNAUTHORIZED : application/json : The Xray license is not valid.
500 INTERNAL SERVER ERROR : application/json : An internal error occurred when importing execution results.