About XUnit
xUnit is a testing framework for .net languages (e.g. C#, F#, VB.NET) mostly focused on unit testing but that can also be used for integration testing.
xUnit works with ReSharper, CodeRush, TestDriven.NET, and Xamarin.
Supported versions
Xray supports https://xunit.net/docs/format-xml-v2.
xUnit Basic Concepts
In xUnit, you have Tests and Collections. A Collection is a way of aggregating a group of tests, along with their results.
There are 2 types of tests in xUnit:
- Facts are tests that are always true. They test invariant conditions.
- Theories are tests that are only true for a particular set of data.
Both of these types of tests are treated similarly by the results importer.
Importing xUnit XML reports
Below is a simplified example of a xUnit XML report containing a Collection with two Test Cases.
<?xml version="1.0" encoding="utf-8"?> <assemblies timestamp="07/31/2018 14:58:48"> <assembly name="C:\Users\smsf\Documents\Visual Studio 2015\Projects\xUnitDemo\xUnitDemo\bin\Debug\xUnitDemo.DLL" environment="64-bit .NET 4.0.30319.42000 [collection-per-class, parallel (1 threads)]" test-framework="xUnit.net 2.3.1.3858" run-date="2018-07-31" run-time="14:58:47" config-file="C:\Users\smsf\Documents\Visual Studio 2015\Projects\xUnitDemo\packages\xunit.runner.console.2.4.0\tools\net452\xunit.console.exe.Config" total="15" passed="14" failed="1" skipped="0" time="0.257" errors="0"> <errors /> <collection total="2" passed="1" failed="1" skipped="0" name="Test collection for xUnitDemo.SimpleTests" time="0.070"> <test name="xUnitDemo.SimpleTests.PassingTest" type="xUnitDemo.SimpleTests" method="PassingTest" time="0.0636741" result="Pass"> <traits> <trait name="test" value="CALC-2" /> <trait name="requirement" value="CALC-1" /> <trait name="labels" value="core UI" /> </traits> </test> <test name="xUnitDemo.SimpleTests.FailingTest" type="xUnitDemo.SimpleTests" method="FailingTest" time="0.0059474" result="Fail"> <failure exception-type="Xunit.Sdk.EqualException"> <message><![CDATA[Assert.Equal() Failure\r\nExpected: 5\r\nActual: 4]]></message> <stack-trace><![CDATA[ at xUnitDemo.SimpleTests.FailingTest() in C:\Users\smsf\documents\visual studio 2015\Projects\xUnitDemo\xUnitDemo\SimpleTests.cs:line 30]]></stack-trace> </failure> </test> </collection> </assembly> </assemblies>
The simplified tags hierarchy of these reports can be represented in the following diagram:
Entities
xUnit’s Test Cases are identified by the pair of attributes “type” and “method” attributes.
Test Cases are imported to Xray’s Generic Test issues, and the “type” and “method” attributes are concatenated and mapped to the Generic Test Definition field of the Generic Test.
If a Test already exists with the same Generic Test Definition, then a duplicate is not created.
Test Cases are imported to a new (or user-specified) Test Execution in the context of some project, along with their respective execution results.
Parameterized Tests are imported to the same Test Run, where each set of parameters is identified by a different context.
xUnit’s Test Suites are not mapped to any special entity. However, the execution details screen will show the Test Suite related to a specific test result.
Mapping of fields from the report to the Test issue
xUnit.net XML report | Test in Jira |
---|---|
"method" attribute | Summary field |
"type" attribute + "." + "method" attribute | Generic Test Definition custom field |
"value" attribute of the "trait" whose name is "labels" under the <traits> tag, within some <test>, containing one or more labels delimited by space | labels |
"value" attribute of the "trait" whose name is "test" under the <traits> tag, within some <test>, containing a Jira key of the Test issue | identification of Test issue key in Jira |
"value" attribute of the "trait" whose name is "requirement" under the <traits> tag, within some <test>, containing one Jira key of the requirement/coverable issue | link to requirement |
Notes:
- If the "test" Trait contains multiple references to Test entities, only the first instance is considered.
- If the "test" Trait is used explicitly, then the Test must exist; otherwise, it will not be imported.
The text will be trimmed to match the 255-character limit on the summary field.
Status
The status of the Test Run will be set based on the attribute result of the Test:
Test Result | Test status |
---|---|
pass | PASS |
fail | FAIL |
no value Skip To Do A non-existent status | TO DO |
Note: Test Cases with the status FAIL may have an error/failure message, which can be seen in the Test Run screen, under the Results section.
References