Page History
...
Expand | |||||||||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| |||||||||||||||||||||||||||||||||||||||||||||
|
...
Expand | |||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| |||||||||||||||||||||||||||||
|
...
Whenever importing results from some frameworks (i.e. JUnit, TestNG, NUnit, Robot framework), Xray can identify the automated test from the report/results file, based on an hardcoded criteria (such as the class name plus the class method corresponding to the automated test). However, depending on the test automation framework, it's possible to specify the Test issue key to which report the results in the test's code.
Independently of whether the test is identified implicitly (based on some attributes present in the test result file) or explicitly (based on the provided Test key), related Test Runs are always reported against the correct Test issue. As a consequence, if you report results multiple times there won't be duplicated Test entities.
When the identification is implicitThis means that in some scenarios, Xray is able to create (Generic) Test entities, if needed. These may be used afterwards, based on their Generic Definition field, if results related with those automated tests are reported once again. However, as mentioned earlier, it's also possible to enforce the Test entity to which report to. if needed, per each automated test; these will be reused afterwards in similar cases.
Whenever processing results from a automation framework, for each automated test result,
- If the Test key is provided and...
- it exists, then create a Test Run for that Test
- it doesn't exist, then don't create any Test Run (since for some reason the explicitly identified Test does not exist)
- if no Test key is provided...
- try to find a Test in the identified project with the same Generic Test Definition (e.g. with the same class name+class method for example)
- if it exists, then create a Test Run for that Test
- if it doesn't exist, then search for a a Test with the same Generic Test Definition in all JIRA projects
- if it exists, then create a Test Run for that Test
- it doesn't exist, then create a Test in the identified project (based on the project's key or the project associated with the provided test execution key)
- try to find a Test in the identified project with the same Generic Test Definition (e.g. with the same class name+class method for example)
For some frameworks, including Cucumber and Behave, Tests must exist previously to the submission of results related to them.
...