A test execution is the activity that occurs between developing test scripts and reporting and analyzing said test results. This activity is more complex than starting a test and monitoring it to ensure that the test is running as expected. Actually, it involves
- Validating the test environment to ensure that it truly represents the production environment. If it is different from the environment designed for the tests to be run against, there is a high probability that tests might not work or will provide misleading data. For the same reason, tests should also be validated to ensure that they reflect the simulation design.
- Running the test based on the plans and test documents once it is determined that the test target is in an appropriate state to have performance tests run against it. After completing the test execution, a brief summary addressing what happened during the test should be compiled and added to the test log for future reference. Also, if the analysis of results leads to new priorities, then it is important to modify the tests, test designs, and subsequent strategies.
Generating the test results and report any errors found to the development team.
Xray provides the Test Execution issue, which acts as an assignable task for executing a group of tests (either manual or automated) and collects the corresponding test results for a specific project version and environment.
A Sub-Test Execution is similar to a Test Execution, but created as a sub-task of a requirement.
Normally, you will create Test Executions from Test Plans, or create ad hoc Test Executions (directly from a Test).
Once you have your Test Executions created, you're ready to execute them. Each time you schedule a given Test for execution, a Test Run will be created and Xray will keep the information related to that specific execution of that specific Test.