Page History
A Test Run (sometimes simply referred to as a "run") occurs when a Test is scheduled to execute within a Test Execution. Therefore, each time you run a Test, you're, in fact, running a Test Run.
A Test Run is an internal instance of a Test scenario with different working conditions, such as a different environment or source codethat is used to validate some specific version of source code/SUT, in a specific environment. Various test runs allow you to easily detect defects in the code defects that could lead to system failures. A Test Run is therefore each association between a different Test and Test Execution issues.
A Test Run is not a JIRA Jira issue itself; it's an Xray internal entity that you can search (see see Enhanced querying with JQL).
Normally, you may see Test Runs within specific panels, namelly such as in a Test Execution (a test run per each testTest) or within the Test issue screen (history of previous runs for that Test).
| Info |
|---|
A Test Execution represents a task for running several Tests. It contains as many Test Runs as the number of Tests associated with it. To be precise, there is one Test Run per associated Test, in order to track the result for each Test. |
| Table of Contents |
|---|
What is stored in a Test Run
A Test Run contains a copy of the testing specification of the Test and associated Pre-Conditions. This specification only comprises some fields of these entities; thus, not all fields are copied to the Test Run.
General custom fields, summary, description fields are not replicated in the Test Run. Thus, you should avoid using those fields to add "relevant" information to the test specification.
Depending on the original type of the Test associated with this Test Run, these are the fields are replicated in the Test Run.
| Manual Tests | Cucumber Tests | Generic Tests | |
|---|---|---|---|
| Test |
|
|
|
| linked Pre-Condition(s) |
|
|
|
Since a Test Run is related to the execution of a Test, obviously, it contains comments and information about linked defects and evidences (i.e., attachments).
Data consistency
...
Test Runs represent the results of running some Test specification at some moment in time, in the context of some Test Execution. Thus, they assure data consistency.
So, what happens if you change the Test specification at any time? Your existing Test Runs will not change (unless you want them to), in order to ensure data consistency and compliance with conformance regulations.
At the execution screen of a given Test Run, you will be presented with two options:
- Merge current Test specification with the current specification in the Test Run
- Reset the current Test Run, by copying the current Test specification to the Test Run and discard all recorded results
| Info | ||
|---|---|---|
| ||
| More information on these actions can be found in the Execute Tests page. |
Execution History
The execution history of a given Test issue is available in the Test issue view page screen in the section named "Test Runs" section.
The Test Runs section is headlined by the Test Runs Filter , that allows the user you to sort for:
...
- the Project in which the Test Execution the current Test is associated with
...
- the target
...
- release Version
...
- tested by the Test Execution the current Test is associated with
...
...
- the Status
...
- of the Test Execution the current Test is associated with
...
...
- the Start
...
- and End
...
- dates of the Test Execution the current Test is associated with
...
...
- the Clear
...
- button, that empties all the Test Runs Filter fields
...
The search results are displayed on the table immediately under the Test Runs Filter, containing the following columns:
...
- the Execution Key of the listed Test Execution the current Test is associated with
...
...
- the Fix Version tested by the listed Test Execution the current Test is associated with
...
- the System Revision tested by the listed Test Execution the current Test is associated with
...
...
- the Exec by
...
- who is the Assignee of the listed Test Execution the current Test is associated with
...
- the Start and End dates of the listed Test Execution the current Test is associated with
...
...
- the Defects
...
- related with the listed Test Execution the current Test is associated with
...
- the Status of the listed Test Execution the current Test is associated with
...
Actions
The "Execute Test" and "Execute Test Inline" actions are available in the Test issue screen. They may also be available in similar Test Runs sections that may be present in other places.
| Table of Contents | ||||
|---|---|---|---|---|
|
...
Step 2: Click the Execute button that appears in the last column of the desired Test Run. The actions menu should popup pop up with the available actions.
...
Execute Test Inline
To execute Test Runs inline Inline from the Test view the Test issue screen, this option must be enabled in the Xray administration page. The option for setting the Test Run status manually, without having to execute all Test Steps (or Examples in the case of Cucumber Tests), must also be enabled in the Xray administration page.
Given that If the above options are enabled , and the user has you have permission to execute the Test, the context menu for executing Test Runs, accessed be the Execute button on each Test Run must show the available transitions.
When executing inline Tests, the status of the manual steps (or Examples in the case of Cucumber Tests), can be changed automatically. The following rules are applied:
- PASS: All steps statuses will be changed to PASS independently independent of their current status.
- FAIL: All steps statuses will remain unchanged, even if they are all PASS. This is because a Test might fail due to a Pre-Condition.
- TODO: All steps statuses will be changed to TODO.
- ABORTED: The Test execution status will be changed to ABORTED.
- EXECUTING: The Test execution status will be changed to EXECUTING.
- CUSTOM: The Test execution status will be changed to chosen the specified custom status.
To execute a Test Run inline:
...
To view the execution details:
Step 1: Open the test Test issue you wish to view the test execution details.
Step 2: Click the Execute button that appears in the last column of the desired Test Run that is in a final state. The actions menu should popup pop up with the available actions.
...
Ad hoc Test Execution
To execute a test in an Ad ad hoc manner:
Step 1: Open the test Test issue you wish to execute.
Step 2: Click the "Execute In" button in Test Runs section and select "New Test Execution...".
Step 3: The dialog comes with pre-populated fields so if you don't want to change anything, just click Create.
Note: if If there are custom required fields, the normal issue create dialog will be displayed with the pre-populated fields. If the revision field is not configured in the create issue screen , of the Test Execution for the selected Project, the field will not appear in the dialog.
...
If the Redirect To Execute Test Page option is selected, you will be redirected to the Execute Test page after the test execution is created.
...