ISTQB defines some terms related to testing and quality in general. Sometimes, there may are subtle differences in terminology.
The following table details some ISTQB terms and how they map to Xray concepts.
| ISTQB | ISTQB definition | Xray | Notes (from Xray's point-of-view) |
|---|---|---|---|
Coverage Item Testable Requirement | An entity or property used as a basis for test coverage A requirement that is stated in the terms that allow the creation of test designs (and subsequently, test cases) and the execution of tests to determine whether the requirement has been met | An issue configured as being part of "coverable issue types", i.e., it can be covered by a Test | In Xray, any issue may be covered by Tests as long as the corresponding Issue Type is configured as a "coverable issue type". A testable/coverable entity can be a requirement (e.g., story, epic, feature) or even a defect, if configured in that way. |
| Defect | A flaw in a component or system that can cause the component or system to fail to perform its required function, e.g., an incorrect statement or data definition. A defect, if encountered during execution, may cause a failure of the component or system. | An issue configured as being part of "defect issue types" | In Xray, defects can be any Issue Types configured as such (e.g., Bug). |
| Test Cycle | The execution of the test process against a single identifiable release of the test object. | Test Execution or a Test Plan and its related Test Executions | This concept does not exist as a direct entity in Xray. It depends on how the test process and its activities are organized. If multiple iterations (e.g., Xray's Test Executions) are going to be made for a specific release of the test object, then a Test Plan may be created for grouping all these iterations related with a specific version/revision of the SUT. If just one Test Execution is going to be scheduled, then the test cycle maps directly to it. |
Test Object SUT | The component or system to be tested | Jira's project | |
| Test Case | A set of input values, execution preconditions, expected results and execution postconditions, developed for a particular objective or test condition, such as to exercise a particular program path or to verify compliance with a specific requirement | Test, along with corresponding Preconditions | In Xray, the full specification of a Test consists of the Test details (e.g., manual steps) plus the initial conditions required by it, as defined in the Preconditions linked to the Test issue. Postconditions are not unsupported. |
| Test Suite | A set of several test cases for a component or system under test, where the postcondition of one test is often used as the precondition for the next one. | Test Set | In Xray, the Test Set provides an ordered list of Tests. |
Test Procedure | A sequence of test cases in execution order, and any associated actions that may be required to set up the initial preconditions and any wrap-up activities post-execution. | Test Execution | A Test Execution contains an ordered list of Tests to be run, a target schedule (begin & end dates) along with the recorded results. |
| Test Execution | The process of running a test on the component or system under test, producing actual result(s). | The activity of executing a Test Run | |
| Test Run | The execution of a test on a specific version of the test object. | Test Run | |
| Test Log | A chronological record of relevant details about the execution of tests | The details of a specific Test Run can be seen in the activity section of the corresponding execution screen/details page. Within the Test issue screen, all Test Runs related to a given Test can be seen in this section. | |
| Test Plan | A document describing the scope, approach, resources and schedule of intended test activities. It identifies amongst others test items, the features to be tested, the testing tasks, who will do each task, degree of tester independence, the test environment, the test design techniques and entry and exit criteria to be used, and the rationale for their choice, and any risks requiring contingency planning. It is a record of the test planning process. | Test Plan | In Xray, a Test Plan is not a document, but an issue where the version of the target SUT is identified through the fix Version; the revision/build can be specified whenever creating Test Executions from the Test Plan. The Test Plan defines the scope of the test activities, including the tests to track and, indirectly, the requirements. Planned Test Executions (i.e., the ones linked to the Test Plan) can be created for different users. Note: generating a test plan as a document can easily be achieved using Xporter along with a proper template (e.g. Xray Test Plan Advanced with Cover Page) |
| Test Planning | The activity of establishing or updating a test plan | Creating/updating a Test Plan | This activity also includes creating the related Test Executions for the respective Test Environments, and assigning them to users. |
| Test case Specification | A document specifying the test conditions (coverage items) for a test item, the detailed test approach and identifying the associated high-level test cases. | - | Generating a test design specification as a document can easily be achieved using a proper template in Xporter (e.g., Xray Test Report). |