ISTQB defines some terms related with testing and quality in general. Sometimes there may be some subtle differences in terminology.
The following table details some ISTQB terms and how they map to Xray concepts.
| ISTQB | ISTQB definition | Xray | Notes, from Xray's point-of-view |
|---|---|---|---|
coverage item testable requirement | An entity or property used as a basis for test coverage. A requirements that is stated in terms that permit establishment of test designs (and subsequently test cases) and execution of tests to determine whether the requirement has been met. | an issue configured as being part of "requirement issue types", that can be covered with Tests | In Xray, any issue may be covered with Tests as long as the respective Issue Type is configured to be handled as a "requirement issue type". A testable entity can be a requirement (e.g. story, epic, feature) or even a defect, if configured in that way. |
| defect | A flaw in a component or system that can cause the component or system to fail to perform its required function, e.g., an incorrect statement or data definition. A defect, if encountered during execution, may cause a failure of the component or system. | an issue configured as being part of "defect issue types" | In Xray, defects can be any Issue Types configured as such (e.g. Bug). |
| test cycle | Execution of the test process against a single identifiable release of the test object. | Test Execution or a Test Plan and its related Test Executions | This concept does not exist as a direct entity in Xray. It depends on how the test process and its activities are organized. If multiple iterations (e.g. Xray's Test Executions) are going to be made for a specific release of the test object, then a Test Plan may be created for grouping all these iterations related with a specific version/revision of the SUT. If just one Test Execution is going to be scheduled, then the test cycle maps directly to it. |
test object SUT | The component or system to be tested. | Jira's project | |
| test case | A set of input values, execution preconditions, expected results and execution postconditions, developed for a particular objective or test condition, such as to exercise a particular program path or to verify compliance with a specific requirement. | Test along with respective Pre Conditions | In Xray, the full specification of a Test comprises the Test details (e.g. manual steps) plus the initial conditions required by it, defined in Pre Conditions linked to the Test issue. Post-conditions aren't unsupported. |
| test suite | A set of several test cases for a component or system under test, where the post condition of one test is often used as the precondition for the next one. | Test Set | in Xray, the Test Set provides an ordered list of Tests. |
test procedure | A sequence of test cases in execution order, and any associated actions that may be required to set up the initial preconditions and any wrap up activities post execution. | Test Execution | A Test Execution contains an ordered list of Tests to be run, a target schedule (begin & end dates) along with the recorded results. |
| test execution | The process of running a test on the component or system under test, producing actual result(s). | The activity of running a Test Run | |
| test run | Execution of a test on a specific version of the test object. | Test Run | |
| test log | A chronological record of relevant details about the execution of tests. | The details of a specific Test Run can be seen in the activity section in the respective execution screen/details page. All Test Runs related with a given Test can be seen in the respective section, within the Test issue screen. | |
| test plan | A document describing the scope, approach, resources and schedule of intended test activities. It identifies amongst others test items, the features to be tested, the testing tasks, who will do each task, degree of tester independence, the test environment, the test design techniques and entry and exit criteria to be used, and the rationale for their choice, and any risks requiring contingency planning. It is a record of the test planning process. | Test Plan | One Xray's Test Plan is not a document but an issue instead, where the version of the target SUT is identified through the fixVersion; revision/build can be specified whenever creating Test Executions from the Test Plan. The Test Plan defines the scope of the test activities, including the tests to track and, indirectly, the requirements. Planned Test Executions (i.e. the ones linked to the Test Plan) can be created for different users. Note: generating a test plan as a document can easily be achieved using Xporter along with a proper template (e.g. Xray Test Plan Advanced with Cover Page) |
| test planning | The activity of establishing or updating a test plan. | creating/updating a Test Plan | This activity also includes creating the related Test Executions, for the respective Test Environments, and assigning them to users. |
| test case specification | A document specifying the test conditions (coverage items) for a test item, the detailed test approach and identifying the associated high-level test cases. | - | Generating a test design specification as a document can easily be achieved using Xporter along with a proper template (e.g. Xray Test Report) |