Page History
...
Exploratory testing is an approach for testing where the testers plays a key role on in uncovering problems that otherwise could stay unnoticed.
...
Learn more here.
Process at a glance
In exploratoryt exploratory testing, usually we start with a goal, or a so-called charter. The charter is a high-level description of what we aim to test and the why, for the purpose of the time we decide to allocate to it.
...
In sum, an exploratory test in Xray will be a Test. A testing session, that represents the testing performed for a given charter, will be abstracted as a Test Run for the exploratory Test.
Exploratory Testing (ET) concept | Xray concept |
---|---|
Test / Test charter | Test
|
Session | Test Run |
Info | ||
---|---|---|
| ||
| ||
Info | ||
| ||
|
Using Xray built-in capabilities
...
- Create an exploratory Test issue
- for the Summary, use a name that summarizes the test charter
- choose the proper test type, as mentioned earlier
- fill out the "Generic Test Definition" field, with the test charter
Performing the test session (i.e., "executing" the test)
To perform Before performing our test session, we start it by scheduling need to schedule an execution for our Test. The way to do this in Xray, is to schedule a Test Execution for our Test (or to include our Test in an existing Test Execution).
Test Executions can be created from multiple places (e.g., from Test Repository, Test Plan, etc), including directly from the Test issue screen.
Info | ||
---|---|---|
| ||
If you have multiple testers involved in your exploratory session, then you can create different Test Executions each one assigned to a different tester; you may use the Test Execution planning dates to define the fixed time-frame allocated to your session. The Test Execution description can be used to specify the tester charter, which complements and focuses the session charter for the tester(s) involved in that Test Execution. Please note that a Test Execution can only contain the same Test onc |
During the test session, on the Test Run execution screen, we can:
- report the overall status of our testing session, using the Execution Status field. For example, as PASS if all seems good enough, or as FAIL if there are major issues; you can also use custom test run statuses if you want
- leave notes, using Comment section. As it supports wiki markup, we can use it to organize our text notes; even though there's no structure, we recommend having a brief summary and having the text notes organized by categories
- report defects, on the Execution Defects section
- attach evidence, on the Execution Evidence section. We can attach logs, or other files that may be relevant, including screenshots; we can use our tool of choice for taking screenshots. You can use copy&paste; whenever "pasting" an image the following dialog will appear
Using Xray together with Xray Exploratory App
The Xray Exploratory App (XEA) is a side companion desktop app that is used to assist testers on exploratory testing related tasks.
Info | ||
---|---|---|
| ||
|
To be able to use Xray Exploratory App, please see its Getting Started documentation that details the installation and configuration process.
Concerning the integration with Xray, there are two main flows to have in mind:
- define the test charter in Xray, by creating an exploratory Test, and then use XEA to assist on note-taking and report results back to Jira, where the team can analyze them and track its impacts
- use XEA to define the test charter, assist on note and evidence taking during testing, and then report results back to Jira, where the team can analyze them and track its impacts
The subtle difference is which tool we'll use to create/define the exploratory Tests. Teams may use the first approach, where Xray is the master, if they wish to follow similar processes as they do for manual scripted test cases, and have some sort of management/planning.
Starting an exploratory testing session for an existent Test in Xray
In this scenario, we're considering that an exploratory test and its charter have already been defined and exists in Xray.
The exploratory test may have been created in Xray (see above instructions on how to use Xray for defining exploratory tests) or by Xray Exploratory App on a previous session (please see information ahead on how to use Xray Exploratory App to create Tests).
Having the Test, how can we start our testing session and use XEA to assist us?
We can either start the session right from Xray, or directly from XEA.
Starting the session from Xray
To start a session from Xray, we need first to schedule a execution for our test by creating a Test Execution with it, or adding it to an existing Test Execution and then use the option to execute it using Xray Exploratory App from the test run details screen.
Then start your session.
First, you need to select the screen/application you wish to record (in case you aim to take screenshots or record videos).
We can then perform exploratory testing and use Xray Exploratory App to assist us during the process. In the end, results can be uploaded to Xray to the original Test Execution created in Xray, being associated with the corresponding Test Run of the exploratory Test.
Info |
---|
While exploring your target app/system, use Xray Exploratory App to:
After finishing exploring, end your session, review it, choose properly the test status you aim to report, and choose the option to upload the results to Xray and Jira (i.e., "Submit Test Run"). |
Please see Xray Exploratory App's documentation for more information on executing exploratory testing sessions and all the available options.
Starting the session directly from XEA
Assuming Xray Exploratory App is installed and the connection to the Jira/Xray instance have been configured, we just need to start Xray Exploratory App and choose "New Session for Existent Test".
Then we need to select the Jira project where we have our Test, and then identify it by its issue key. Then we just need to load it and start our testing session.
We can then perform exploratory testing and use Xray Exploratory App to assist us during the process. In the end, results can be uploaded to Xray to a newly created Test Execution having the Test we selected along with its result (i.e., the corresponding Test Run).
Please see Xray Exploratory App's documentation for more information on executing exploratory testing sessions and all the available options.
Create a new exploratory test using XEA and report results to Jira and Xray
Assuming Xray Exploratory App is installed and the connection to the Jira/Xray instance have been configured, we just need to start Xray Exploratory App and choose "New Session".
Then we need to select the Jira project where we aim to create our Test on, along with some additional information:
- Summary: for a brief summary of our test charter
- Test Type: make sure to select the proper you will be using as an abstraction of exploratory tests (e.g., "Exploratory", "Generic")
- Charter: the test charter
We can then perform exploratory testing and use Xray Exploratory App to assist us during the process. In the end, results can be uploaded to Xray to a newly created Test Execution having the newly created Test along with its result (i.e., the corresponding Test Run).
Please see Xray Exploratory App's documentation for more information on executing exploratory testing sessions and all the available options.