Please note that the articles in this documentation feature media from app version 2.0.5. Interface elements may differ in newer versions. Refer to the latest app version for the most accurate information. |
While performing the test sessions, testers collect and organize information so they can share it with the stakeholders or with the internal team.
In this article, we'll cover the essentials of collecting data that can be turned into actionable information, so that we can improve the product or system we're building.
To share findings & evidence properly, we need to think about our goal: why are we doing it?
Perhaps we're collecting information for ourselves, or maybe we're thinking about reporting defects so the developers can have a look at them. Maybe we have an idea to share with the PO or with the team for further discussion. Maybe something really works well, and the team needs to know that. Knowing to whom we're collecting information is relevant as we may need tailored information for them.
Let's see a brief summary of information that we can collect during exploratory testing and how it can be organized.
Notes in the form of text are the most common way of storing our findings. These notes can be of different natures.
Testers use labels, or categories to distinguish text notes, to make them easier to find afterward.
Next are some examples of possible labels/categories that are frequently used, along with their meaning:
The evidence consists mainly of files that we either extract from the application (e.g., logs, other artifacts) or that are strongly connected to the system (e.g., screenshots, videos).
Audio recordings, usually taken more as a verbalized text note, can also be included in this section as they will be stored as a file that we can upload as evidence.
Evidence is like facts that are of great help later on, for example, for bug analysis and fixing.
In sum, we can have the following:
Start with a few categories (e.g., Problem, Question, Idea, Praise) and adapt to your needs.
In the end, add the mindmap or a link to it, as evidence or as a note in your test session.
Whenever taking text notes, avoid mentioning someone as the cause of the problem. Focus instead on technical facts/evidence. It's ok to mention some decisions that may have been taken in the past as it can be connected to the problem and may also point to other related impacts.
Screenshots are quite valuable as they make information visible, which would be hard to put into words. However, we often want to refer to a certain part of the application/site. By drawing around what we want, we can make it clearer for review later on.
In the Xray Exploratory App, it is possible to draw squares, circles, or just freestyle drawings on top of the screenshot.
Take a screenshot and add some text, marking it as a Problem, Idea, or other. This will provide more context to the screenshot you're taking.
In the app, this is done at the same time you take the screenshot, right after the screenshot, at the bottom.
To provide greater focus and avoid some noise, record just the browser, or the application's UI. Record the whole screen if it makes more sense for your use case (e.g., the interaction of a desktop application with other applications).
In the Xray Exploratory App, the selection of what to record (i.e., the desktop, a specific application) whenever performing video recordings, or screenshots, is done at the start of the testing session. It can also be performed after resuming a paused session.
To provide greater focus and avoid some noise, record just the browser, or the application's UI. Record the whole screen if it makes more sense for your use case (e.g., the interaction of a desktop application with other applications).
In the app, the selection of what to record (i.e., the desktop, a specific application) whenever performing video recordings, or screenshots, is done at the start of the testing session. It can also be performed after resuming a paused session.
If you prefer to verbalize while testing, then record audio notes and review them later.
In the app, you can easily take audio nodes.
Combine screen recording with audio notes, so you can stay even more focused on testing.
In the app's global settings, you can enable "Audio on screen capture."
Upload some files as evidence and add some text, marking it as a Problem, Idea, or other. This will provide more context to the screenshot you're taking.
his is done at the same time you upload evidence, at the bottom.
Record the browser network traffic and upload it to XEA. In Chrome, this can be found in "Developer Tools > Network"; make sure to select "Preserve Log" option.
Later on, the .har file can be uploaded from the same place to be analyzed in further detail.
Add the .har file as standard evidence to your test session. Switch to the "Timeline" view and then drag the file to XEA.
Problems often occur due to having the target system/application in a certain state, which in turn is the consequence of a sequence of steps. To replicate problems/situations, we need to remember the sequence of actions performed.