Overview
While performing the test sessions, testers collect and organize information so they can share it with the stakeholders or with the internal team.
In this article we'll cover the essentials of collecting data that can be turned into actionable information, so that we can improve the product or system we're building.
Findings & evidence
In order to share findings & evidence properly, we need to think about our goal: why are we doing it?
- To ease replication of a certain situation or bug
- To make RCA (Root-Cause Analysis) easier, and get to the source of the problem
Maybe we're collecting information for ourselves. Or maybe we're thinking about reporting defects so the developers can have a look at them. Maybe we have an idea to share with the PO or with the team for further discussion. Maybe something really works surprising well and the team needs to know that. Knowing to whom we're collecting information is relevant as we may need tailored information for them.
Let's see a brief summary of information that we can collect during exploratory testing and how it can be organized.
Text notes
Notes in the form of text are the most common way of storing our findings. These notes can be of different nature.
Testers use labels, or categories to distinguish text notes, to make them easier to find afterwards.
Next are some examples of possible labels/categories that are frequently used, along with their meaning:
- Concern
- Something that represents a potential, negative risk that can somehow afect value, now, or in the future.
- Problem
- A clear problem that impacts value to some stakeholder (internal or external).
- Question
- Something that is not clear; may or not affect value.
- Suggestion
- A recommendation/suggestion to change some existing aspect/feature.
- Idea
- Some idea to make the product better and thus improve it, usually by recommending new features. Eventually you may merge "Suggestion" with "Idea".
- Praise/Positive
- Some positive aspect we have found and that we want to share with the stakeholders, usually the internal team.
- Comment
- A general note/comment.
Evidence and audio recordings
Evidence consists mainly of some sort of files that we either extract from the application (e.g., logs, other artifacts) or that are somehow strongly connected to the system (e.g., screenshots, videos).
Audio recordings, usually taken more as a verbalized text note, can also be included in this section as they will be stored as a file that we can upload as evidence.
Evidence are like facts that are of great help later on, for example for bug analysis and fixing.
In sum, we can have:
- screenshots
- video/screen recordings
- audio recordings
- logs
- application logs
- JS console logs
- crash dumps
- ...
Tips for collecting and organizing information
Text notes
Assess the different categories you want to have to mark information
Start with a few categories (e.g., Problem, Question, Idea, Praise) and adapt to your needs.
Organize text notes using mindmaps
- some users prefer a more visual and hierarchical way of organizing findings; mindmaps may work well, either using tools such as Miro, Mindmup, Xmind, or even the free Freemind tool
- mindmaps allow a totally flexible yet non-standard way to organize way notes; this may create some challenges navigating through different test session results and even search for what matters in a given session
In the end, add the mindmap or a link to it, as evidence or as a note in your test session.
Focus on the problem and not on the individual
Whenever taking text notes, avoid mention someone as the cause of the problem. Focus instead on technical facts/evidence. It's ok to mention some decision that may have been taken in the past as it can be connected to the problem and may also point other related impacts.
Screenshots
Use visual annotations to highlight relevant areas to make explanations clearer
TBD
Combine screenshots with categorized notes
Take a screenshot and add some text, marking it as a Problem, Idea, or other. This will provide more context to the screenshot you're taking.
In XEA this is done at the same time you take the screenshot.
Video/screen recordings
Use visual annotations to highlight relevant areas to make clear some other explanation
TBD
Record only what matters
To provide greater focus and avoid some noise, record just the browser, or the application's UI. Record the whole screen if makes more sense for your use case (e.g., interaction of a desktop application with other applications).
Audio recordings
Use audio annotations
TBD
Evidence
Upload evidence and add categorized notes to provide better context
Upload some file as evidence and add some text, marking it as a Problem, Idea, or other. This will provide more context to the screenshot you're taking.
If using the browser, record the traffic to a .har file
Record the browser network traffic and upload it to XEA. In Chrome, this can be found in "Developer Tools > Network"; make sure to select "Preserve Log" option.
Later on, the .har file can be uploaded from the same place to be analyzed in further detail.
Add the .har file as evidence to your test session.