Page History
Warning | ||
---|---|---|
| ||
Please note that the articles in this documentation feature media from app version 2.0.5. Interface elements may differ in newer versions. Refer to the latest app version for the most accurate information. |
Widget Connector | ||
---|---|---|
|
Table of Contents |
---|
Overview
While performing the test sessions, testers collect and organize information so they can share it with the stakeholders or with the internal team.
In this article, we'll cover the essentials of collecting data that can be turned into actionable information, so that we can improve the product or system we're building.
Findings and evidence
To share findings & evidence properly, we need to think about our goal: why are we doing it?
- To ease the replication of a certain situation or bug
- To make RCA (Root-Cause Analysis) easier, and get to the source of the problem
Perhaps we're collecting information for ourselves, or maybe we're thinking about reporting defects so the developers can have a look at them. Maybe we have an idea to share with the PO or with the team for further discussion. Maybe something really works well, and the team needs to know that. Knowing to whom we're collecting information is relevant as we may need tailored information for them.
Let's see a brief summary of information that we can collect during exploratory testing and how it can be organized.
Text notes
Notes in the form of text are the most common way of storing our findings. These notes can be of different natures.
Testers use labels, or categories to distinguish text notes, to make them easier to find afterward.
Next are some examples of possible labels/categories that are frequently used, along with their meaning:
- Concern
- Something that represents a potential, negative risk that can somehow affect value, now, or in the future.
- Problem
- A clear problem that impacts value to some stakeholders (internal or external).
- Question
- Something that is not clear; may or not affect value.
- Suggestion
- A recommendation/suggestion to change some existing aspect/feature.
- Idea
- Some ideas to make the product better and thus improve it, usually by recommending new features. Eventually, you may merge "Suggestion" with "Idea."
- Praise/Positive
- Some positive aspects we have found and that we want to share with the stakeholders, usually the internal team.
- Comment
- A general note/comment.
Evidence and audio recordings
The evidence consists mainly of files that we either extract from the application (e.g., logs, other artifacts) or that are strongly connected to the system (e.g., screenshots, videos).
Audio recordings, usually taken more as a verbalized text note, can also be included in this section as they will be stored as a file that we can upload as evidence.
Evidence is like facts that are of great help later on, for example, for bug analysis and fixing.
In sum, we can have the following:
- screenshots
- video/screen recordings
- audio recordings
- logs
- application logs
- JS console logs
- crash dumps
- ...
Tips for collecting and organizing information
Text notes
Assess the different categories you want to have to mark information
Start with a few categories (e.g., Problem, Question, Idea, Praise) and adapt to your needs.
Organize text notes using mindmaps
- some users prefer a more visual and hierarchical way of organizing findings; mindmaps may work well, either using tools such as Miro, Mindmup, Xmind, or even the free Freemind tool
- mindmaps allow a totally flexible yet non-standard way to organize way notes; this may create some challenges in navigating through different test session results and even searching for what matters in a given session
In the end, add the mindmap or a link to it, as evidence or as a note in your test session.
Focus on the problem and not on the individual
Whenever taking text notes, avoid mentioning someone as the cause of the problem. Focus instead on technical facts/evidence. It's ok to mention some decisions that may have been taken in the past as it can be connected to the problem and may also point to other related impacts.
Screenshots
Use visual annotations to highlight relevant areas to make explanations clearer
Screenshots are quite valuable as they make information visible, which would be hard to put into words. However, we often want to refer to a certain part of the application/site. By drawing around what we want, we can make it clearer for review later on.
In the Xray Exploratory App, it is possible to draw squares, circles, or just freestyle drawings on top of the screenshot.
Combine screenshots with categorized notes
Take a screenshot and add some text, marking it as a Problem, Idea, or other. This will provide more context to the screenshot you're taking.
In the app, this is done at the same time you take the screenshot, right after the screenshot, at the bottom.
Record only what matters
To provide greater focus and avoid some noise, record just the browser, or the application's UI. Record the whole screen if it makes more sense for your use case (e.g., the interaction of a desktop application with other applications).
In the Xray Exploratory App, the selection of what to record (i.e., the desktop, a specific application) whenever performing video recordings, or screenshots, is done at the start of the testing session. It can also be performed after resuming a paused session.
Video/screen recordings
Record only what matters
To provide greater focus and avoid some noise, record just the browser, or the application's UI. Record the whole screen if it makes more sense for your use case (e.g., the interaction of a desktop application with other applications).
In the app, the selection of what to record (i.e., the desktop, a specific application) whenever performing video recordings, or screenshots, is done at the start of the testing session. It can also be performed after resuming a paused session.
Audio recordings
Use audio notes
If you prefer to verbalize while testing, then record audio notes and review them later.
In the app, you can easily take audio nodes.
Combine screen recordings with audio notes
Combine screen recording with audio notes, so you can stay even more focused on testing.
In the app's global settings, you can enable "Audio on screen capture."
Evidence
Upload evidence and add categorized notes to provide better context
Upload some files as evidence and add some text, marking it as a Problem, Idea, or other. This will provide more context to the screenshot you're taking.
his is done at the same time you upload evidence, at the bottom.
If using the browser, record the traffic to a .har file
Record the browser network traffic and upload it to XEA. In Chrome, this can be found in "Developer Tools > Network"; make sure to select "Preserve Log" option.
Later on, the .har file can be uploaded from the same place to be analyzed in further detail.
Add the .har file as standard evidence to your test session. Switch to the "Timeline" view and then drag the file to XEA.
General
Sequence matters, so record it
Problems often occur due to having the target system/application in a certain state, which in turn is the consequence of a sequence of steps. To replicate problems/situations, we need to remember the sequence of actions performed.