What is a risk?

A risk is something that can impact (negatively or positively) the value of what we're aiming to provide.

...

Testing is many times seen as a risk mitigation strategy, but it's also a strategy of preventing risks, depending on who and when is envolved while testing and the decisions that are taken upon the findings of testing.

Using risks to drive testing

We use testing to obtain quality related information. Risks and quality have a close relation. Risks can affect the value as seen by some stakeholder, at a given moment. In other words, risks can impact quality.If we implement a substantial feature without listening to users or having made some preliminary research, then there's a risk of that feature not addressing the needs of our users. If we use external libraries, then there's a maintenance and dependency risk that can expose us to more problems ahead.

...

1. Think about who are your users, external and internal. Don't forget the unexpected users;
2. Think about what matters most to them;
3. Think about what could go wrong.

Contextualizing risks

There is no straightforward  and ordered "checklist" of the risks we should tackle. That requires expertise from tester and... context!

It's impossible to tackle all risks; it's impossible to test "everything". Therefore, whenever testing we have to think about where we will invest our effort in, so that we cover aspects that can give us valuable information about quality.

Contextualizing risks with time

Software and the overall development process is not a localized event; its  a journey. It has a past, a present, and a future.

...

1. How was it used in the past?

   • Info
     title Why?
     It gives clues about, among other: features that users were using (and not using) and to what extent flows that users were performing, and not performing

2. How is it currently used?

   • Info
     title Why?
     It gives clues about, among other: features that users are using (and not using) and to what extent flows that users are performing, and not performing if there was a change in user behaviour and underlying needs if there was a change in the software that may have affected the current usage behaviour

3. How do we foresee it being used and evolving in the future?

   • Info
     title Why?
     It gives clues about, among other: concerns about performance and scaling how to monitor/track success how to quickly perform experiments with users existing features that may need to be rethought or that may be affected somehow, and thus may need tailored testing

Contextualizing risks with needs

Software is made to address needs. Needs can be of different types though; they're not always "functional". What is the purpose that we're trying to achieve and to whom? Are there any existing references we should have in mind?

...

But needs exist not only for external users but also for internal stakeholders and even for the team supporting and developing the product. Are we using deprecated dependencies or dependencies will well-known security issues, for example? Can a component or a service provider easily be replaced by another one? Is our infrastructure properly tracked, is it's setup scriptable, and are those scripts prone to error handling, for example?

Contextualizing risks with current practices

Is testing currently just implemented at the surface? Are areas/features covered with automated test scripts? What level and type of tests (e.g., unit, integration, system, functional, performance, security)?

...

What are we covering already? To what extent? Do we have quick feedback loops about it? Do we have an history about it?

Remember the unknowns

The risks that we are aware of, we can handle somehow, including with more or less testing depth.

...

• We can check the risks we know we know
• We can check & explore the risks we know we don't know
• We can pair with others to tackle unknown knowns, and expose our biases or things we assume or skip without knowing
• We can keep learning and exploring to uncover unknown unknowns

From risks to test charters

Say we have selected a risk.. how can we turn into a test charter?

...

Our test charters should be around maximizing probability in one side but also consider situations that have a relatively low probability but still have a major impact.

A few additional tips

• Internals
  • listen to your team, pair with other team members including developers to expose risks that otherwise could escape
• Product Context & Market
  • listen to your users, to understand what's important to them, what they most value, and the things that frustrate them the most; these will give ideas of potential risks
  • listen to your business and what's important to them, to be on the same page and don't ignore aspects that ultimately matter to your company and management
  • experiment similar products, as you'll gain knowledge about what common and sometimes implicit expectations users may have whenever using your own product
• Success
  • try to understand what "success" means to different stakeholders
  • understand "where the money comes from" vs "what are the common user/usage flows"
• Background Knowledge
  • learn more about quality attributes, to become aware of different aspects that people value differently; these are the different dimensions of quality that we can have in the back of our mind, that inform us about what quality aspects can be targeted by risks
  • learn more about heuristics tailored for testing, as these can be used to provide some ideas for test charters but also ideas for many diverse experiments during the test session itself

...