---
title: "Performing Risk-Based Testing (RBT) with Xray"
canonical: "https://docs.getxray.app/space/XRAYCLOUD/329973808/Performing%20Risk-Based%20Testing%20(RBT)%20with%20Xray"
format: markdown
---
> Macro (toc)
# Overview
We can define** **Risk as the “effect of uncertainty on objectives” ([ISO 31000:2018](https://www.iso.org/obp/ui/#iso:std:iso:31000:ed-2:v1:en)) that can be positive (opportunity) or negative (threat). From the QA perspective, proper evaluation of risks is crucial. It allows teams to focus their efforts on the areas that are most likely to cause serious damage to the company’s reputation and bottom line. The broad name for the process of prioritizing and managing tests according to risk factors is risk-based testing (RBT).
> ℹ️ *For more background information, please see our **[Risk Management](https://docs.getxray.app/space/XRAY/301673593)** and **[Risk-Based Testing (RBT)](https://docs.getxray.app/space/XRAY/301476977)** articles.*
In this article we will cover implementing RBT with the help of Xray and Jira Cloud, talking about different levels of risk management and a few important considerations for each.
# Risk Levels and how Xray fits
At a high level, your RBT journey involves the following stages:
- Choose the methodology (just risk score or something like “impact * probability” with risk score calculation; number of dimensions; inherent vs residual risk, etc.)
- *This step is not the focus of this article. The content below prioritizes “risk score directly” and “impact * probability = score” approaches for inherent risk. While nuances will be affected, the general direction of the other three stages would still be relevant if you choose a different methodology.*
- Assign risk metrics to individual work items
- Establish the project management process (including QA) that takes those metrics into account
- Report on that process with specialized or generic templates/visualizations
You could achieve these 4 with just Jira and Xray or add a specialized app (e.g. Risk Manager Plus by SoftComply) or a broader-purpose one (e.g. ScriptRunner by Adaptavist, Jira Misc Custom Fields by Appfire) - many of the overall considerations we cover below are universally applicable.
> ℹ️ *Disclaimer: All examples provided herein, using apps from other vendors, are just informative and need to be evaluated properly for your use cases, considering also performance impacts, among others. We don't provide support for these apps nor for their configuration.*
## Work Item level
At this level, you have two primary decisions: 1) which item type(s) to use; 2) how to apply risk-related metrics.
For the first one, you have more flexibility. Stories/Tasks/Bugs are popular choices but you can create a dedicated “Risk” type or even use the Xray “Test” directly.
You would want to designate risk-related work items (except “Test”) as [Coverable in Xray settings](https://docs.getxray.app/space/XRAYCLOUD/44566845/Project+Settings:+Test+Coverage) - that way you can leverage the “tests/is tested by” linkage to showcase your risk treatment/mitigation activities (via “Test” items) more easily. If you want to display risk metrics on both Coverable and Test items AND those metrics can be copied by Jira Automation, you can set up the rules [similar to this one](https://docs.getxray.app/space/XRAYCLOUD/44565672/Generic+automation+of+Testing+processes#Copy-fields-from-requirement/Story-to-Test-whenever-creating-a-Test-or-linking-it-to-a-story) to minimize manual effort.
For the second decision, it’s important to keep in mind - being accessible through JQL/saved filters in Jira makes the risk metrics more valuable across different use cases. Bonus points if the metrics can also appear as columns in table-based layouts, gadgets, etc. The exact implementation highly depends on the methodology and the 3rd-party choices.
"Number", "Short text (plain text only)", or "Select List (single choice)" field types are popular options. If you use "Short text (plain text only)", note that the sorting will be based on text/string comparison (thus, for example, the text "Medium" will be later than "High"). In this case, you may need to add a prefix in order to guarantee the correct order (e.g. "L2: Medium", "L3: High"). Using "text fields", namely for the risk level, may be tricky as it will make the search and filtering harder; thus, we don't recommend their usage.
On the other hand, "Select List (single choice)" type has well-defined possible values along with a respective order. Such fields are also easier to use during filtering, as you don't have to type their values by hand and you can filter by multiple values at the same time.
For the avoidance of doubt, regardless of the field type, if you decide to pursue the composite approach where multiple metrics combine into a score (e.g. “impact X probability = score”), you would need to figure out a way to calculate the “final” field automatically (3rd-party apps, custom dev work, etc.).
From the Xray perspective, a few considerations that can affect your 2nd decision:
- If the risk-related fields are JQL-eligible, you can use them to select tests for entities like Test Plan, Test Set, and Test Execution. If those fields can be displayed as columns, you can re-rank tests inside those other entities.
- Grouping in e.g. Test Coverage Report **only** supports "Select List (single choice)" and “Checkboxes” types (and does not support 3rd-party fields that are not set to those types explicitly, even if they look similar).
With that said, let’s see a simplified “capstone” example for this stage. Assume we choose Story as our risk-designated work item type. Then, we will showcase 3 options for the metrics - for the avoidance of doubt, you don’t need all 3 implemented at the same time, but it helps us demonstrate different use cases later on.
First, without using any 3rd-part app, we add a manually-created "Select List (single choice)" custom field - “Aggregate Risk Score” (Figure 1-1).
Second, we leverage SoftComply Risk Manager Plus to configure 2 metric groups:
- (Figure 1-2) Risk Values module. Risk Score is calculated automatically based on the defined model. Impact, probability, and risk score can be accessed [via JQL](https://softcomply.atlassian.net/servicedesk/customer/portal/2/article/606246393184282), but do not appear as columns in e.g. Test Execution (since they are not technically custom fields).
- (Figure 1-3) CVSS Score (Text Field) is calculated based on multiple criteria selected within the story’s “CVSS Metrics” section. Can be accessed [via JQL](https://softcomply.com/riskmanagerpluscloud-userguide/#elementor-toc__heading-anchor-6) and does appear as a column.
*Figure 1. Different examples of risk metric implementation for a Story*
## Project level
Once your implementation at the work item level is in place, you can start project-level operations and analysis. Two main aspects that we focus on in this article - bulk management of QA-related entities and reporting on them.
### Bulk management
The starting point is typically grouping Risk entities in Jira’s saved filters using JQL. You can create several filters, each one aggregating work items designated as Risks based on just the score or the combination of factors. Using intuitive names for the saved filters can help you out in the long term across Jira.
> ℹ️ Example for Risk Manager Plus - the query [issueType in (Story, Task) AND risk = “Initial Medium”] becomes the "risks_medium" filter).
Next, for Xray, a couple of helpful use cases can be unlocked (many of them rely on [JQL Functions](https://docs.getxray.app/space/XRAYCLOUD/44565225/Enhanced+Querying+with+JQL), currently in beta):
- Quickly identify tests connected to coverable items with different risk levels
- Example: issuetype = Test AND key in requirementTests(“risks_medium”)
- Easily group tests with the similar risk characteristics when those are applied directly to the Test items (to use in gadgets, reports, etc.)
- Example: the query [issueType = Test AND risk = “Initial Medium”] becomes the "tests_risk_medium" filter
- If the risk metric presents as a column, you can rank tests by it
- In Test Execution or Test Set, sort the column values or manually drag the test to the desired position (make sure the Rank column value is applied)
- In Test Plan, manually drag the test to the desired position
- To bring risk-related columns into the view, you may need to use the “Columns” customization menu (top-right dropdown in the image below)
*Figure 2. Sorting tests in a table layout based on the risk metrics*
- You could also apply the risk metric to the Test Plan, Test Set, or Test Execution directly, to clearly show the nature of tests inside (if Summary/Description/Label are not optimal for your needs)
For defect prioritization, you could again leverage saved filters associated with different risk entities and levels:
- Coverable items - *issue in defectsCreatedForRequirement("risks_medium")*
- Test items - *issue in**** ****defectsCreatedDuringTesting("tests_risk_medium")*
### Reporting
For default Jira gadgets, you can now use the saved filters from the previous stages and/or the eligible risk metrics, for instance in Filter Results or Issue Statistics.
*Figure 3. Examples of Jira gadgets using risk metrics*
If you are using a specialized 3rd-party plugin, you can leverage features like Risk Table or Risk Model matrix for bulk review of Jira work items based on the risk factors.
*Figure 4. Risk Table from SoftComply Risk Manager Plus (you can customize columns and display linked Tests in e.g. Linked Issues for more consolidated insights).*
*Figure 5. Risk Model matrix from SoftComply Risk Manager Plus, listing applicable risk items for each combination of Impact and Probability (or other dimensions you define)*
In Xray reports and gadgets, you can
- group the Coverage analysis by single-choice list fields like our manually-created “Aggregate Risk Score”
*Figure 6. Xray’s Test Coverage Report grouped by Aggregate Risk Score*
- add the column-eligible metrics to e.g. Test Runs List (if you defined risk at the Test level)
*Figure 7. Xray’s Test Runs List report with added columns for Aggregate Risk Score and CVSS score*
- limit the scope of gadgets using saved filters. For most gadgets, the grouping mechanism is not available, so you need to create multiple instances of them, configuring each one with a different filter based on the certain risk level.