---
title: "Reporting and Tracking Defects"
canonical: "https://docs.getxray.app/space/XRAYCLOUD/1027735560/Reporting%20and%20Tracking%20Defects"
format: markdown
---
> Macro (rw-ui-expands-macro)
> 
> > Macro (rw-expand)
> 
> > Macro (toc)

# Concepts

## What is a Defect?

[Defects](https://getxraydocs.atlassian.net/wiki/spaces/XRAYCLOUD/pages/44565860), sometimes also referred to as “bugs”, are issues (e.g., actual bugs, vulnerabilities, gaps, inefficiencies) that affect the perceived value by stakeholders; in other words, they affect different aspects of quality (i.e., quality criteria) as seen by different team members, business, and end users.

## Could We Simply Report Test Failures?

Isn’t it enough to mark a [Test Run](https://getxraydocs.atlassian.net/wiki/spaces/XRAYCLOUD/pages/44565109) as failed? Why report Defects?

Let’s see how these two are different and actually serve slightly different purposes:

- Reporting a Test Run as **failed** represents the result of performing a check over an expectation that is materialized, usually in a [step of a Test](https://getxraydocs.atlassian.net/wiki/spaces/XRAYCLOUD/pages/44565316); this check can be accurate or a false positive.
- Reporting a **Defect** is based on human judgment, after marking a Test Run as failed and deciding that there is an actual Defect that needs to be handled properly.
- Defects, as Jira Work items, are more visible and can be estimated, planned, added to a sprint, and made part of the release Work items; those actions can’t be performed with just failed Test Runs.

Therefore, reporting Test failures and reporting Defects is not the same; the latter complements the former and provides mechanisms to make sure fixes are addressed properly by the team.

## Defect Reporting

### Reporting Defects as the Foundation to Improve Quality 

100% perfect or 100% bug-free software doesn’t exist. Defects are common and a natural consequence of building software.

However, we should aim to:

- Prevent Defects as much as possible.
- Fix important Defects quickly and “safely” (i.e., without impacting other quality aspects or introducing additional Defects).

Before fixing Defects, we must be aware of them first. And that’s where Defect reporting comes into the picture, so that we have a clear view of them, understand their impacts, choose how to deal with them, and eventually plan and fix them.

### Why Is Basic Defect Reporting Not Enough?

Some teams raise Defects by simply creating bugs/issues in their project management tool, like creating Bug Work items in Jira.

What they usually do (eventually all or a subset of it):

- Highlight in which version, build, and environment where the problem was found.
- Describe the problem found via the *Description* field of the raised issue.
- Describe the steps to reproduce the Defects.
- Attach screenshots and/or logs, or evidence in general.

### Where Basic Defect Reporting Falls Short

- **Lack of traceability**: how does it relate to business requirements specified in the project management tool (i.e., traceability)? What is actually impacted?
- **Brittle reporting is prone to errors**: since the user details the problem at their own discretion, some details may be forgotten or contain errors, including the context where the Defect was found.
- **Escaped Defects cause confusion**: no clear distinction between Defects found during Testing activities *versus* Defects found in production, which complicates the analysis of how many Defects escape to production and how effective the team is being on building quality.

> ✅ Xray covers all these needs.

### Questions We Aim to Answer with Proper Defect Reporting

- *What were the Defects opened for a given version?*
- *And on a given environment/device/browser?*
- *What is the impact of the identified Defects found on the features we have?*
- *In what exact context were the Defects found, so that we can replicate/retest them?*
- *If executing a manual Test scenario, exactly when (in which step) was the Defect found, so we can retest it later on?*

# Defects in Xray

## A Defect - from Xray’s Perspective

Let’s start by clarifying what we mean by Defects. Defects can be any Work item type that is [configured](https://docs.getxray.app/space/XRAYCLOUD/44568504/Project+Settings:+Defect+Mapping) to be handled as such. 

Usually, teams configure the Bug Work item type, which is often available by default in the software Space templates in Jira. Nevertheless, any Work item type, including custom ones, can be considered a “Defect”. 

Users can take advantage of this flexibility to distinguish functional bugs from severe problems from security vulnerabilities, for example.

> ℹ️ There’s not a specific Defect Work item type created by Xray; instead, Xray provides the flexibility so that users can define different Work item types as a way to semantically distinguish different kinds of Defects that may have their own workflows.

## Finding Defects

There are two scenarios:

- **Defects found while Testing,** during the development lifecycle => reported and managed explicitly by Xray:
  - Associated with Test Runs, while executing a Test in the scope of a [Test Execution](https://getxraydocs.atlassian.net/wiki/spaces/XRAYCLOUD/pages/44565117).
  - Can be associated with a given [Test Step](https://getxraydocs.atlassian.net/wiki/spaces/XRAYCLOUD/pages/44565316) (in case of manual/step-based Tests) or globally to the [Test Run](https://getxraydocs.atlassian.net/wiki/spaces/XRAYCLOUD/pages/44565109) (in case it’s a side Defect, not explicitly found as part of a given step expectation).
  - Eventually, explicitly linked to the related Test, Test Execution, and Requirement.
- **Defects found in production**, reported by end users (i.e., escaped defects) => reported using Jira but trackable in Xray:
  - Usually reported directly (i.e., Work items created directly in Jira, as any other Work item).
  - No association whatsoever to Test Runs/Tests.
  - Most times, no association to an existing Requirement.

## Reporting Defects Using Xray

Defects in Xray are always reported while [running (i.e., reporting the results for) a Test](https://docs.getxray.app/space/XRAYCLOUD/44565122/Execute+Tests). This way, defects will be clearly associated with the:

- Test that was being run.
- Requirement(s) and parent Requirements, if any.
- Context of the product being tested (i.e., the release, the environment, the build/revision), through the Test Execution attributes (Fix Version, Test Environment, Revision).

On the execution screen of the Test Run, where testers report the result for a given Test as part of the respective Test Execution, testers can:

- Add existing Defects (i.e., associate existing Defects to the Test Run).
- Create Defects (i.e., create a Defect Work item and associate it with the Test Run).
- Create sub-tasks (i.e., a Defect created as a sub-task of a parent Work item).

> ℹ️ Creating Defects as sub-tasks is usually part of a strategy, during a development cycle, where teams opt for creating Defects as sub-tasks of the Requirements/stories. Why? To implement rules and even automate them, where the team can’t close Requirements until all the related Defects (sub-tasks) are closed.

![Figure 1 - Xray defects](media://07573b52-98b1-41d4-9938-252f07404be4)

Defects reported using Xray:

- Will have the *Affects Versions* field filled based on the Test Execution’s *FixVersion* field.
- Can inherit labels and components from corresponding Test Work item, Test Execution Work item, or both.

For even more explicit traceability and easier navigation, [you can configure Xray](https://docs.getxray.app/space/XRAYCLOUD/44566331/Global+Settings:+Miscellaneous#Defect-Links-Options) to create a link between the Defect and the corresponding:

- Test.
- Test Execution.
- Requirement(s).

## Tracking Defects

### Tracking Defects Using Reports

[Xray provides several reports](https://getxraydocs.atlassian.net/wiki/spaces/XRAYCLOUD/pages/338526916) where Defects are visible, namely on the two traceability reports. Additionally, for Test Executions and Test Plans, you can track global counters of open/closed Defects. 

#### [Requirement Traceability Report](https://docs.getxray.app/space/XRAYCLOUD/44565208)

Tracks all types of Defects, including the ones found while Testing and the ones reported by end users (Figure 2).

![Figure 2 - Requirement traceability report](media://835b6af8-f3eb-4c65-8e74-2beef07e624e)

#### [Defect Traceability Report](https://docs.getxray.app/space/XRAYCLOUD/338363920/Defect+Traceability+Report)

Tracks all types of Defects (Figure 3), including the ones found while Testing and the ones reported by end users (based on the Defect Mapping setting in Xray).

![Figure 3 - Defect traceability report](media://60ac6cf6-ef97-4920-b809-e3143fe6cae5)

#### [Test Executions List Report](https://docs.getxray.app/space/XRAYCLOUD/44565202/Test+Executions+List+Report)

See open/closed Defects reported while executing Tests in the scope of each Test Execution (Figure 4).

![Figure 4 - Test executions list report](media://b6e30d53-72f3-4010-83d4-dbde7dde506a)


#### [Test Plans Metrics Report](https://docs.getxray.app/space/XRAYCLOUD/44565191/Test+Plans+Metrics+Report)

See open/closed Defects reported while executing tests in the scope of each Test Plan (i.e., on the associated Test Executions; Figure 5 - 1).

![Figure 5 - Test plans metrics report](media://05d626d9-e1f3-4d46-9b33-4ea297c5e288)


### Tracking and Sharing Defects Using Dashboards

With Xray you can take advantage of Jira’s native gadgets, such as the Filter Results gadget (Figure 6), to list Defects, for example. For that, you can make use of some helpful [JQL functions](https://docs.getxray.app/space/XRAYCLOUD/44565225/Enhanced+Querying+with+JQL) such as:

- `defectsCreatedDuringTesting()`
- `defectsCreatedDuringTestExecution()`
- `defectsCreatedForRequirement()`

These JQL functions allow saved filters as arguments, which can be quite handy in case we aim, for example, to list all Defects found while executing Tests on a certain Test Execution, like the ones assigned to a given release and/or environment.

![Figure 6 - Filter results](media://a1060c23-6220-4cdd-ad5b-1172e6cb7848)

# Additional Tips

- Defects like Work item types (e.g., Bug) can be configured as coverable Work items. Thus, they may also be covered with Tests similarly to user stories. Some teams prefer this approach to make it more visible that Defects need to be tested and represent a “requirement” that needs to be addressed.
- It’s possible to combine coverage information with Defect reporting, so we can try to look at Test Runs that failed but don’t have reported Defects. A similar approach can be done for requirements, looking at the ones considered to be NOK from a coverage perspective, but that don’t have Defects (we can use a mix of JQL functions, including `requirements(`) and `defectsCreatedForRequirement()`).

# References

- [”Execute Tests” article from Xray Cloud documentation](https://docs.getxray.app/space/XRAYCLOUD/44565122/Execute+Tests)
- [https://docs.getxray.app/space/XRAYCLOUD/44568504/Project+Settings:+Defect+Mapping](https://docs.getxray.app/space/XRAYCLOUD/44568504/Project+Settings:+Defect+Mapping)
- [https://docs.getxray.app/space/XRAYCLOUD/44566331/Global+Settings:+Miscellaneous#Defect-Links-Options](https://docs.getxray.app/space/XRAYCLOUD/44566331/Global+Settings:+Miscellaneous#Defect-Links-Options)
- [Requirement Traceability Report](https://docs.getxray.app/space/XRAYCLOUD/44565208)
- [Defect Traceability Report](https://docs.getxray.app/space/XRAYCLOUD/338363920/Defect+Traceability+Report)
- [Test Executions List Report](https://docs.getxray.app/space/XRAYCLOUD/44565202/Test+Executions+List+Report)
- [Test Plans Metrics Report](https://docs.getxray.app/space/XRAYCLOUD/44565191/Test+Plans+Metrics+Report)
- [https://docs.getxray.app/space/XRAYCLOUD/44565225/Enhanced+Querying+with+JQL](https://docs.getxray.app/space/XRAYCLOUD/44565225/Enhanced+Querying+with+JQL)
- [https://docs.getxray.app/space/XRAY/301473949/Retesting+with+Xray](https://docs.getxray.app/space/XRAY/301473949/Retesting+with+Xray)


> Macro (rw-ui-expands-macro)
> 
> > Macro (rw-expand)
> 
> If you have questions or technical issues, please [contact the Support team via the Customer Portal (Jira service management)](https://jira.getxray.app/servicedesk/customer/portal/2/user/login?destination=portal%2F2%2Fcreate%2F28) or [send us a message using the in-app chat](https://getxraydocs.atlassian.net/wiki/spaces/XRAYCLOUD/pages/44577312).