This article aims to explain some Xray Anti-patterns that can lead to future constrains or even data integrity issues. Please use this information carefully and taking into consideration your own setup and team processes.
Anti-Pattern
If you have a huge number of requirements and that are not covered by any Test, this may represent an anti-pattern.
As a general rule, each requirement should be covered at least by a Test.
Solution
It’s important to conduct regular audits of requirements and Test cases to identify gaps in coverage. You can review these audits in team meetings to address any issues.
Xray Data Center
You can make use of the Agile boards and include as a field the "Requirement Status" which would allow you to see at a glance which requirements are covered or not (Figure 1).
Figure 1 - Agile boards
Xray Cloud
Unfortunately, Jira Cloud does not allows us yet to assess the coverage status directly on the Agile boards; however it's possible to track it on the requirement/story issue screen among other places. The built-in gadgets/reports, like Test Coverage Report provide the ability to filter by UNCOVERED requirements.
Collaborative Test Planning
Involve testers early in the requirement gathering and planning stages. It ensures they understand the requirements and can create relevant Test cases proactively.
However, if your organization is set up in a way that developers and testers are not in unison, then we suggest the usage of dashboards and reports to keep track of uncovered requirements in real time.
Reporting
Make use of existing gadgets and existing reports to track your requirement coverage.
Built-in Xray Report | Traceability Report
This report (Figure 2) allows you to see the lifecycle of your requirements and track related Tests and Defects.
Figure 2 - Report
Document Generator Report | Requirement Traceability Matrix
This document based report allows you to see the requirements, their status, and related Tests and Defects.
Figure 3 - Requirement
Gadget | Overall Requirement Coverage
Xray provides a gadget for viewing requirements and evaluating their completeness for a particular release.
- Requirements List gadget for Xray DC
- Example of JQL to use on the the gadget configuration:
"Requirement Status" = UNCOVERED AND createdDate < -7d
Figure 4 - List
Learn More
We understand that company structure or training readiness may not be scalable in times of need. Nonetheless, we suggest you provide comprehensive training for team members on how to use Xray effectively, focusing on requirement coverage and traceability.
Xray Academy
Join for free our Xray Academy and take the Understanding Coverage Course to help you get familiarized with how to have coverage and traceability of your requirements.
Figure 5 - Xray Academy
In fact we encourage your team to go through all the courses → Xray Academy.