Page History
...
One of the primary points of being Agile means having a tool that adapts to your team's needs. This document sums up the different possibilities you have in working with Xray in an agile context and provides some guidance in getting started. Of course, we always suggest you evaluate what best fits your team's needs.
In order to have a successful Agile process, you need to have these processes/mindsets in place:
- immediate information/no handoffs: Xray provides immediate information visibility right within your work item. This way, you don't have to ask someone to know about results, testing progress, or what is the coverage status of an Epic/Story. For example, you can see how a user story is from a QA standpoint, right from the Story issue screen; likewise, you can see QA feedback in your existing Scrum Boards
- collaboration: everyone is invited to collaborate, no matter their role; therefore, anyone can interact and collaborate on user stories, test cases, test planning, test execution. This is because Xray works inside Jira and uses different issue types as an abstraction of its own entities, meaning you can collaborate inside Jira, the Jira way
...
| Info | ||
|---|---|---|
| ||
As Xray is a flexible tool, and every team has its own demands, we highly recommend to start simple and explore some of the possibilities offered by the tool. Following the Agile principles, your processes should be reviewed often by the team and adapted accordingly with the team's needs. You should also not dictate long and very strict processes, as you want to promote "individuals and interactions over processes and tools". Therefore, you need to find the right balance. |
...
- review the user stories with your team (remember: testers are part of the team)
- create Tests for each user story
- organize the Tests
- create one or more Test Plans (see above) and add the relevant Tests
- schedule multiple Test Executions, from the Test Plan or directly from each Story as Sub-Test Executions
- track overall, consolidated progress at the Test Plan level
- use reports to assess additional information, such as...
- Overall Coverage Report, for evaluating the current coverage status of the coverable entities (e.g. user stories)
- Traceability Report, to quickly filter out relevant entities based on their coverage, check the runs of related Tests and reported defects
- Test Executions Report and/or Test Plans Report, as a means to see the results obtained on different environments, on different revisions of the SUT, and to check the reported defects and defects if they're still to be resolved or not
- use Jira dashboards to share information about your project, including testing progress; use some specific gadgets for the that later
What belongs to a release?
Tests, Pre-Conditions and Test Sets are reusable entities; thus, in typical scenarios, they do not belong to the scope of a release , or of a sprint.
In On the other hand, Test Executions, Sub-Test Executions and Test Plans represent work targeted for a given release and/or a given sprint. Thus, they can be tracked as any other issue that needs to be handled in the scope of your release/sprint. If you have these issues assigned to some release through the FixVersion/s field, then they will appear in the summary information of the Releases project page.
...
| Info | ||
|---|---|---|
| ||
As of Xray v3.4, Tests created from the "requirement"/story screen will inherit the FixVersion/s from the parent issue. The version assigned to the Test case has no special meaning , unless you want to handle it as specification work that you need to address in the context of that release. If that's not the case, you can simply ignore. (Xray may change this behaviour in the future.) |
Coverage: Epics and Stories
...
That means that Tests covering a given Story will implicitly cover the related Epic. Thus, from the Epic screen, you can track its coverage, including the latest Test results , based on the coverage of related Story issues.
...
Addressing the Tests directly related with some sprint (i.e. with the stories or any other entity tested and assigned to the scope of the sprint) is relatively easy: you just need to link the Tests to the stories and you . You don't even need to organize the Tests in any special, structured way.
...
Thus, you can add fields and/or labels as a means to clearly categorize them, so that you can find them later on using standard issue search mechanisms, including JQL functions.
But there are more structured approaches to keep your test cases organization sane and manageable.
Overall, you have these organization organizational possibilities:
- Implicit, using labels and/or additional custom fields;
- In flat lists called "Test Sets"; a Test Set is implemented as an issue, thus you can add additional fields or labels and search for them using JQL;
- Within folders and sub-folders, inside the Test Repository of your project;
- Using the Structure app, if you need more complex organization scenarios (e.g. if you need to present Test cases and other entities in the same visual representation)
Thus, which Which approach should you use?
There is no single answer that can suffice all sorts of team needs; the best is trying out the different approaches and choose choosing the one that makes your team more "comfortable" with and effective. If your team loves JIRA and is keen on JQL, they'll probably prefer using flat lists (i.e. Test Sets). In On the other hand, if your team has no deep Jira knowledge, or if they're are coming from legacy tools, or if they aim to manage thousands of Tests per project, then the hierarchical organization provided by the Test Repository may be the best approach.
In big enterprise scenarios, following SAFe or similar frameworks for "Agile at scaleScale", Structure app may cover more complex needs; please check the Test Repository approach first and go with Structure only if you need more advanced scenarios where you want to show Tests alongside other entities.
...
The list of Tests you wish to run for regression testing may be quite dynamic and change over time, or it can be quite static; it all depends on your internal process.
...
If you wish to maintain an explicit and well-defined list of Tests you wish to be used for regression testing, you can have a Test Set with those Tests. You can build this Test Set by searching Tests based on their attributes/fields, based on other existent Test Sets that group Tests by some criteria, or even based on some of the Test Repository folders. Maintaining this Test Set up-to-date may require some work.
...
During the lifetime of a sprint, you will perform testing as a way to ensure you have a shippable product. Therefore, you'll need at least one Test Plan as a means to identify what you want to test and also to track its results.
...
Thus, you can simply have one Test Plan per sprint. However, you may have more than one if you wish to manage & and track independently different subsets of Tests.
...
The Test Execution, which represents a task to run a bunch of tests against a certain version+revision of the project (e.g. SUT, AUT, HUT), can be assigned explicitly to the sprint or not. By assigning it to the sprint, you can manage it as an artifact/work that needs to be done in the scope of the sprint. Similar to other issues being addressed within the sprint, you can take advantage of the workflow status to track its progresprogress, from a work point-of-view.
...