Organizing your tests is a crucial part of your daily testing activities. Since your project may have hundreds or thousands of tests, it is imperative that you organize them efficiently so you can find relevant tests quickly.

While many organize using labels, components, priority, etc., they are usually insufficient and would not scale according to your needs.

Xray provides two different but very effective ways of organizing Tests:

  1. using Test Sets issues (this was the only available option before v3.0);
  2. using the Test Repository (since v3.0); 

Xray has a specific issue type for grouping tests: the Test Set. This is a simple way to create different groups of Tests; they're basically flat lists of Tests. You may have as many Test Sets as you wish and a Test may be included in multiple Test Sets. Test Sets are great if you want to have full control over certain groups of Tests. You can easily select them whenever you want (e.g., when creating Test Plans or Test Executions). Test Sets can also be used as a dynamic way to cover requirements.

On the other hand, the Test Repository provides a tree-like organization at the project level. Users can hierarchically organize Tests within folders and sub-folders. This folder concept is common in some tools and resembles user's own files/documention organization in their computer's operating system. More details can be found in Working with the Test Repository.

To decide whether to use one or the other, we recommend that you review some pros and cons detailed in Test Repository. For some clients, the question may be as simple as what their test team prefers: organizing tests in list or hierarchical form?

We also put together some tips that might give you some guidance on how to choose the right approach.




  • No labels