--- title: "Tips for Organizing Tests" canonical: "https://docs.getxray.app/space/XRAYCLOUD/44565297/Tips%20for%20Organizing%20Tests" format: markdown --- > Macro (rw-ui-expands-macro) > > > Macro (rw-expand) > > > Macro (toc) > ❌ This page will be archived soon. > ❌ > ❌ Its content has been consolidated into a [new page](https://getxraydocs.atlassian.net/wiki/spaces/XRAYCLOUD/pages/659259394). # Overview Properly organizing Tests is a core activity of effective Test management, as it enables quick retrieval of the Tests that matter for some context (e.g., for a new release, for some explicit validation). After finding the Tests, users use those to create [Test Plans](https://getxraydocs.atlassian.net/wiki/spaces/XRAYCLOUD/pages/44565153), or schedule Test Executions, for example. At a high level, Xray provides two ways to organize Tests: using flat lists, also known as [Test Sets](https://getxraydocs.atlassian.net/wiki/spaces/XRAYCLOUD/pages/44565163), or using folders within a hierarchical structure (the Test Repository). **In general, the ****[Test Repository](https://getxraydocs.atlassian.net/wiki/spaces/XRAYCLOUD/pages/44565166)**** scales better and provides easier filtering, finding, and management, namely organization, of Tests.** To be able to organize the tests well, first of all, we need to “prepare them”; in other words, follow some rules that will enable better and systematic findings later on, and proper organization. # Organization ## What Relevant Information Can we Have on Tests? We have to think about the Test’s **purpose (i.e., the “why”)**, **how** we actually achieve it, **where**, and **what** quality aspect we’re looking at. - Type of Test (e.g., functional, non-functional). - Level of Test (e.g., unit, integration, API, e2e). - Type of Test scenario (positive, negative). - Priority. - Covered feature (e.g., story, epic). - Quality attribute being tested (e.g., correctness, performance, security). - Attributes (e.g., labels). - Component/module. ## The Perfect Organization There is no “one size fits all” type of organization. Some teams and some team members may prefer to organize Test cases differently, and that’s fine - as long as it works for them and other team members. Just remember that if you’re using [Test Repository](https://getxraydocs.atlassian.net/wiki/spaces/XRAYCLOUD/pages/44565166), it’s a project-level organization of Test cases (i.e., the only one, yet shared, structured folder organization for the Test cases of that project) and thus affects all project users. ## Meet the Test Repository The Test Repository organization follows a hierarchical, tree-like way of organizing Tests within folders. Since a Test can only be part of one folder within the Test Repository, you have to think in a hierarchical way: "what is more important? How do I look for the Tests to schedule executions?" > ℹ️ - A given Test can only be within one folder; we cannot add the same Test to multiple folders. > ℹ️ - It’s possible to filter Tests based on Test attributes, so we can then use that selection to do other operations with those Tests. ## Preparing Test Cases for Organization Start by enabling **discoverability**, adding proper semantics to your Tests, using the most adequate fields: - *labels* to tag your Tests (or your Test Sets); if you start having many testers and your project grows, having a well-defined list of labels may help. - The *priority* field so you can distinguish between Tests. - Assign the Test to the proper *component.* - The meaningful *Test types.* > ⚠️ Don’t add the previous semantics to the Test summary field, as that makes the summary longer, harder to filter; use dedicated custom fields for that. ## Strategies to Organize Tests In general, while using the Test Repository we can create folders and sub-folders by: - Features - Component/module + whole system. - User journeys. - Initiatives > Epics > Stories. - APIs - Quality criteria (e.g., performance, security). A mix of the previous strategies is common and may also work. An example of it could be: - System - Components: - Component A. - Component N. - Features: - Subscription. - Payments. - Recommendation system. - Admin. - APIs. - Performance. - Security. - Smoke Tests. ## Regression Testing Picking the Tests used for regression Testing is a common tester activity. If we create a folder (e.g., “regression”) in the Test Repository, would that work? Before answering it, let’s think about what regression Testing is all about. Regression Testing is about checking for regressions on features that previously worked fine (i.e., testing already existing features). That means that we usually run Tests that cover features from previous versions; we don’t run all of them; usually, we pick a set of them for time constraint reasons. Therefore, these are not “new Tests” made explicitly for regression Testing; they are existing Tests covering existing requirements. As a consequence of this, we wouldn’t be able to have these tests in both the “regression” folder and some other folder at the same time. Usually, we would manage the regression Testing effort as part of a dedicated Test Plan, or even a folder inside the [Test Plan Board](https://getxraydocs.atlassian.net/wiki/spaces/XRAYCLOUD/pages/44565285). However, the question arises: which is the source for selecting the Tests for regression testing? We could reuse an existing Test Plan, clone it, and adapt it. But that way seems to indicate that we don’t have a proper place that acts as a consistent source of truth for highlighting the Test cases. These are some possible strategies to identify the Tests for regression testing: - Whenever creating a Test Plan, or a Test Execution, for regression Testing, whenever adding the Tests, we can pick Tests that cover requirements implemented on previous/specific versions (Figure 1). ![Figure 1 - Versions.png](media://fb15fafc-981a-4af3-bb66-4abc4259653c) - Using the previous strategy, we can create another Test Plan/[Test Execution](https://getxraydocs.atlassian.net/wiki/spaces/XRAYCLOUD/pages/44565117) for doing regression Testing in another version, cloning the previous one, and adding the Tests of the previous version. - We can also use a Test Set for regression Testing and eventually have regression Test Sets, one per version; managing can become harder if we decide to remove/add Tests at a certain point in time to some requirements, as we would then need to go to all those Test Sets and update them accordingly. - Use a dynamic Test Plan, creating a [JQL](https://getxraydocs.atlassian.net/wiki/spaces/XRAYCLOUD/pages/44565225) expression: - Based on Tests having the label “regression” - Based on Tests covering requirements on previous versions: ## Tips and Anti-Patterns These are some recommended good practices that are known to work well: - Create top-level folders for the things that you value the most whenever searching Tests, such as their goal (e.g., "performance", "security", "smoke Tests"). - Create the hierarchy from the most relevant things, and then decompose each one; for example, if your Tests are the "components" of the system, you can create a top-level "components" folder and sub-folders for each sub-component. - Don’t try to over-organize; even though we could organize Tests on many multiple levels based on all sorts of criteria, it’s impossible to organize Tests from all possible angles and even if it were, it would cause too much clutter. - Focus on **enabling discoverability** by using fields such as *priority*, *component*, and *labels*, so you can find the Tests you may need at a given point in time, no matter how they’re organized within the Test Repository. - Define a process with your team to follow similar rules in terms of Test classification and organization. - Deprecate Tests whenever appropriate, using a workflow status, for example, to be able to quickly exclude those Tests (learn more about *[Using Workflows for Testing Purposes](https://getxraydocs.atlassian.net/wiki/spaces/XRAYCLOUD/pages/44567832)*). - Avoid duplicating Tests (e.g., to enable having the “same” Tests in multiple folders) as that will impact maintenance of Tests and add confusion. Some usage patterns can work for some time, but that can cause some issues in the future. Therefore: - Avoid creating folders for versions of the SUT (e.g., v1, v2, v3): - Avoid mixing execution or planning-related aspects in the Test Repository, as the same Tests may apply to v1 and v2 of the SUT, for example. - Don't organize the Tests based on temporary attributes. # References (Internal Use Only) - [https://medium.com/@case_lab/best-practices-for-categorization-of-test-cases-in-software-testing-9a2e560e3d3c](https://medium.com/@case_lab/best-practices-for-categorization-of-test-cases-in-software-testing-9a2e560e3d3c)
Support/Troubleshooting If you have questions or technical issues, please [contact the Support team via the Customer Portal (Jira service management)](https://jira.getxray.app/servicedesk/customer/portal/2/user/login?destination=portal%2F2%2Fcreate%2F28) or [send us a message using the in-app chat](https://getxraydocs.atlassian.net/wiki/spaces/XRAYCLOUD/pages/44577312).