You are viewing an old version of this page. View the current version.

Compare with Current View Page History

Version 1 Next »

Why it Matters

Enterprise teams generate a massive volume of test artifacts, automated results, screenshots, logs, and evidence. Without a clear storage strategy, even the most powerful Xray instances can become harder to manage over time. But Storage Management Practices are not just a technical concern, it’s a reflection of your process maturity, tooling alignment, and governance culture. This guide goes beyond limits. It offers real-world storage strategies tailored to enterprise testing realities.

Choose Your Storage Pattern

Decentralized, Unlimited Approach

  • Profile: Large enterprises with unlimited storage who prioritize autonomy over control.
  • Behavior: No file type or size restrictions. Teams manage their own test execution and reporting without enforced templates.
  • Benefits: High flexibility, fast enablement.
  • Risks: Uncontrolled growth of test artifacts, Inconsistent practices across teams, Breach of compliance requirements.

While Xray Enterprise offers unlimited storage, maintaining performance in Jira environments still benefits from healthy attachment practices and routine housekeeping. Here’s what you can do:

  1. Encourage Routine Test Run Housekeeping
    1. Establish a retention policy that defines how long Test Executions should be kept based on criticality, audit requirements, and team workflows. Once in place, periodically review and delete older Test Executions that exceed the defined retention period.
  2. Monitor Storage-Heavy Projects
    1. Identify projects where attachment usage is high.
    2. Leverage the Global Storage Settings to identify the projects both within the tool or through our available download options. Learn more about what you can do in our documentation.
    3. Consider follow-ups with teams for cleanup or optimization.
  3. Optimize Evidence Handling in Automation
    1. Validate imported results when using the REST API. Ensure only relevant test executions, statuses, and evidence are submitted. Avoid importing excessive or redundant data such as logs, large screenshots, or debug outputs that are not required for traceability or audit purposes.
Remember that you can and should use your CI/CD tool to track your build logs in full detail, so try to keep verbose information there. You can always link the build to the Test Execution issues, using a custom field on it, to ensure full traceability between Jira entities and build information.

Governance Visibility → Storage Control at your fingertips with the tutorial we’ve made for you!



General Guidance for a decentralized approach in Xray

Scenario

Approach

Operate in a high-speed, autonomous environment

Letting teams manage their own evidence formats, but guiding via optional best practices

Run frequent automated tests with large volumes of results

Define which types of evidence are required for long-term traceability (regulatory validations), and apply lighter evidence standards to low-risk, routine automation runs to avoid unnecessary storage growth.

Externalized Evidence Strategy

  • Profile: Automation-heavy teams integrating CI/CD pipelines with Xray.
  • Behavior: Evidence (logs, videos, reports) is stored outside Xray, often in cloud buckets or CI platforms. Xray execution comments are used to paste the artifact URL.
  • Benefits: Minimal Xray storage usage. Enables traceability to CI/CD job details and environments by linking external automation evidence directly from Xray.
  • Risks: Evidence may become unavailable over time due to CI tool retention limits setup or access restrictions not aligned with audit needs.

While CI/CD platforms like Jenkins, GitHub Actions, Azure DevOps, or CircleCI conveniently generate and store test artifacts, they’re not designed for long-term retention or compliance-driven traceability. Instead of uploading every raw output into Xray, many teams choose to link back to these artifacts, but this choice comes with trade-offs. Here are key considerations to keep in mind:

Retention Policies

  1. CI platforms often purge artifacts automatically. A 30, 60, or 90 days is typical. Teams unaware of these limits may unintentionally lose critical test evidence.
    1. If your organization requires longer retention (for compliance or regression traceability), be sure to configure your automation tool to retain artifacts accordingly.
    2. In contrast, Xray allows you to archive work items without losing access to the execution evidence, archive closed Test Plans and Executions to retain traceable evidence without cluttering active work views.


Your CI system is built for speed. Xray is built for permanence. - Your CI system is optimized for speed & designed to run fast feedback loops, discard intermediate data, and prioritize throughput over retention. Xray, on the other hand, is designed for permanence, with structured traceability, auditability, and long-term access to test history, evidence, and coverage across releases.


Some teams use external drives (Google Drive, OneDrive) to store test evidence. While this can reduce Xray storage usage, it comes with risks which include broken links, inconsistent access controls, and loss of audit trails. If you go this route, you risk compromising traceability and compliance.


Access Control Risks

  1. Automation platforms often lack the granular, role-based permissions found in Xray, making it harder to restrict who can access or alter critical test artifacts.
    1. Review and align group permissions with internal role structures.
    2. For audits, releases, or regulatory checkpoints, define a policy that certain evidence must be stored in Xray regardless of CI usage.
    3. Consider a quartetly review of who has access to each evidence repository, whether in CI, or cloud buckets.


Real-World Scenarios Where Storage Matters

Medical Devices Example: A company with multiple versions of the same hardware must retain test evidence per version. Why? Different hardware builds may not qualify for the same software release. Losing artifacts equals a compliance risk.


These scenarios highlight a simple truth:

  • Not all test evidence is created equal, and not all of it can live outside Xray.
  • For teams relying on CI tools but still needing governance, here’s how to balance flexibility with control.

General Guidance for Teams using CI + Xray

Scenario

Approach

Short-lived artifacts (debug logs)

You may prefer to link them in comments to keep execution records lightweight and focused.

Critical test output or evidence needed for audits

Best stored in Xray’s TestRun Evidence panel to ensure long-term availability and traceability.

Large reports (test coverage in HTML format)

Consider summarizing the content in Xray, with a link to the full version hosted externally.

Regulated environments

Ensure retention and access policies in your CI system are aligned with compliance needs. Xray can serve as a stable system of record if required.

Structured & Standardized Process

  • Profile: Teams subject to strict QA policies or audits.
  • Behavior: All evidence must follow naming rules, formats, and upload standards (e.g., PDF execution summary + JSON log). Storage settings are controlled via Global Settings.
  • Benefits: High auditability and consistent reporting.
  • Risks: Increased complexity that can compromise velocity and adherence from teams.

A well-structured and standardized approach to evidence storage doesn’t have to feel like a heavy burden. With Xray’s built-in features and smart configurations, teams can effortlessly maintain consistency, improve traceability, and accelerate test cycles.

Here’s how Xray can help enable a light, actionable evidence process:

  • Use Xray custom fields in Test Executions or Test Runs to help you capture key metadata. Not only evidence but test environment, revision for example.
  • Opt to configure field validations or mandatory fields to ensure critical evidence information is always provided without manual policing.
  • While storing all evidence directly within Xray’s Test Runs, ensuring a single source of truth, opt for use Xray’s traceability reports to quickly review evidence completeness and quality across test cycles.

Remember, Xray helps you to:

  1. Faster Reviews: Clear, consistent evidence reduces back-and-forth and accelerates approvals.
  2. Better Traceability: Easily track what evidence was collected, by whom, and when.
  3. Scalable Onboarding: New team members or partners ramp up quickly using templates and embedded guides.

General Guidance for a Structured & Standardized Process

Scenario

Approach

Works under strict QA/compliance policies (e.g., ISO, FDA, DO-178C)

Using Xray-specific workflow settings that block transitions unless all executions have been performed, alied with validators in your Jira workflow that block transitions unless mandatory fields or evidence are present

You have rotating or external QA resources

Creating pre-filled Test Execution templates and onboarding kits with ✔️/❌ examples


Structure doesn’t have to mean rigidity. When done right, it creates clarity, trust, and the space for teams to move faster with confidence.


This interactive tutorial walks you through smart storage decisions in Xray. Based on the type, purpose, and compliance needs of your test evidence.


Where Do You Go From Here?

You don’t need a one-size-fits-all policy. You need a storage strategy that fits each team’s reality.

  • Flexibility works, if teams are empowered and accountable.
  • Standardization delivers, when traceability is non-negotiable.
  • Automation scales, when backed by smart evidence governance.

Start by reviewing how your teams currently use storage. Then, map them to the patterns outlined above and adjust your global settings, evidence formats, and expectations to support both autonomy and control.


Learn More:


  • No labels