Overview
Enterprise teams generate a massive volume of Test artifacts: automated results, screenshots, logs, and evidence. Without a clear storage strategy, even the most powerful Xray instances can become harder to manage over time. But Storage Management isn't just a technical concern; it reflects your process maturity, tooling alignment, and governance culture. This guide goes beyond limits. It offers real-world storage strategies tailored to Enterprise Testing realities.
Choosing Your Storage Pattern
Decentralized, Unlimited Approach
- Profile: large enterprises with unlimited storage that prioritize autonomy over control.
- Behavior: no file type or size restrictions; teams manage their own Test Execution and reporting without enforced Templates.
- Benefits:
- High flexibility.
- Fast enablement.
- Risks:
- Uncontrolled growth of Test artifacts.
- Inconsistent practices across teams.
- Breach of compliance requirements.
While Xray Enterprise offers unlimited storage, maintaining performance in Jira environments still benefits from healthy attachment practices and routine housekeeping. Here’s what you can do:
- Encourage regular Test Run housekeeping: establish a retention policy for test executions based on criticality, audit needs, and workflow habits. Periodically review and delete older Test Executions that exceed the defined retention period.
- Monitor storage-heavy projects:
- Identify projects with high attachment usage.
Use the Global Storage Settings to detect them within Xray or by exporting data.
- Follow up with teams to encourage cleanup or optimization.
- Optimize evidence handling in automation: when using the Xray REST API, validate imported results. Only submit relevant Test Executions, statuses, and evidence. Avoid importing large logs, screenshots, or debug outputs unless required for traceability or audit purposes.
Shift verbose data to CI/CD tools: keep detailed build logs in your CI/CD system and link the build to the Test Execution Issue in Jira using a custom field for full traceability.
Governance Visibility
Figure 1 - Storage
General Guidance for a Decentralized Approach in Xray
Scenario | Approach |
|---|---|
Operate in a high-speed, autonomous environment | Allow teams to manage their own evidence formats, while guiding them with optional best practices |
Run frequent automated Tests with large result volumes | Define which evidence types are required for long-term traceability (e.g., for regulatory validation), and apply lighter standards to low-risk, routine runs to avoid unnecessary storage growth |
Externalized Evidence Strategy
- Profile: automation-heavy teams using CI/CD pipelines with Xray.
- Behavior:
- Evidence (logs, videos, reports) is stored outside Xray, often in cloud buckets or CI platforms.
- Xray execution comments are used to paste the artifact URL.
- Benefits:
- Minimal Xray storage usage.
- Enables traceability to CI/CD job details and environments by linking external automation evidence directly from Xray.
- Risks: evidence may be lost due to CI tool retention limits, configuration gaps, or access restrictions not aligned with audit needs
CI/CD platforms (like Jenkins, GitHub Actions, Azure DevOps, or CircleCI) are great for generating and storing Test artifacts, but not for long-term retention or audit-level traceability. Linking to these artifacts instead of uploading them into Xray reduces clutter but introduces trade-offs.
Key Considerations
Retention Policies
CI platforms often purge artifacts automatically (e.g., after 30, 60, or 90 days). Teams unaware of this may unintentionally lose critical test evidence.
Recommendiations
If longer retention is needed (for compliance or regression traceability), configure your automation tools accordingly.
Xray allows you to archive work items (e.g., Test Plans or Executions) while retaining evidence access, without cluttering active views.
Your CI system is built for speed. Xray is built for permanence.
CI tools are designed for fast feedback and throughput, discarding intermediate data quickly. Xray is designed for structured traceability, auditability, and long-term visibility into test history and evidence.
Some teams use shared drives (e.g., Google Drive, OneDrive) for storing evidence, but this introduces risks like broken links, inconsistent access control, and audit gaps.
Accessing Control Risks
Automation platforms often lack the granular, role-based permissions found in Xray, making it harder to secure critical Test artifacts.
Recommendations
- Review and align group permissions with internal role structures.
Define a policy requiring certain types of evidence to be stored in Xray - regardless of CI integration - especially for audits or regulated releases.
Review access to each evidence repository (CI or cloud storage) at least quarterly.
Real-World Scenarios Where Storage Matters
Example: Medical Devices
A company with multiple hardware versions must retain test evidence for each one. Why? Because different builds may not qualify for the same release. Losing artifacts creates compliance risk.
The key takeaway:
Not all test evidence is created equal - and not all of it should live outside Xray.
For teams relying on CI tools but still needing governance, here’s how to balance flexibility with control:
General Guidance for Teams using CI + Xray
Scenario | Approach |
|---|---|
Short-lived artifacts (e,g., debug logs) | Link them in comments to keep execution records lightweight |
Critical Test output or evidence needed for audits | Best stored in Xray’s Test Run Evidence panel for long-term availability and traceability |
Large reports (e.g., Test coverage in HTML format) | Summarize in Xray and link to the full report hosted externally |
Regulated environments | Align CI retention and access policies with compliance needs; use Xray as the source of truth where needed |
Structured and Standardized Process
- Profile: teams working under strict QA or audit requirements
- Behavior: all evidence follows naming conventions, file format rules, and upload standards (e.g., PDF + JSON). Storage is governed via Global Settings.
- Benefits: high auditability, consistent and reliable reporting.
- Risks:
- Higher complexity.
- Potential slowdowns if teams lack proper guidance or Templates.
A standardized process doesn’t need to feel heavy. Xray’s built-in features can help teams maintain consistency while moving faster.
How Xray can help enable a lightweight, effective evidence management:
Use custom fields in Test Executions or Test Runs to capture metadata like Test environment and revision.
Add field validations or required fields to ensure essential information is captured.
Store all evidence in Xray to ensure a single source of truth.
Use traceability reports to assess evidence completeness and quality across cycles.
Xray enables:
- Faster reviews: clear, consistent evidence approval delays.
- Better traceability: know who uploaded what and when.
- Scalable onboarding: templates and guides help new members ramp up fast.
General Guidance for a Structured and Standardized Process
Scenario | Approach |
|---|---|
Working under strict QA/compliance policies (e.g., ISO, FDA, DO-178C) | Use Xray-specific workflow settings that block transitions unless all executions have been performed, with validators in your Jira workflow that block transitions unless mandatory fields or evidence are present |
Rotating or external QA resources | Provide pre-filled Test Execution templates and onboarding kits with clear ✔️/❌ examples |
Structure doesn’t mean rigidity. When done right, it builds clarity, trust, and speed.
See this interactive tutorial to help you walk through smart storage decisions in Xray (based on the type, purpose, and compliance needs of your Test evidence).
Figure 2 - Storage
Next Steps
You don’t need a one-size-fits-all policy: just a storage strategy that fits each team’s reality.
- Flexibility works when teams are empowered and accountable.
- Standardization delivers when traceability is non-negotiable.
- Automation scales when backed by smart evidence governance.
- Start by reviewing how your teams currently use storage.
- Map the teams to the patterns above, then adjust global settings, evidence formats, and expectations to balance autonomy with control.