Welcome to Xray! Are you ready to get started?

Select your profile and follow our quick guide.



Before you start


Before you start, make sure your Jira Admin has performed the basic configuration.
If so: 

  1. You can create issues of issue types like: Test, Test Set, Test Plan and Test Execution
  2. In your Requirements (issue type Story or any other that you have set up as a requirement) you will see the "Test Coverage" panel.


If you don't see this configuration, ask your Jira Admin to follow this quick setup.

Create a Test

Start by creating a regular "manual" scripted test case.


  1. Click the Create button or Create Test from the "requirement."




  2.  After the issue is created, edit the issue and in Test Details tab select the test type Manual.



  3. Then click on Create Step to start creating the Test Steps where you can define the action to be reproduced by the tester, data to be used by the tester, expected results and add attachments.



  4. Confirm that your Test is linked with a requirement. In order to analyze coverage, you need to link your tests with requirements. Go to Link issues tab, or check it Linked Issues.



Want to know how to create an automated Test or a generic Test (unstructured)? Check out the Tester Onboarding Course.


Plan Tests


Info

This activity can be performed by a QA Manager, if so, you should ask for the creation of the Test Plans.


With Test Plans you will decide which Tests you want to perform, who will execute them and when they will be executed.
Also, you may want to prioritize some of the Tests and focus on specific requirements.
Test Plans are used to define the scope of your testing, track and consolidate the related results, no matter how many times you've run the tests.

  1. Click Create button and select Test Plan issue. 



  2. Besides Summary, you may want to fill out the Fix Version to indicate that you're tracking the results of the Tests for a specific version. This only makes sense if you have multiple versions.



  3. After creating the Test Plan, you can add Tests.



  4. Choose the Tests you want, by using the Select or Search tabs and then selecting some or all of them. Click Add Selected or Add All.



Execute Tests


The easiest way to execute your tests is directly from the Test Plan, but you can also perform ad-hoc Test Executions directly from a Test, or even create a new Test Execution issue. 


  1. Inside the Test Plan, click on Create Test Execution.



  2. Go to Test Execution, and click on Execute (play button).



  3. You are now at the Test Execution Screen, and from here you can follow the test steps and set the execution status: TODOEXECUTINGFAIL and PASS.



  4. Did you find a defect and want to report it? Create Defects from the Test Execution Screen, click on + button in the Findings section and select Create Defect (or expand the Findings section and in the Defects panel select Create Defect)

           Besides defects, you can also add evidence of your test results, at "Evidence + ". 



Analyze results 


  1. Go to the Requirement issue page and check the Test Coverage.



  2. Go to the Testing Board inside your project and select the report you want to see.


You can analyze the status of your project from two different perspectives:

  • Based on requirements: you have reports such as Test Coverage or Traceability Report which provide insight into the status of requirements.
  • Based on Tests: which look directly at related Test artifacts such as Tests, Test Sets, Test Plans and Test Executions. Xray also provides reports with metrics for these entities.



Learn More

This is just a start! There is so much more to learn and to explore about Xray. 

You may want to learn more about: 

  • How to create automated & generic tests
  • How to use preconditions
  • How to work with Test environments
  • Understand how coverage works

Take the onboarding Tester course for a deeper understanding.




Decide your project organization strategy

Select the best approach to match your QA strategy with Jira project configuration. 



AdvantagesDisadvantages

All-in-one project

Requirements, Defects and Tests in the same Jira project

(tick) Info all together and visible to the whole team

(tick) Same context - testers and developers

(tick) Less administration work

(tick) Simple to understand

(error) Extra info that may be confusing to the team


Separated projects

Separate Tests from Requirements and Defects

(tick) Dedicated project for Testers

(tick) Less information inside the project 

(tick) Permissions can be more granular 

(error) Extra administration effort

(error) May promote a team split


If you want to go for the easiest and simplest approach go for the all-in-one project. 



Share your strategy with your Jira Admin

Once you know how you will organize your project, share your choice with your Jira admin and ask them to perform the needed actions for setup: 

  • Enable Xray for existing projects or
  • Create new projects

Select your Planning Strategy 

When designing your planning strategy you will decide which tests you want to perform, and who and when will execute the tests. Also, you may want to prioritize some of the tests and focus on specific requirements.

Your planning strategy must reflect your methodology, here are some options: 


(tick) SCRUM based approach - Create one Test Plan per Sprint


(tick) Feature Approach - Create one Test Plan for the features (maybe epics in Jira) you are able to address at your WIP. More suitable to Kanban. 


(tick) Release Approach - Create one Test Plan per release (Fix version in Jira) or interim release/milestone. More suitable to waterfall methodology.


(tick) Test Categories Approach  -  Create one Test Plan per category of tests, for example, Regression Tests, Security Tests, Non-functional tests.


Check the next step to see how to Create a Test Plan. 

Create a Test Plan

  1. Click "Create" button and select the Test Plan issue type. 


2. Besides the Summary, you may want to fill the Fix Version field to indicate that you're tracking the results of the Tests for a specific version. This only makes sense if you have multiple versions. 


3. After creating the Test Plan you can add Tests. 



Choose the Tests you want by using the Select or Search tabs and then selecting some or all of them.

Click Add selected or Add all button.



Info

In order to add Tests to your Test Plan, you need to have them created beforehand. Learn more about creating Tests in the Tester Onboarding Course.

Explore different possibilities to organize your Tests

You can organize tests using two different approaches: 

  • Test Repository - a project page which enables the hierarchical organization of Tests at the project level by allowing you to organize Tests in folders and sub-folders. You can also perform several actions from this page including, creating new Tests, creating new Test Plans, Test Sets, Test Executions for a group of Tests, etc. This is the recommended approach whenever you have a large number of Tests and want to manage them more effectively.


  • Test Set - an issue type for organizing tests, and it is a flat list of Tests and is a simple way to create different groups of Tests. You can easily use them to add several Tests at once into Tests Plans or Test Executions.


Select your favorite reports

Xray provides several ways to make it easier for the entire team to have a clear view of the overall readiness of the project.

As part of your daily tasks, use the dedicated reports and gadgets available in your customized dashboard to evaluate your testing progress.

The most popular reports are: 

ReportWhat do you get
Test Coverage Report

Provides the coverage status of requirements for a given scope (i.e. Version/Test Plan).

  • Analysis can be done by “Version” or by “Test Plan” and for a given Test Environment
  • Takes into account executions performed on that scope
  • Groups requirements by Component, Priority, etc
Traceability Report

Shows the requirement traceability, from requirements to defects.

This helps you analyze the requirements and related Tests, Test Runs and defects, so you can quickly identify uncovered or incomplete/faulty requirements.

Test Plans Metrics

Shows a list of Test Plans with consolidated information for each one, including the test statuses count, overall progress and Test Environment-related metrics

Test Run List 

Shows a list of Test Runs with detailed information for each one, including related Test, Test Execution and Test Plan entities, execution dates, and linked defects.

If you want to analyze the current status of your release, you might find it useful to check the Test Coverage Report and the Traceability Report. You might also use these reports to perform an analysis by Test Plan, in that case it only takes into account results performed on that scope of the test plan.



  1.  Go to the Testing Board (inside your project) and select the report you want to see. 




Explore and learn more by taking the QA Manager onboarding course.




Before you start: 

When setting up Xray, you may need the input from the testing/QA team about usage and preferences. 

We recommend you discuss the following points before you begin to configure Xray: 

  • Which entities must be considered as requirements and defects?
  • What is the correct project organization for the team? Should there be Tests and Requirements in the same project or in separate projects?

Check the QA Manager getting started to get more info on this.


Configure Requirements & Defects


First, you need to set which issue types are going to be considered Requirements and Defects. This is done at the project level.


1. Access your Project Settings > Xray Settings > Test Coverage

Perform the desired mapping and click "Save."


2. Access your Project Settings > Xray Settings > Defect Mapping

Perform the desired mapping and click "Save."


Info

Requirements can be covered by Tests, and are the expected results for the system. They don't have to be requirements in a strict sense, but we use this term to simplify the explanation.

Examples: Requirement, Story, Epic, Feature, Improvement and others.


Defects are identified during Tests Executions, and represent a condition in the system that does not meet the requirement or end-user's expectations.

Examples: Defect, Bug, Fault and others.



Add Xray Issue Types to your schemes


You may want to include Xray Issue Types (such as Test, Test Plan, Test Set, Test execution) to your already created Issue Type Schemes and Issue Type Screen Schemes. 


1. Access your Project Settings > Xray Settings > Summary and select the Add Xray Issue Types.


If you prefer, you can add Xray Issue Types manually. This will give more flexibility in choosing the issues types that you want to use. 

To add issue types manually, you can click on Configure the Issue Type Scheme manually available on the Summary section within the projects settings or access the issue type scheme on Jira settings.

After editing the issue type scheme, you must drag and drop Xray Issue Types: Test, Test Plan, Test Set, Test Execution, Precondition and Sub Test Execution to "Issue Types for Current Scheme" and then click "Save".



         

For Team-managed Projects you need to create Issue Types manually and then map them with Xray Entities in Xray Settings > Issue Types Mapping. Learn more in "Administrator Onboarding Course."



Do you want to customize your Xray experience? Review these configuration details. 


Xray is a very flexible tool! Do you want to customize your experience even more? Get together with your Xray users and find out how they want to organize their projects, if they want to customize custom fields, workflows or other settings. 

  • Review Custom Fields for Xray Issue Types - Check if you need extra custom fields in your Test Issue Types. 

  • Review Test Steps Fields - Check if you need to add extra info to test steps. By default, Xray has fields like Action, Data and Expected Results. 
  • Review Test Run Fields - Check if you need to add extra info to test runs.

  • Review Test Types - Check if you need any extra Test Types.
  • Create Test Environments - Check if your teams needs to create Test Environments.
  • Review Test Status - Check if you want to add any extra Test Status. 
  • Review Test Step Status - Check if you want to add any extra Test Step Status.
  • Review Workflows - Check if you need to have a custom workflow for each Xray Issue types, such as Test. 
  • Review Permissions - Check if permissions are set according to your team's needs.


Explore more in Administrator Onboarding Course






  • No labels