Test Plans

Until now you had to organize your tests in Test Sets. And that makes all sense.

But one thing is organizing the tests, by grouping them, and another thing is planning tests for a given release.

With Xray 2.0 we've introduced the Test Plan, so you can create and manage your tests effectively. Assign one, or more, Test Plans to a given release and track progress of the tests within each Test Plan.

You can even see the requirement coverage taking into account the Test Plan tests.

You can now easily create Test Executions for the failing tests, or other, from within a Test Plan.

The Test status was calculated by looking at the test's last execution. But this does not work well if you execute the same test in two different devices and you want the status to be like a "merge" of the two results.

Within a Test Execution you can now specify the Test Environment(s) where the tests will be executed. A Test Environment is similar to a label, but Xray provides some special logic to deal with it. Let's say that you have executions for three Test Environments: "Chrome", "Safari" and "Firefox". The test is considered as PASS only if the latest executions for Chrome, Safari and Firefox are all PASS; it will be FAIL if any of them has failed.

Learn more here.

Xray supports, since day one, Pre-Conditions as a way to define initial conditions that should be met before executing a Test. This is very useful whenever you have to start by doing exactly the same thing in your tests. But sometimes you have to do several things  (e.g. restore DB, authenticate) before your Test steps.

Each of those things may be used independently (e.g. some Test may require just to restore the DB, another may require just to authenticate, and others may require that both things, or even more, need to be done before starting executing the steps).  Now you can assign multiple Pre-Conditions to a given Test. 

The execution page had a major facelift!

Our aim was to improve the overall user experience.

Since testers spend most of their time executing tests, we analyzed the process and made it more obvious and pleasant to work with.

You may notice that the "inputs" and the "output/results" of the test are now horizontally grouped. Besides that, the step status now reflects itself on the color of the step.

From within the Agile board or in the backlog you can see the requirement status, so you're always in control. 

In the Agile board, you can also see requirement's related executions, using Sub Test Executions. 

Learn more  here.

A Sub Test Execution has exactly the same functionalities as the Test Execution issue type. The difference between them is that the Sub Test Execution is a sub-task and can be created in a requirement's context.

Besides the direct visibility on the Agile board, creating a Test Execution as a sub-task of the requirement issue will provide the user the ability to better track time estimates.

Learn more here.

Although we already supported  custom test statuses, Xray now adds the ability of defining custom step statuses and their mapping to the Test status, giving you more configuration flexibility.

Learn more here.

We have improved the traceability in the REST API in order to make it easier to correlate and navigate between entities.

As an example you can now easily obtain the Test Sets or Test Executions related with a given Test... or you can obtain the Test and Test Execution related with a given Test Run.

Of course, we've added specific endpoints for Test Plans.

And, in between, we have updated the whole REST API with more examples.

Learn more here.

The documentation was completely reviewed, reorganized and is now more extensive and detailed in many areas. We have added many diagrams to make things clearer.

Some pages worth giving a look include the whole Testing with Xray, Project Organization Use Cases, Test Process, Terms and Concepts, Examples of Reports Built with Xporter, Examples of custom gadgets, FAQ, just to name a few.

We invite you to explore Xray's complete documentation starting with the Xray Documentation Home page.