Versions Compared

Old Version 7

changes.mady.by.user Sergio Freire

Saved on

compared with

New Version 8

changes.mady.by.user Sergio Freire

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

This version closes the gap to enterprise test management tools and introduces hierarchical organization for Tests and goes even further.

Xray v3.0 introduces hierarchical Tests organization both at the project level, with the Test Repository, and also at Test Plan level, with the Test Plan Board.

Hierarchical Test organization can be enabled or disabled, working side by side with the other existing entities, including Test Sets.

...

Panel
borderColor#F3F3F3
bgColor#FAFAFA
borderWidth1px
Section
Column
width50%

Test Repository

Until now you had to organize your tests in Test Sets: flat, ordered lists of Tests. Test Sets can be used to group tests together in any logical way (e.g. by business case, component, etc).

The Test Repository

Xray v3.0 introduces the Test Repository concept which enables the hierarchical organization of Tests at the project level by alowing users to organize Tests in folders.

This is a different approach from the traditional way of organizing Tests in flat lists by using Test Sets, which was the only available method before Xray v3.0.

Users coming from legacy Test Management tools may find this approach more natural and intuitive. Other organizations may prefer to not enable this feature at all and choose the Test Set approach instead. There are more some pros and cons on using this kind of organization but both concepts can live together.

A new custom field "Test Repository Path" for Test issues can be included in the respective screens, to see the Test Repository folder where the Test is included in.

Learn more about the Test Repository hereYou can now easily create Test Executions for the failing tests, or other, from within a Test Plan.


Column
width50%


Image RemovedImage Added
Panel
borderColor#F3F3F3
bgColor#FAFAFA
borderWidth1px
Section
Executions for multiple

Test

Environments

Plan Board


Column
width40%

Image RemovedImage Added

Column
width60%

While the Test Repository concern is related with the organization of Tests in the scope of a given project during the specification phase, the Test Plan Board concern is on organization at planning level (and as consequence, at execution).

The Test Plan Board gives you the possibility to organize the Tests that are part of your Test Plan in multiple folders and sub-folders. Implicitly, you can also define the order of the Tests and the relevance of those grouped Tests (i.e. folders) by ranking them within the tree presented within the Board.

By providing you the ability to define groups and sub-groups (i.e. folders and sub-folders) for the Test Plan's Tests, you're able to track them independently and see right way their current status, even if the scope of the Test Plan is broader. You can also more easily do operations for those subset of Tests, such as creating Test Executions

The Test status was calculated by looking at the test's last execution. But this does not work well if you execute the same test in two different devices and you want the status to be like a "merge" of the two results.

Within a Test Execution you can now specify the Test Environment(s) where the tests will be executed. A Test Environment is similar to a label, but Xray provides some special logic to deal with it. Let's say that you have executions for three Test Environments: "Chrome", "Safari" and "Firefox". The test is considered as PASS only if the latest executions for Chrome, Safari and Firefox are all PASS; it will be FAIL if any of them has failed .

Learn more here.





Panel
borderColor#F3F3F3
bgColor#FAFAFA
borderWidth1px
Section
Multiple Pre-Conditions

Tests ranking within the Test Plan

Column
width60%

A long awaited feature was the ability to rank Tests within the Test Plan, so it could be used to define some sort of priority for the Tests being tracked within the planned.

This ranking can then be taken into account for the Test Executions created from within the Test Plan.

Ranking of Tests in the Test Plan can in fact be done in two ways: directly in the Test Plan issue screen or in the Test Plan Board, in the case hierarchical test organization is disabled or enabled respectively. This gives users the possibility to do ranking of Tests, independently on the decision of using hierachical test organization.

Xray supports, since day one, Pre-Conditions as a way to define initial conditions that should be met before executing a Test. This is very useful whenever you have to start by doing exactly the same thing in your tests. But sometimes you have to do several things  (e.g. restore DB, authenticate) before your Test steps.

Each of those things may be used independently (e.g. some Test may require just to restore the DB, another may require just to authenticate, and others may require that both things, or even more, need to be done before starting executing the steps).  Now you can assign multiple Pre-Conditions to a given Test. 

Column
width40%



CHANGE SCREENSHOT


Image Added

Image Removed

Panel
borderColor#F3F3F3
bgColor#FAFAFA
borderWidth1px
Section
Redesigned execution page

Test Evolution Gadget


Column
width50%

Image RemovedImage Added

Column
width50%

The execution page had a major facelift!

Our aim was to improve the overall user experience.

Since testers spend most of their time executing tests, we analyzed the process and made it more obvious and pleasant to work with.

new Tests Evolution gadget enables you to see, at a glance, the evolution/trend of the status of a group of Tests, taking into account the results of those Tests in certain Test Executions.

The calculated status (e.g. TODO, PASS, FAIL, etc) of the Tests per day is shown.

Possible usage scenarios:

  • see the progress of the calculated status of certain Tests for a specific version
  • see the progress of the calculated status of certain Tests for a specific version on a specific Test Environment
  • see the progress of the calculated status of Tests in a given Test Plan, taking into account the Test Executions associated with the Test Plan.

Learn more hereYou may notice that the "inputs" and the "output/results" of the test are now horizontally grouped. Besides that, the step status now reflects itself on the color of the step.



Panel
borderColor#F3F3F3
bgColor#FAFAFA
borderWidth1px
Section
Agile Enhancements

HP ALM/QC importer


Column
width50%

Xray v3.0 provides a basic built-in tests importer for HP ALM/QC v12.5x.

This importer is able to import test cases from HP ALM's "Test Plan" (not in any way related with Xray's Test Plan issue) by using HP ALM's REST API.

The current implementation is essentialy limited to import manual test specifications, their steps and their attachments

From within the Agile board or in the backlog you can see the requirement status, so you're always in control. 

In the Agile board, you can also see requirement's related executions, using Sub Test Executions. 

Learn more  here.

 

A Sub Test Execution has exactly the same functionalities as the Test Execution issue type. The difference between them is that the Sub Test Execution is a sub-task and can be created in a requirement's context.

Besides the direct visibility on the Agile board, creating a Test Execution as a sub-task of the requirement issue will provide the user the ability to better track time estimates.


Learn more here.

Column
width50%


Image RemovedImage Added


Panel
borderColor#F3F3F3
bgColor#FAFAFA
borderWidth1px
Section

Custom Manual Test Step Statuses


Column
width50%
Column
width50%


Although we already supported  custom test statuses, Xray now adds the ability of defining custom step statuses and their mapping to the Test status, giving you more configuration flexibility.

Learn more here.


...

Panel
borderColor#F3F3F3
bgColor#FAFAFA
borderWidth1px
Section

Revamped Documentation, once again


Column
width100%

The documentation was completely reviewed, we

reorganized and is now more extensive and detailed in many areas. We have added many diagrams to make things clearer.

Some pages worth giving a look include the whole User's Guide, Project Organization, Test Process, Terms and Concepts, Examples of Reports built with Xporter, Examples of custom gadgets, FAQ, just to name a few.

We invite you to explore Xray's complete documentation starting with the Xray 3.0 Documentation Home page.

And

...

more...


Ability to associate tests to existing Pre-Condition(s) in the Test creation dialog 
Create Tests, and Sub Test Executions, directly from the requirement issue screen
Execute tests in and change status in bulk for multiple tests 
New Test Coverage section in the requirement issue screen, with list of tests and their status 
Filter by requirement's Resolution in requirement coverage panel 
Ability to configure columns both in the Test Runs panel inside the Test issue screen and also in the Tests panel inside the Test Set issue screen
  • Completely revamped execution screen, more consistent, clearer and user friendly
    • Ability to transition Test Execution status accordingly with the results of its tests  
    Ignore requirements in coverage charts based on requirement's specific label(s)
  • Ability to define a project as a "requirements project"
    • CSV export improvements (e.g. list of tests related with test executions, list of pre-conditions related with tests)
    JQL functions to support Test Plan and more
    Translations for French and Spanish
    AdministrationJQL functionsOther
    • Greater control of who can Add Xray Issue Types in Project settings


    • Enable/disable hierarchical Test organization

    • Ability to define the project roles allowed to edit the Test Repository

    • Ability to change the Test Run's started date
    • JQL function for obtaining Tests of a Test Execution by the “executed by“
    • JQL function for obtaining tests within a Test Repository folder

    • JQL function for obtaining tests within a Test Plan Board folder
    • Test Set filter in Test Execution screen shows only relevant Test Sets


    • Xporter improvements related with exposing info about Xray attachments on Tests and Test Runs


    • Test Case Importer
      • Comments
      • DateTime fields
      • Test Repository folder (for existing folders)


    • Inherit dates for planned Test Executions, from the Test Plan

    • Ability to change the Test Run's started date
    UX & UIAdministrationOther

    New Features and Bug Fixes in this release

     

    Jira
    serverXpand Add-ons Issue Tracker
    columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
    maximumIssues20
    jqlQueryproject = XRAY and fixVersion = "R3.0.0" and level is EMPTY order by issuetype desc
    serverIdb6309ad5-5416-33d6-9a09-a3e02f20d3e6

    Xray v3.0, there is a basic built-in tests importer for HP ALM/QC v12.5x.

    This importer is able to import test cases from HP ALM's "Test Plan" (not in any way related with Xray's Test Plan issue) by using HP ALM's REST API.

    The current implementation is essentialy limited to import manual test specifications, their steps and their attachments.