Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

How does Xray handle "versioning" of Test cases

Info

Test case versioning is only available in Xray Enterprise.


We can handle the versioning Versioning of Tests works in the same way as the versioning of other issue types in Jira. Since Xray's Tests are Jira issues, we follow the same approach so your users work in the same manner they already do.

Considering that Test issues are like reusable test case templates, should you assign Test issues to a given version? If so, what would be the purpose of it?

However, even though you can see all the changes of a Test case using the Jira History tab, it is not easy to revert or view a previous Test version easily unless you create a new Test issue for each version. This is also not ideal as you end up with multiple issues for the same entity.


Another downside of using Jira versions to manage Test versions is related with the lifecycle of a Test. Jira versions are created at the project level. Hence, all issues within the project are affected by the same version numbers. 

Test cases can have a different lifecycle than other issues in the same project, such as requirements or bugs. For instance, we can have multiple versions of a Test case for the same project version (e.g., the test has evolved with the changes in the requirement; they have different types).


This is why Xray has introduced a new way of versioning Tests in v7.0. Starting from this version, it is possible to create new versions within the same Test issue. You can revert back to a specific version and view all versions of a Test case easily.

However, this doesn't mean that you can't use Jira versions. In fact, you can use both. Although The test specification has a lifecycle. Although your tests may live across many versions of your project, they're made in the context of some version, most probably for the version of the requirement they aim to validate. Thus, assigning them to a version Fix Version serves the purpose of tracking the specification+implementation of the test case, the first time they have been used. This way, you can also track it in the Release as one artifact that needs to be "done" (i.e., specified/reviewed/approved).

But you may want to know the "version" (let's call it revision to avoid misunderstanding) of the Test that you ran in some iteration. Xray does not track revisions of Tests as numbers, in order to avoid unnecessary complexity. It tracks changes and keeps a version of the Test specification within each Test Run.

Xray addresses versioning the Jira way, which we think is more elegant, while at the same time, ensures data consistency:

...

It is still important to note that Xray makes a copy of the Test specification whenever you schedule a Test for execution in a Test Execution.

  • A , Xray creates an instance of the Test that we call Test Run. That Test Run effectively contains the "version"/revision of the Test specification Test specification of the Test version chosen when the Test Run was created (at that moment).
  • You can go to an already recorded Test Run and reset the specification to the last/actual current specification on the Test version which originated the Test issueRun. Or, you can merge it, which means that only changed steps will be updated and the other step results are not changed. More info can be found on the Execute Tests page.  Normally, you work with the last "version"/revision of the Test (as you do for a user story, etc.). However, if you really want to manage different versions of Tests at the same time (assuming that they're actually different), you can create different Tests, one per version.

By persisting the Test specification on the Test Runs,  Xray ensures data consistency. The "version" of the Test that was executed in a Test Run, is persisted in the Test Run itself. That means that if you change a Test specification today, that won't affect your already recorded runs for that Test (unless you want to).

...