Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Xray provides a set of custom fields that can be used for some issue types, as detailed in Custom Fields and Screen Configuration.

Besides thisAside from these, you can add your own custom fields to any Xray issue type: Tests, Pre-Conditions, Test Sets, Test Executions, Test Plans.

Currently, you can't add custom fields to Test Runs since because they're not JIRA Jira issues.


Table of Contents

Xray-provided custom fields

As mentioned in Custom Fields and Screen Configuration, whenever you install Xray, it will also add some custom fields that are available only for some issue types. Bellow, please find a more detailed explanation Below are explanations of some of the most important ones.:

Begin Date

The start date for the Test Execution or Test Plan.

...

The Cucumber Scenario Type (i.e., Scenario or Scenario Outline).

...

Folders and their respective sub-folders are delimited by "/" (e.g., "components/compA").

The "Test Repository Path" is case-insensitive and each folder is trimmed (i.e., spaces are removed from the start/end of it).

...

Info
titlePlease note

You can move a Test to an existing folder within the Test Repository by editing the custom field in the Test issue.

Whenever editing this custom field, have  keep in mind that it is case insenstive-insensitive. This means that "components/compA", "components /compA",  and " components/COMPA" , are all the same and will be mapped to the same folder within the Test Repository.

If the folder is not found, then the Test is not associated with any folder; it will be seen within in the "Orphans" meta-folder in the Test Repository UI.

...

The definition of a generic Test (e.g., full classname of some automated test, name of some script, goals of exploratory testing).

...

Whenever we're speaking about the status associated with a requirement, we may in fact be talking about different things:

  • workflow status associated to the requirement issue (e.g., "New", "In Progress", "Closed")
  • (coverage) status for some version, taking into account executions made for that version of the Tests that do validate the requirement
  • "Requirement Status" custom field, the one that which we will detail next.

The "Requirement Status" custom field is a calculated field, usable in the context of requirement issues, that calculates the requirement coverage for a specific version. The version that for which it calculates the coverage for depends on the behavior for that field defined in a global configuration (see configuration details here).

As an For example, you may prefer to always calculate this "Requirement Status" based on the latest test run(s), independently in regardless of which version it/they was/were executed. .. or Or, you may prefer to calculate it for the earliest version yet to be released (i.e., the next version). In the later latter case, it takes into account the order of the versions within the project settings page.  

Have Keep in mind that a requirement  is assigned to and implemented in some version. However, the requirement will still live after that version, at least, for some time. Therefore, when someone mentions a "status of  a requirement", you have to ask "In which version?". The status of the requirement is thus dependant therefore dependent on the executions you make for the associated Tests in some version of the system, which may by different from the version in which the requirement was initialy initially implemented in.


In the following example, the first image shows the calculated value for the status of requirement for the next-to-be-released version (i.e., v3.0), while the second image shows the calculated values for all the unreleased versions. This behaviour behavior was changed in Custom Fields Preferences, in Xray settings, in this case from "Earliest unreleased version" to "Unreleased Versions" in the Custom Fields Preferences of the Xray settings.

 


This field is searchable, so you can make statistics with it or use it in gadgets. However, if you wish to search for requirements in some status, we recommend using you use the requirements() JQL function instead (see more info information here). You can also add it to your the cards of your Agile boards , to have real-time feedback of its status shared across your team members.

...

Info
titleLearn more
For more information on Requirement Status calculation, please see the Requirements Coverage Analysis.

...

Test Type

The Test type (e.g., Manual, Cucumber, Generic).

...

Whenever we're speaking about the status associated with a Test, we may in fact be talking about different things:

  • workflow status associated to the Test issue (e.g., "New", "In Progress", "Closed")
  • (coverage) status for some version, taking into account the runs made for that Test in some version of the system
  • "TestRunStatus" custom field,  the one that which we will detail next

The "TestRunStatus" custom field is a calculated field, usable in the context of Test issues, that calculates the "status" (not the workflow status) of the Test for a specific version, having in mind considering the executions (i.e., test runs) for that version. The version that for which it calculates the status for depends on the behaviour for that field behavior defined in a global configuration (see configuration details here).

The calculated value for the field also depends on the Test Environments, if they're being used , (as mentioned here). As an For example, you may prefer to always show the latest test run result independently in which version it was executed... or regardless of the  version. Or, you may prefer to show the status of the latest run executed on the earliest version yet to be released (i.e., the next version). In the later latter case, it takes into account the order of the versions within the project settings page.  

Although a Test in Xray is essentialy essentially a test case template, i.e., thus not related with to any execution whatsoever, you can add this field to the Test view screen as a way to have a glance of display the current status of it for some version right from on the screen of the "test template" screen (i.e., Test).


      



Info
titlePlease note

The At first glance, the name of this custom field may be a bit misleading at first sight. Although it is named "TestRunStatus", don't confuse it with the status (i.e., the result) of some recorded Test Run.

As mentioned above, the TestRunStatus custom field eventually takes into account , eventualy, many Test Runs.



This field is searchable, so you can make create statistics with it or use it in gadgets. If you wish want to do a search based on it, please see the proper syntax here.

...

Tests association with a Test Set 

Associated Tests associated with the Test Set.

Tests association with a Test Execution 

Associated Tests associated with the Test Execution.

Test Environments

The Test Environment(s) associated to with a given Test Execution. This is an open field, similar to a label; you should try to reuse existing Test Environments. It may be empty (default)By default, it is empty.

Test Execution Defects

Shows the number of defect issues associated with a Test Execution issue. These can be directly linked with the issue using JIRA Jira issue links or defects that were created during testing within the Test Runs.

Test Execution Status

This custom field provides information about the progress (i.e., the "Overall Execution Status") of the Test Execution issue.

It can be used , for example, in gadgets, included in the cards of Agile boards, or used as a column in JIRA the Jira issues listing.

This field is non-searchable.

...

The "Test Set Status" custom field  is a calculated field, usable in the context of Test Set issues, that calculates the "status" (not the workflow status) of all the Tests within the Test Set, for a specific version, having in mind considering the executions (i.e., test runs) for that version. The version that for which it calculates the status for depends on the behaviour behavior for that field defined in a global configuration (see configuration details here).

Although a Test Set in Xray is basically a list of Tests, i.e., thus not related with to any execution whatsoever, you can add this field to the Test Ser Set view screen as a way to have a glance of display the current status of the Tests that are part of it , for in some version right from the screen of the Test Set. This can be quite useful if you have some highly relevant critical Test Sets and you wish want to quickly have an a high-level idea of the status of the Tests that belong to it.

...

This custom field provides information about the progress (i.e., the "Overall Execution Status") of the Test Plan issue.

It can be used , for example, in gadgets, included in the cards of Agile boards, or used as a column in JIRA the Jira issues listing.

This field is non-searchable.

...

You may add CFs to Tests, however, they’re not copied to the Test Run. Custom fields at in Tests can be seen as complementary, non-relevant information.

But adding Adding custom fields is not limited to the Test issue type; they can also be added to other Xray issue types, including the Test Execution and the Test Plan. For example, in the Test Execution, you could add a custom field to identify the browser version. Note that, in this example, if you wanted to analyze your Test Execution per browser, it would be preferable to reuse use the Test Environments custom field instead; otherwise, you can add some custom fields to the Test Execution if their purpose is just to add more context to it.

...

Recommendations

  • Don’t reinvent the wheel; use . Use standard fields, such as labels and components, and avoid creating unnecessary creation of custom fields without need.
  • Remember that custom fields added to Tests are not copied to Test Runs, so they’re volatile/informative only; they’re not part of the specification.