...

Is it possible to use Time Tracking with Xray? Is it possible to log time associated with the running of Test Runs?

Yes. Please have a look at refer to the Time Tracking documentation.

Upgrading

What should I do to update/upgrade my current version of Xray?

If you have Xray installed in a standard JIRA Jira server, you can just update it as detailed in FAQ (please in the FAQ. Make sure to check the JIRA Jira compatibility matrix first).

If you are using JIRA Jira Data Center, you should restart you must restart all nodes after installing the new version of Xray.

...

There is no "Requirement" issue type installed by Xray. In fact, Xray does not install any requirement-related issue types. If you had the opportunity to attend one of our webinars, namely particularly the Xray walkthrough, you may see will learn that with Xray, you are able to define which issue types Xray should considered consider as requirements so you can cover them with Tests. Besides the standard "Story" and "Epic" JIRA Jira issue types, JIRA Jira allows you to create your own issue types. You just need to configure Xray properly, as detailed here, to let Xray internaly internally treat those issue types as being requirements.

I'm unable to see

...

the "Test Coverage" panel and the "Requirement Status" in my requirements. Why? 

You need to assure two thingsensure that:

1. that your project is defined as being a requirements project;, and
2. that your issue type is configured to be a requirement issue type.

Both can be achieved in Xray's administration (see FAQ for instructions), although the first one may also be done also in the project settings page.

...

The "Requirement Status" is a calculated field, so it shows the information for some version depending on a Xray setting available in Custom Field Preferences. Currently, it is not possible to specify explicitly the version that you want to calculate the coverage on. Please read read Using custom fields for a deeper explanation.  

...

The association of a Test with a project version is not direct. Your tests Tests are written in order to validate some requirement, that itself belongs to a specific version. Although a Test may be associated with a requirement of a specific version, you are able to run tests Tests for the versions you would like to. In terms of requirement coverage, you have different behaviors to choose from in the administration settings (see Requirement Coverage Strategy). You may also to have a look at Test Versioning.

Do you support versioning of Tests?

Please have a look at refer to Test Versioning.

How do I define which tests should be used for regression testing?

Please take a look at FAQ.

Can I

...

link Tests to

...

other Tests?

Yes. You can create normal links between Test issues, as you would do for any standard JIRA Jira issue. Xray will not process those relations relationships in any special way; it will ignore them unless they are of "tests" or "tested by" relationsrelationships.

The "TestRunStatus" custom field does not show the "expected" value. Why? 

The "TestRunStatus" is a calculated field, so it shows the information for some version, depending on a Xray setting available in Custom Field Preferences. Currently, it is not possible to specify explicitly the version that you want to calculate the coverage on. Please read Using custom fields for a deeper explanation.

Test Sets

Can I clone a Test Set? Will that clone

...

the Tests that belong to the Test Set?

You can clone a Test Set and the cloned Test Set will have the same Test Sets that were in the original Test Set. The tests  themselves themselves are not cloned. If you really want to clone a bulk list of tests, then you must clone them manually (you may find add-ons for this purpose), and then associate  them to the new Test Set. In fact, whenever you clone either Test Sets, Test Executions or Test Plans, the tests Tests that they contain are not cloned; just the internal association.

...

You can do it easily within the Test Plan screen , by pressing via the "Create Test Execution" button. You may also create a new Test Execution for some of the tests listed in the table bellow below the Filters, by pressing the bulk icon  and then pick the relevant tests (you may use a filter to help you out), and then choose the Create action in the actions icon , available on the right side of the table.

...

Kind of. A specific Test Execution represents a an execution task of a list of tests, for a specific version/revision, done in at a specific time. With Xray, the normal way of executing a set of tests multiple times is by creating a Test Execution for each time you want to execute the tests. All you need  to do is clone an existing Test Execution; the newly-created Test Execution will contain the same list of tests, in clean state since you haven't executed them yet.

Do I have to use Test Environments?

No. But , but if you execute the same tests on in different environments, you certainly will find it as the most useful and accurate way of testing a requirement.

...

No. It’s up to you to organize your Tests per environments, this environment. This is the reason we use the Test Environment Custom Field as a label.

Can I take screenshots and attach them

...

when executing tests?

Xray does not provide by itself this feature out-of-the-box by itself. However, you may use a simple tool dedicated to take taking screenshots (e.g., LightShot) and easily attach the screenshot to the evidences during the execution of the test.

Test Plans

A “Test Plan” seems like a group of Test Executions or a group of

...

Tests.

A Yes, a bit of both, in fact. A Test Plan contains a list of Tests. For calculating the status of each test, it takes into account test runs from Test Executions explicitly linked to the Test Plan, through the field “Test Plan”.

Can I clone “Test Plans”?

Yes.  The list of related Tests is copied; the Tests are not cloned.

Can I have multiple Test Plans for a version?

Yes, if you wish to track different groups of Tests (e.g., regression tests vs tests versus tests for new requirements).

Do I have to use "Test Plans"?

You don’t have to, but soon you will find great advantages if you start using them.

Agile Integration

Can I add a Sub-Test Execution to multiple Requirement issues?

A sub-task can only have one parent issue, so it won’t be possible to have multiple requirement issues with the same Sub-Test Execution. However, you can add Tests from another Requirement issue to a Sub-Test Execution. You’ll be able to execute them, but you are going to have the same behavior as the Test Execution, while using Agile Boards. So, if you intend to track the executions in the Agile Board, we recommend that you to have a Sub-Test Execution for each Requirement issue.

Do I have to associate all Tests that are testing a Requirement with a Sub-Test Execution?

The Tests already linked with the Requirement issue will be automatically added to the Sub-Test Execution. You can remove them if you intend towant. If you add a new Test to the Requirement issue after creating the Sub-Test Execution, then you’ll need to add them manually.

Configuration

Is it possible to use custom workflows for

...

Test-related issues? Does Xray install or

...

enforce any workflow?

Xray does not install any "default" workflow for the issues it provides. You may define whatever worfklow you think fits best your scenario/organization. Xray is workflow-aware though, and provides some configurations specificaly specifically for that purpose (see Global Preferences page), .   

Licenses

Where can I find the pricing for Xray and licensing information?

More information can be found in the respective Xray section in of the Atlassian's Marketplace.

How do I install a license provided by Xpand IT?

Once you have a license provided by Xpand IT, you should perform the following steps:

• On In your JIRA Jira Administration, select the option " Manage Add-Ons".
• Go to " Xray Configuration ", section "> License Management".
• Select Click on the button "Click here to install a license directly from Xpand IT".
• Put Enter your license on in the text box. Remember, this shoud be a license provided by Xpand IT and , not via Atlassian Marketplace.
• Click on addAdd.
• Your Xpand IT After the executing the previous steps, your license should be installed.

How can I replace my

...

Marketplace license with

...

one provided by Xpand IT?

If you need to replace your marketplace Marketplace license with a license provided by Xpand IT, you should perform the following steps:

• On In your JIRA Jira Administration, select the option " Manage Add-Ons".
• Click on the Add-On "Xray". Detail about the add-on will app will be shown.
• Edit the license key, and remove all the text, then click on " Update". Your marketplace license has been removed.
• Go to " Xray Configuration ", section "> License Management".
• Select Click on the button "Click here to install a license directly from Xpand IT".
• Put Enter your license on in the text box. Remember, this shoud be a license provided by Xpand IT and , not via Atlassian Marketplace
• Click on addAdd.
• Your Xpand IT After the executing the previous steps, your license should be installed.

What happens

...

when my paid license expires?

If you have a paid license and it expires, you won't be able to upgrade Xray. You won't also be able to access our support.Note that , but you still be able to fully use Xray.

...

Yes. Even for trying Xray, you need a trial license. More information can be found in the respective Xray section in Atlassian's Marketplace.

...

Yes, up to 5 times. More information can be found in the respective Xray section in Atlassian's Marketplace.

Installation

What

...

Jira versions do you support?

Please see our Installation page.

Do you support

...

Jira Data Center (i.e., high availability)?

Yes.

...

How do I roll back to a previous version of

...

Xray?

If you need to rollback you roll back your installation to a previous version of the add-on, perform the following steps:

• Uninstall your current version of the add-on: app. Navigate to your JIRA Jira administration , section > Manage Add-ons , select the "Xray" add-on and click on "Uninstall"> Xray.
• Click Uninstall.
• Download the version that you want from https://marketplace.atlassian.com/plugins/com.xpandit.plugins.xray/versions (Click over click on the version and choose "download Download this version").
• In After executing the previous step, also in the Manage Add-Ons page, select " Upload Add-on ", and browse to the file downloaded in the previous step.
• The add-on previous version of the app should now be installed with the previous version.

What happens if I uninstall Xray?

...

Note: If you uninstall and install Xray once again, nothing special should happen. In other words, this it should be idempotent. 

How

...

do I enable debug

...

logging? 

If you need to enable logging for the add-onapp, you should perform the following steps:

• Go to the JIRA Jira Administration , section > System.
• Selection option "Select Logging & Profiling".
• In section " Default Loggers", click on "Configure logging level for another package".
• Set the Package name to com.xpandit and Logging Level to DEBUG and click on .
• Click Add.

Once you performed the previous steps, you should reproduce recreate your problem and retrieve the atlassian-jira.log file.

Import/Export

Import JSON execution results from

...

Cucumber returning error

 If you get a message like : “Error importing execution results to database: User XXX does not have permission to execute test execution with key TEST-XX” 

• Check Ensure that the User that upload who uploaded the result file have has the RESOLUTION permission 
• Check Ensure that Test Execution is not in a Workflow status marked as non-executable in Xray "Global Preferences"
  

Is it possible to import Tests and link them to other issues (e.g., requirements)?

Yes. You can add one or more columns in your CSV and then map them to "Link Tests"'. Please see the documentation of in FAQ.

Can I migrate my Tests from one

...

Jira server to another

...

?

Yes, with some constraints. Please see the proper section within the documentation of in FAQ.

How can I migrate my legacy executions to

...

Jira?

The built-in Test Case Importer is only able to import Test specifications and not results. There is no immediate way to achive what you need. However, there are some options.

One way eventualy would be by using is to use the REST API: FAQ. This will require some development effort from on your side and will also require that you have the Tests already created in JIRAJira.
Another option could be using is to use the built-in JIRA Jira CSV importer (not Xray's Test Case Importer) , in order to create the Test Executions with the Tests. However, you would still need to use the REST API in order to submit the results for the Tests contained in those Test Executions.

...

Can I trigger/start Jenkins/Bamboo builds from Xray?

Currently, this is not possible.

JQL functions

When searching for issues, custom fields (e.g., TestRunStatus) do not return the correct values.

Columns presented in the results do not take into account the context of the JQL query. The TestRunStatus custom field will always present the same result independently independent of the JQL query. 

For instance, imagine that tou Let's say you want to see the tests of a given Test Execution, using testExecutionTests(...), and their results. If a Test A is executed by TE 1TE1 and TE2 and you're filtering by TE2; , then you will get the "global" status calculated, and this can be the value in TE1 if TE1 is the latest test run.

...

We really prefer that you contact us through our Service Desk, so we the right person can properly address your question on time and by the right personasap.

I would like a demo

...

. Is that possible?

Sure. Please leave us a message in our Service Desk and we'll come get back to you right away. We are also publishing webinars have webinars (both live and recorded) that you can assist (you may also see a recorded webinar). attend.