Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Tests are always executed within a Test Execution context, even if it's an ad hoc execution. You can execute tests inline (i.e., define the test run status), directly in the Test Execution issue screen, or you can use the Execution Screen which is detailed extensively below.

 


A Test Execution is validated if all the Tests that verify the specific environment target passed. In order to execute, monitor the progress and report the results, each Test has its own Execution page details. Each Test associated with a Test Execution is called a Test Run. A Test Run is like an instance of the Test that contains a copy of the test definition (e.g., description, steps) and the result of that test in the context of that Test Execution .

...

The execution screen is composed by the following sections:

Table of Contents

Image Added

Image RemovedImage RemovedImage Added

Navigation bar

Always available, the navigation bar contains breadcrumbs to allow easy navigation between the Test issue hierarchy. It also contains a toolbar with the Return to Test Execution button that allows you to jump to the current Test Execution issue page, as well as the Next and Previous buttons, to quickly navigate to the execution screen of the other Tests included on the same Test Execution, if applicable.

Image Removed

 Image Added


Export Automated Test Definition

...

If you're executing a Cucumber Test, an option to export the Test as a feature file will appear.

Export Cucumber Test definitionImage RemovedImage Added

Generic

If you're executing a Generic Test, then an option to export the Test as text will appear.

Export Generic Test definitionImage Removed

 Image Added


Execution Status

...

  • Assignee -  the User assigned to perform the current test execution
  • Executed By - the last User that changed the status of the current test run
  • Versions - the target release version tested by the current test execution
  • Revision - the source code and documentation version used in the current test execution
  • Started On - the date and time the execution of the current test started
  • Finished On - the date and time the execution of the current test finished

Image RemovedImage Added

 


You can change the Test Run assignee directly on the field. It supports inline editing and once activated, you can filter by user and then select the one you wish to assign to the Test Run.

 


Image RemovedImage RemovedImage AddedImage Added


If the dates are configured to be edited, you can edit the Started On and Finished On dates date by clicking in the calendar icon and choosing the date and time. You can check your configurations by going to Jira Administration > Add-ons > Global Preferences.

Image RemovedImage Added

Note: The Jira jira.date.time.picker.java.format and jira.date.time.picker.javascript.format properties in Jira Administration advanced settings should belong to the same pair of properties in order for the datetime picker work properly. See the following link for more information

...

Info

This section is only visible if the "Separation of Concerns" option is disabled in Xray Requirement Requirements Coverage administration settings page.

Image RemovedImage Added


All affected requirement statuses will be updated whenever a step (or example in the case of Cucumber automated Tests) status is changed. The status for any affected requirement can always be changed by clicking on the status and choosing the desired option.

...

The Execution Defects field lists the overall defects related with the current execution. It is possible to create a new Defect issue, create a new Sub-task or Add existing Defects by clicking on the respective option.

Image RemovedImage Added

Execution Evidences

The Execution Evidences field lists the overall attachments related with the current execution and clicking on the Add Evidences, opens the Attach Files dialog box.

Image RemovedImage Added

Comment

The Comment field is inline editable to input overall appreciations about the current execution without reloading the page

The field supports the Jira markup wiki language.

Image RemovedImage Added

Test Description

...

The field supports the Jira markup wiki language.

Image RemovedImage Added

This section shows all links associated with the Test. 

Image RemovedImage Added

Test Issue Attachments

This section shows all attachments associated with the Test. 

Image RemovedImage Added

Pre-Condition

The Pre-Condition field shows the Pre-Conditions issues associated with the current Test.

Image RemovedImage Added

Pre-Condition Attachments

The Pre-Condition field also shows the Pre-Conditions attachment in a inline dialog.

Image RemovedImage Added

Manual Test Step Results Table

...

Note: Changing the Status on the each Step will affect the Execution Status field on the Execution Screen.

Image RemovedImage Added

Automated Test Details

An Automated Test execution screen contains the Automated Test Details field, which list the Steps to be used as reference by the external tool while testing.

Image RemovedImage Added

Examples

A scenario outline ofan automated Cucumber test will normally contain an examples table containing several inputs and outputs for the given scenario.

...

A scenario outline test is TODO if at least one of the examples did not run and all the other examples passed or todo.

Image RemovedImage Added

Results

Xray for Jira supports the import of multiple execution results for the same Test Run. Multiple execution for the same Test Run specify different results for different context/environments. Multiple executions are only supported for Automated Tests. Manual Tests do not have this concept.

...

  • Context: An execution context name (in the case of Xamarin Test Cloud results, the mobile device and operating system version) 
  • Error Message: A possible error message if the result has failed.
  • Duration: The complete duration of an execution result.
  • Status: The execution results status.


 Image Added

Image Removed

Attaching Screenshots

To attach a screenshot, paste the image from your clipboard using the relevant keyboard shortcut. (For Mac OSX, use CMD+V; for Windows, use Ctrl+V.)

Image RemovedImage Added

Enter a file name for the screenshot you are attaching (defaults to "screenshot-" concatenated with the next number available.) and choose where you want to place the evidence (Overall Execution or Test Steps).

Image RemovedImage Added

Click Upload to attach the captured image to your Test Run.

...

The Activity field lists all the activities related with the current execution.

Image RemovedImage Added

Updating the definition of the Test after execution started

When a Test definition has changed, a new dialog box will appear on the top of the Execution page informing you that the current execution is obsolete, and will prompt you with Reset or Merge buttons that, when pressed, will load the updated definition as well as delete all the execution data or merge the new Test definition into current state of execution. 

Image RemovedImage Added

Reset Execution

You can choose to reset the Execution or keep it for historic record. If you reset the execution, all information previously saved (status, evidences, defects) will be erased. 

Merge

...

Execution

You can choose to merge the new Test definition into the current status of execution. This will keep steps that weren't changed along with the associated results/comments/evidences. It will also load new steps (moved, erased or created), pre-conditions or attachments definitions. This will always force a recalculation of Test Run Status. For instance, if the current Test Run Status is FAIL because of a step and this step is removed from Test, if you merge, then the Test Run Status will change to whatever the value combination of the others steps returns.