What is a test environment

Generically speaking, a test environment is an environment that contains all necessary elements, including the SUT, so you can perform testing on it.

Depending on your context, a test environment may represent:

• a testing stage (e.g. "development", "staging", "preproduction", "production")
• a device model or device operating system (e.g. "Android", "iOS")
• an operating system (e.g. "Windows", "macOS", "Linux")
• browser (e.g. "Edge", "Chrome", "Firefox")

Some tests may be tailored for or only run in some environments.

Whenever performing testing on different environments, results 

Test Environments in Xray

In Xray, Test Environments are focused on the execution aspect, providing the means to schedule tests and analyze their results in different environments.

Thus, they're explicitly associated with Test Execution issues.

Overview

The status of a Test  (i.e., TestRunStatus) is generally calculated by looking at the Test's last execution (more detail here). However, this does not work well if you execute the same Test in two different test environments (e.g., devices) and you want the two results to be consolidated. 

...

Using multiple environments at the same time

...

Sometimes, you may have multiple semantics for environments (e.g. browser, operating system, testing stage); in theory, you can think as it being something multidimensional

• You could name the environment such as “windows_edge” or “mac_chrome” but…
• You could have a ton of composed environments which wouldn’t be manageable at all
• You couldn’t analyze coverage just from the perspective of one of those variables (e.g. “mac” or “chrome”); you would be restricted to analyze it from the perspective of the composed environment

...

The calculated status of the test, per Test environment, will be the following.

Tips and Recommendations

Do's

...