...

Using multiple environments at the same time

...

• You may have multiple semantics for environments (e.g. browser, operating system, testing stage); in theory, you can think as it being something multidimensional
• You could name the environment such as “windows_edge” or “mac_chrome” but…
• You could have a ton of composed environments which wouldn’t be manageable at all
• You couldn’t analyze coverage just from the perspective of one of those variables (e.g. “mac” or “chrome”); you would be restricted to analyze it from the perspective of the composed environment

• Just assign each variable value (e.g. name of operation system, name of browser) as an individual environment

How to

Assign each environment (e.g. name of operation system, name of browser vendor) as you do for a single environment; in other words, just add the multiple environment names as multiple, distinct labels.

Whenever creating a Test Execution:

Image Added

Whenever updating an existing Test Execution:

Image Added

Example

Test executed in the context of Test Execution assigned to several environments at the same time

1. windows, chrome (fail)
2. windows, edge (pass)
3. mac, edge (pass)

Image Added

The calculated status of the test, per Test environment, will be the following.

Tips and Recommendations

Do's

• Use Test Environments only if you want to run the same Test on case in different environments and track results per each environment. If you do use it, then assign just one Test Environment per Test Execution.
• Simplify the names of Test Environments (i.e. lowercase it, shorten it)
  • Example: macOS => mac
• Evaluate if you really need to assign multiple environments at the same time; using just one is preferable if you can afford that simplicity

Don'ts

• Don't create dozens or hundreds of Test Environments as it will harden their usage and add some performance overhead
• Don’t make composed environment names, such as ”<os>_<browser>_<stage>” as it will pollute the environment namespace and harden management
• Don't try to do data-driven testing using Test Environments; they're not tailored for that

• You may filter using Test Environments in your Test Executions panel to see how the executions are doing, per environment.

...