Page History
...
Using multiple environments at the same time
...
- You may have multiple semantics for environments (e.g. browser, operating system, testing stage); in theory, you can think as it being something multidimensional
- You could name the environment such as “windows_edge” or “mac_chrome” but…
- You could have a ton of composed environments which wouldn’t be manageable at all
- You couldn’t analyze coverage just from the perspective of one of those variables (e.g. “mac” or “chrome”); you would be restricted to analyze it from the perspective of the composed environment
- Just assign each variable value (e.g. name of operation system, name of browser) as an individual environment
How to
Assign each environment (e.g. name of operation system, name of browser vendor) as you do for a single environment; in other words, just add the multiple environment names as multiple, distinct labels.
Whenever creating a Test Execution:
Whenever updating an existing Test Execution:
Example
Test executed in the context of Test Execution assigned to several environments at the same time
- windows, chrome (fail)
- windows, edge (pass)
- mac, edge (pass)
The calculated status of the test, per Test environment, will be the following.
Status | Why? | |
---|---|---|
windows | PASS | due to the last result obtained in "windows" environment on CALC-5262 |
mac | PASS | due to the last result obtained in the "mac" environment on CALC-5262 |
edge | FAIL | due to the last result obtained in "chrome" environment on CALC-5261 |
chrome | PASS | due to the last result obtained in "edge" environment on CALC-5263 |
"All Environments" (if analyzing the status of the test without identifying a specific environment) | FAIL | as the last result for one of the environments ("chrome") was FAIL (i.e. on CALC-5261) |
Tips and Recommendations
Do's
- Use Test Environments only if you want to run the same Test on case in different environments and track results per each environment. If you do use it, then assign just one Test Environment per Test Execution.
- Simplify the names of Test Environments (i.e. lowercase it, shorten it)
- Example: macOS => mac
- Evaluate if you really need to assign multiple environments at the same time; using just one is preferable if you can afford that simplicity
Don'ts
- Don't create dozens or hundreds of Test Environments as it will harden their usage and add some performance overhead
- Don’t make composed environment names, such as ”<os>_<browser>_<stage>” as it will pollute the environment namespace and harden management
- Don't try to do data-driven testing using Test Environments; they're not tailored for that
Info | ||
---|---|---|
| ||
Besides other usage issues, if you have a large number of environments (>>10), it will impact the calculations that need to be done and the size of the Lucene index. Please try to have a limited, restricted and well-defined list of Test Environments. |
- You may filter using Test Environments in your Test Executions panel to see how the executions are doing, per environment.
Info | ||
---|---|---|
| ||
For advanced Test Environment management capabilities, please check our Integration with Apwide Golive. |
...