Page History
...
This analysis can be done using the Traceability Report or the Overall Coverage Report, among others.
Depending The exact behavior upon choosing a specific Test Environment depends on the report , by specifying a specific Test Environment you will be filtering and showing only ; this is valid for the "Traceability Report", "Test Executions List Report" or "Test Runs List Report", for exampleitself but, either explicitly or implicitly, Test Runs will be filtered by the selected Test Environment and reports will reflect it.
Traceability Report being used to analyze the results on the "edge" test environment.
...
Sometimes, you may have multiple semantics for environments categorizations for a given environment; in theory, you can think as it being something multidimensional.
Consider a very basic example: whenever performing web/UI based testing you will be using a browser and an operating system and you may want to analyze the results per a browser perspective or per an operating system perspective.
The recommended way to deal with environments having multiple dimensions is to treat each dimension (e.g. browser name, operating system , testing stage); in theory, you can think as it being something multidimensional
...
name) individually. In other words, add the values of each dimension to the "Test Environments" field separately.
Whenever you assign "mac" and "edge" to the Test Environments of a given Test Execution, it's equivalent to saying that your Test Run is scheduled for/was run in the "mac" and also in the "edge" environment.
This approach will limit the number of environments to the total number of possible values for each dimension, as opposed to having <number_of_values_dimension_1>*<number_of_values_dimension_2>*... environments.
The drawback of this solution is that you won't be able to analyze the results for an environment tagged as "mac" and "edge" at the same time, for example; you can just analyze results from a specific dimension.
| Info | ||
|---|---|---|
| ||
One way to deal with these kinds of environments would be to flatten them and treat them as usual, i.e. you could name the environment such as “windows_edge” or “mac_chrome” but…
|
...
|
...
|
How to
Assign each environment (e.g. name of operation system, name of browser vendor) as you do for a single environment; in other words, just add the multiple environment names as multiple, distinct labels.
...
- Use Test Environments only if you want to run the same Test case in different environments and track results per each environment.
- Simplify the names of Test Environments (i.e. lowercase it, shorten it)
- Example: macOS => mac
- Evaluate if you really need to assign multiple environments at the same time; using just one is preferable if you can afford that simplicity
| Info | ||
|---|---|---|
| ||
For advanced Test Environment management capabilities, please check our Integration with Apwide Golive. |
Don'ts
- Don't create dozens or hundreds of Test Environments as it will harden their usage and add some performance overhead
- Don’t make composed environment names, such as ”<os>_<browser>_<stage>” as it will pollute the environment namespace and harden management
- Don't try to do data-driven testing using Test Environments; they're not tailored for that
...
| Info | ||
|---|---|---|
| ||
Besides other usage issues, if you have a large number of environments (>>10), it will impact the calculations that need to be done and the size of the Lucene index. Please try to have a limited, restricted and well-defined list of Test Environments. |
- You may filter using Test Environments in your Test Executions panel to see how the executions are doing, per environment.
| Info | ||
|---|---|---|
| ||
For advanced Test Environment management capabilities, please check our Integration with Apwide Golive. |
Test Environments and the TestRunStatus custom field
...