Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Gliffy Diagram
nametest_environments
pagePin61


Note: If you don't use Test Environments, then only the latest result matters for assessing the current status of the Test.

...

ScenarioTest Environment(s) of TE 1Test Environment(s) of TE 2Test run status in TE1Test run status in TE2Calculated value for the overall, consolidated status of the Test (i.e. for the "All Environments")Other

A

AndroidiOSPASSPASSPASSThe test will be considered to be PASS in both Android and iOS environments.

B

iOSiOSPASSFAILFAIL

The test will be considered to be FAIL in iOS.

C

iOSiOSFAILPASSPASSThe test will be considered to be FAIL PASS in iOS.

D

iOS-FAILPASSFAILThe test will be considered to be FAIL in iOS and PASS for the empty environment.

E

--PASSFAILFAILThe test will be considered to be FAIL for the empty environment.

F

--FAILPASSPASS The test will be considered to be PASS for the empty environment.

...


StatusWhy?
windowsPASSdue to the last result obtained in "windows" environment on CALC-5262
macPASSdue to the last result obtained in the "mac" environment on CALC-52625263
edgechromeFAILdue to the last result obtained in "chrome" environment on CALC-5261
chromeedgePASSdue to the last result obtained in "edge" environment on CALC-5263

"All Environments"

(if analyzing the status of the test without identifying a specific environment)

FAILas the last result for one of the environments ("chrome") was FAIL (i.e. on CALC-5261)

...

Info
titleLearn more

For advanced Test Environment management capabilities, please check our Integration with Apwide Golive.

Don'ts

  • Don't create hundreds or thousands of Test Environments as it will harden their usage and add some performance overheadDon’t make composed environment names, such as ”<os>_<browser>_<stage>” as it will pollute the environment namespace and harden management
  • Don't try to do data-driven testing using Test Environments; they're not tailored for that

...

Info
titlePlease note

Besides other usage issues, if you have a large number of environments (>100>500), it will impact the calculations that need to be done and the size of the Lucene index.

Please try to have a limited, restricted, and well-defined list of Test Environments.