Page History
Learn about optimizing coverage, traceability, and the E2E scenario count in TCD, taking Guidewire implementation as an example.
Table of Contents
Core Testing Challenge
While functional testing is a common entry point for Test Case Designer implementations and we have multiple articles describing the benefits achieved, integration testing (both system-to-system and E2E) is often even more applicable because of the increase in scale and number of dependencies (and, consequently, in number of important possible interactions).
...
We will describe how these benefits come to life on the generalized example of the Guidewire suite implementation at a large insurance client (specifically, “Auto Policy Bind and Rewrite” workflow).
Test Case Designer Solution & Modeling Process
Before we start the design, the following decisions need to be made with the feedback from all stakeholders involved in the process:
| Info |
|---|
Note 1: The variation % of 70 is used for example purposes and highly depends on the project infrastructure, rule complexity, etc. But that value is fairly common across the implementations we have seen. Note 2: The execution/design split is almost never 100 to 0 in either direction, so the decision is more between, e.g., 70/30 or 30/70. Note 3: Release history and changes to the functional/integration testing could alter the approach decisions throughout the project lifecycle. |
In this article, we will focus on a more “strategic” level (“Yes” for the first diamond) and will briefly touch upon the extra steps/considerations to achieve the “No -> Design” tree path.
Building a Model – Step 1 – Parameter & Value Selection
| Info |
|---|
...
| The general logic of “Parameter = a step in the flow that impacts system outcomes and can be represented with finite variation” still applies throughout the article, the notes below build on top of it. |
The general rules for E2E parameter & value identification:
...
Value expansions can further serve as test data specification, if needed:
| Info |
|---|
...
| For maintenance reasons, we recommend keeping TCD values at the category level (especially dates) and setting up execution in a way that would calculate the actual date/would query the credit database given the score range/etc. |
Repetitive model elements can be handled by appending the “extra detail” to the parameter name (e.g. "- Original State"):
...
This approach allows teams to communicate clearly and collaborate efficiently by confirming early in the testing process that 1) all critical aspects have been accounted for, and 2) they have been included at the right level of detail (which is one part of the Test Case Designer answer to “how much testing is enough?”).
Building a Model – Step 2 – Generating Optimal Scenarios
| Info |
|---|
...
| Implementing system logic via TCD constraints would occur at this step, but there are no aspects unique to E2E testing about constraint-handling, so we are skipping it. |
When it comes to efficiently covering all critical interactions in a system using as few test as possible, the TCD test generation algorithm will dramatically outperform humans in speed, accuracy, thoroughness, and number of tests used. Even if there are more than 10,000 critical interactions in a system, for example, the algorithm will cover every single one, guaranteed. And do so in just several minutes using as few tests as mathematically possible. No human brain could come close. And production data wouldn’t come close.
...
And next, we will discuss the last piece of the core testing “puzzle” – given the total scope, how we can use TCD visualizations to select the right stopping point.
Building a Model – Step 3 – Coverage Results Comparison
When end-to-end scenarios are created by hand, they often represent a very fragmented view of the system and struggle with redundancy or omissions. Instead, Test Case Designer maximizes the interaction coverage in fewer scenarios and provides complete control and traceability for each of the steps in the test case.
...
Taking this analysis a step further, given typical schedule deadlines, etc., we can identify the exact subset of the total scope that will be sufficient for the immediate testing goals and communicate that decision clearly to the management with the combination of the Mind Map + Coverage Matrix.
Building a Model – Step 4 – Scripting & Export
| Info |
|---|
...
| Some teams may choose to execute directly from the test cases table. They would leverage “Save As” dropdown on the Scenarios screen and skip this section. |
We are observing more & more teams switching to BDD, so we will be covering TCD Automate in this article, but most general principles also apply to Manual Auto- Scripts.
First, the overall script structure is completely up to you. The number of steps, length of each, number of parameters per each, etc. depend on your guidelines for both test design and execution – Test Case Designer has the flexibility to support a wide range of preferences.
Second, for the review and export efficiency, we will be using {[]} filters to separate Full Term and New Business scenarios (assuming they have different validation steps for example purposes).
| Info |
|---|
...
| You can check the “Usage” button on the Automate screen |
...
| for more details about the syntax rules. |
The sequential time aspect can be accounted for by “On Day X,…” parts of the steps. The system-to-system transition can be reflected in a similar manner as well as in commented-out lines. Parameters that didn’t “qualify” for model inclusion and static validations can be hard coded (i.e. you don’t need to include <> syntax in every line). Test data generated during the execution can be captured using steps like this:
...
Lastly, we strongly recommend sharing the TCD models with automation engineers early in the process to allow them enough time to review and provide feedback on step wording, value names, etc.Note: if
| Info |
|---|
| If present, only value expansions are exported in the script (the same is true for CSV), so you can have abstract/categorical value names and then provide execution-ready details in the expansion. |
Once the script is finalized, you can proceed to export the scenarios in the format compatible with the test management tools and/or automation frameworks. E.g., without any extra actions, you can generate the CSV for Xray alongside Java files.
...
- Rapidly create clear, consistent steps that leverage Behavior Driven Development principles.
- Export 1 Scenario block into multiple scripts based on the references to the data table.
- Improve collaboration across business and technical teams to understand the testing scope as a group.
Building a Model – What is Different for “No -> Design” decision tree path
The extension from “strategic” to “highly detailed” still follows the same steps, but there are 3 nuances.
...
Lastly, additional model elements may require more precise scripting (if conditional or “vague” steps are not an option). It becomes even more important to keep track of 1) which {[]} filters are used; 2) how mixed-strength affects the possible combinations of {[]} filters (you may not need to create a scenario block for each possible combo).
Summary & Case Studies
In conclusion, the combination of TCD features will allow you not only to quickly generate the optimal set of scenarios but also to answer the “how much testing is enough?” question with clarity & confidence.
...