Page History

Learn about optimizing coverage, traceability, and the E2E scenario count in DesignWiseTCD, taking Guidewire implementation as an example.

• Core Testing Challenge
• DesignWise Solution & Modeling Process
  • Building a Model – Step 1 – Parameter & Value Selection
  • Building a Model – Step 2 – Generating Optimal Scenarios
  • Building a Model – Step 3 – Coverage Results Comparison
  • Building a Model – Step 4 – Scripting & Export
• Summary & Case Studies

Core Testing Challenge

While functional testing is a common entry point for DesignWise Test Case Designer implementations and we have multiple articles describing the benefits achieved, integration testing (both system-to-system and E2E) is often even more applicable because of the increase in scale and number of dependencies (and, consequently, in number of important possible interactions).

...

In such situations, teams use DesignWise TCD to quickly determine the optimal answers to both (1) “how many tests?” and (2) “which specific tests?”.

We will describe how these benefits come to life on the generalized example of the Guidewire suite implementation at a large insurance client (specifically, “Auto Policy Bind and Rewrite” workflow).

...

Test Case Designer Solution & Modeling Process

Before we start the design, the following decisions need to be made with the feedback from all stakeholders involved in the process:

...

In this article, we will focus on a more “strategic” level (“Yes” for the first diamond) and will briefly touch upon the extra steps/considerations to achieve the “No -> Design” tree path.

Building a Model – Step 1 – Parameter & Value Selection

Note: the general logic of “Parameter = a step in the flow that impacts system outcomes and can be represented with finite variation” still applies throughout the article, the notes below build on top of it.

...

Such nesting also allows test designers to “connect” DesignWise plans in TCD models in a way, since the same profile can be reused in the models responsible for parallel (create Umbrella policies) or sequential (test Renewals later) steps.

...

Note: for maintenance reasons, we recommend keeping DesignWise TCD values at the category level (especially dates) and setting up execution in a way that would calculate the actual date/would query the credit database given the score range/etc.

...

For convenience, we suggest keeping the parameters in the flow order on the first screen. For the same purpose, elements like this can serve as visual dividers on the Parameters/Scenarios screens and are optional:

Once the draft plan model is created, the auto-generated Mind Map presents an intuitive view of the model elements and can be easily shared for collaboration and approval.

...

This approach allows teams to communicate clearly and collaborate efficiently by confirming early in the testing process that 1) all critical aspects have been accounted for, and 2) they have been included at the right level of detail (which is one part of the DesignWise answer Test Case Designer answer to “how much testing is enough?”).

Building a Model – Step 2 – Generating Optimal Scenarios

Note: implementing system logic via DesignWise TCD constraints would occur at this step, but there are no aspects unique to E2E testing about constraint-handling, so we are skipping it.

When it comes to efficiently covering all critical interactions in a system using as few test as possible, the DesignWise TCD test generation algorithm will dramatically outperform humans in speed, accuracy, thoroughness, and number of tests used. Even if there are more than 10,000 critical interactions in a system, for example, the algorithm will cover every single one, guaranteed. And do so in just several minutes using as few tests as mathematically possible. No human brain could come close. And production data wouldn’t come close.

...

This is how they would look inside DesignWiseTest Case Designer:

For the factors which are not explicitly specified, you should preferably let the DesignWise TCD algorithm handle them. In other words, if your plan model contains 10 parameters but the special edge-case requires 4 specific values, then specify only those 4. DesignWise TCD will fill in the blanks automatically and, while doing so, identify the most varied, highest-coverage, least-wastefully repetitive 6 values possible.

Pro tip: one subtle trick for one-off testing in DesignWise TCD is that Forced Interactions can overwrite Constraints, and vice versa. You can use that workaround for the scenarios that are deemed “very low probability” by the business but are still required to be tested from an IT standpoint.

...

The last point at this step is to select an appropriate level of thoroughness for your needs. It is rare for E2E DesignWise TCD models to utilize anything other than 2-way (at the “Strategic” level) or Mixed-strength (at any level). 3-way or higher coverage strengths would typically be overkill. The dropdown settings in Mixed-strength are generally chosen based on the following parameter logic:

...

The effective combination of the level of detail in Parameters and the business-relevant coverage strength in Scenarios guarantees that the DesignWise Test Case Designer algorithm optimizes your total model scope to have a minimal number of tests that cover all the important interactions.

And next, we will discuss the last piece of the core testing “puzzle” – given the total scope, how we can use DesignWise TCD visualizations to select the right stopping point.

Building a Model – Step 3 – Coverage Results Comparison

When end-to-end scenarios are created by hand, they often represent a very fragmented view of the system and struggle with redundancy or omissions. Instead, DesignWise maximizes Test Case Designer maximizes the interaction coverage in fewer scenarios and provides complete control and traceability for each of the steps in the test case.

If we now analyze the coverage achieved across, e.g., 8 critical parameters and compare it with the typical manual solution, the results would often look like this:

As you can see, DesignWiseTCD-generated tests benefit from Intelligent Augmentation that ensures coverage of both (1) all specified requirements and (2) every critical system interaction. Our scenarios consistently find more defects than hand-selected test sets because interactions are a major source of system issues.

Taking this analysis a step further, given typical schedule deadlines, etc., we can identify the exact subset of the total scope that will be sufficient for the immediate testing goals and communicate that decision clearly to the management with the combination of the Mind Map + Coverage Matrix.

Building a Model – Step 4 – Scripting & Export

Note: some teams may choose to execute directly from the test cases table. They would leverage “Save As” dropdown on the Scenarios screen and skip this section.

We are observing more & more teams switching to BDD, so we will be covering DesignWise TCD Automate in this article, but most general principles also apply to Manual Auto-Scripts.

First, the overall script structure is completely up to you. The number of steps, length of each, number of parameters per each, etc. depend on your guidelines for both test design and execution – DesignWise Test Case Designer has the flexibility to support a wide range of preferences.

...

Lastly, we strongly recommend sharing the DesignWise TCD models with automation engineers early in the process to allow them enough time to review and provide feedback on step wording, value names, etc.

...

• Rapidly create clear, consistent steps that leverage Behavior Driven Development principles.
• Export 1 Scenario block into multiple scripts based on the references to the data table.
• Improve collaboration across business and technical teams to understand the testing scope as a group.

Building a Model – What is Different for “No -> Design” decision tree path

The extension from “strategic” to “highly detailed” still follows the same steps, but there are 3 nuances.

First, the “hard” DesignWise limits are 200 Test Case Designer limits are 256 parameters and 4000 5000 tests per model. Highly detailed models will require you to consider non-traditional DesignWise TCD parameters (e.g. test data elements, more expected results) that can exhaust the limit fairly quickly, so the prioritization of the scope and the balance between design & execution becomes even more critical.

...

Lastly, additional model elements may require more precise scripting (if conditional or “vague” steps are not an option). It becomes even more important to keep track of 1) which {[]} filters are used; 2) how mixed-strength affects the possible combinations of {[]} filters (you may not need to create a scenario block for each possible combo).

Summary & Case Studies

In conclusion, the combination of DesignWise TCD features will allow you not only to quickly generate the optimal set of scenarios but also to answer the “how much testing is enough?” question with clarity & confidence.

The image above should be familiar from our other educational materials, and hopefully, it underscores the notion that the process & methodology are not strongly dependent on the type of testing, type of system, industry, etc.

The goal of applying DesignWise Test Case Designer is to deal with such challenges of manual test creation as prolonged and error-prone scenario selection, gaps in test data coverage, tedious documentation, and excessive maintenance.

...