Holiday Readiness Performance Testing

Purpose of the article: The blog talks about the paradigm shift in Retail Industry, challenges faced by Retailers, key drivers shaping the digital future trends, strategies and approach to be followed by the Retailers, the different types of Performance Tests to be executed and the key-metrics to be captured.

Intended Audience: Retail eCommerce giants, Web-application owners who ought to ensure that their applications are ready for the holiday seasons – which would yield better conversion rates and profits.

Tools and Technology: Performance Testing Tools (Microfocus Load Runner, Apache JMeter, Google Page Speed, Java, AngularJS, Front-end tools like Chrome Dev tools, Lighthouse, Gremlin, Chaos Monkey).

Key words: Retail Performance Testing, Holiday Readiness, CDN, Chaos Engineering, Outages, Amazon, Flipkart, faster web pages, Faster web-pages.


Technology has completely revolutionized the way we shop and live. More to say, the COVID-19 pandemic is reshaping the way people shop and spend for the holiday season in more ways than one. The full-blown potential of high-compute mobile devices with very high-speed internet bandwidths is being utilized to fulfill the needs of today’s millennial consumer behavior. It is just easier and faster to get your groceries delivered to your home just by the click of a mouse than the traditional way of shopping by standing in a queue and getting your items added to a cart.

According to a survey by, 90% of consumers spent more time shopping online since the COVID-19 pandemic began. While 87% of consumers opted for in-store shopping during the holiday season in 2019-20, this has changed with 60% of consumers indicating plan to go in-store shopping in 2020-21.

Added to it, the big retail giants with their yearly holiday shopping spree events like Amazon prime-day, Thanksgiving days, Black Friday, Cyber Mondays, Boxing days, etc., have revolutionized the way online consumer shopping happens. These spurt duration events provide an excellent opportunity for the retailers to level up their sales by 5X or 10X etc. So, what are the various challenges, approaches, and strategies to succeed in this Holiday Readiness preparation?

Retail Dive reported that online sales reached $9 billion on Black Friday 2020, 21.6% ahead of 2019. Salesforce tracked them at 12.8 billion and 23% growth.

Typical Challenges Faced by Retailers

  • Global presence, e-commerce, and online shopping trends
  • To provide a compelling, high-performant and secure interface to the online shoppers
  • To sustain the competition with attractive sales / offers
  • To develop a highly attractive product campaign and elegant website
  • Supply chain and ever-changing retail suppliers & vendors add another layer
  • To ensure that there are no payment processing delays
  • To see that there are no Security glitches while processing transactions via e-commerce channels
  • To ensure that POS systems are performant enough
  • Mobile App performance negatively affecting customer experience
  • Increased/Heavy app loading time
  • Legacy applications supporting massive merchandise data
  • 24/7 Uptime for POS application
  • Relatively new concept of online grocery
  • To retain the highest consumer satisfaction

The Paradigm Shift in Retail Industry

Unprecedented Traffic Spikes to Come – The COVID-19 pandemic has a tremendous ripple effect on every industry, and it is evident in the retail sector more so than ever. We have seen a tremendous increase in online retailing in 2020 & continuing thus far: fewer in-store purchases, more web and mobile traffic, etc. Peak shopping days (Black Friday, Cyber Monday, Boxing Day, Amazon Great Indian Sale Flipkart Big Billion Days), frontend traffic is upwards of 3X more than usual — and backend load increases by 10X or more.

Slow Is the New Downtime – Customer expectations are higher than ever. According to a report by Salesforce, 57% prefer moving to a new seller for a better purchase experience offered by a competitor, while 40% look for alternatives owing to a poor mobile experience. A 2019 ITIC report found that loss of revenue for 98% of companies due to an hour of downtime could be at least $100,000. And 86% of businesses say that the cost for one hour of downtime is $300,000 or higher. The impact is even more significant during the holiday shopping season, which is more than 30% of annual retail revenue. When retailers move to a multi-cloud or hybrid environment, they have must ensure that performance does not regress.

Key Drivers Shaping the Digital Trends of the Future

Increasing number of internet and social media users – The average number of internet and social media are ever-increasing compared to 5 years ago.

Increasing number of smartphone users and increasing m-commerce – According to Statista, 2019, experts forecast that in 2021, 53.9% of the total annual retail e-commerce sales in the US will come from mobile commerce. In other words, more than half of the money US consumers will leverage smartphones and tablets for online shopping.

Favorable Demographics – As per Deloitte findings in Indian Chamber of Commerce, India has the world’s largest millennial (aged 18-35 years) population that is tech-savvy and prefers digital means for communication and shopping. They account for 46% of its workforce and a third of the country’s population.

Digital Infrastructure – Rapid development of digital infrastructure and digital money transfer options would increase presence of Omni-channel retailers. The boundaries between virtual and physical retailers are already collapsing.

Strategies & Approach for Retailers to Gear-up for the Holiday Season

Making faster web pages yields better conversion rates: It is a known fact that a 2 second slow response time page invites 4X more opportunities to the competition. At, a load time improvement of one second increases the conversion rate by 2%. trimmed 1 second from its median load time and improved its conversion rate by 10%.

Techniques to speed up the loading of pages:

  • Enabling Gzip compression to speed up the website’s load times significantly
  • Developing super-fast and ultra-attractive landing pages to sustain the competition
  • Facilitate Lazy Image loading so that the images get loaded at the user’s viewpoint – thereby helping in the faster page display
  • Minify CSS, JavaScript and HTML
  • Reduce Redirects
  • Leverage Content Delivery Network (CDN), which would cache the pages and display them faster without retrieving from the servers every time
  • Eliminate unnecessary plugins
  • Google AMP Pages will significantly increase the rankings of the site

Execute end-to-end Performance Testing

1 Gather the Right Workload Models: Defining the right workload models makes half of the performance engineer’s life easier, because the right candidates get performance tested and help in early fixing of bottlenecks, if any.

2 Deploy the Right Monitoring Tools: With the ever-increasing rise of distributed architecture consisting of Microservices and cloud architectures, IT systems are growing increasingly complex. So, it mandates deploying the right monitoring tools, which would discover the end-to-end components in the architecture, thereby helping in detecting the performance bottlenecks much early in the lifecycle. But not to forget, the greatest tool a performance engineer would have are not just the tools but the mindset to identify the performance problems and fix them well.

3 Execute Performance Tests for Open-Load and Closed-Load Conditions: Giant retailers usually face the challenge of Open Load conditions where they face huge unpredictable load traffic hitting their sites on the D-day. The retailers must be prepared for both the load conditions. It is recommended to test with a minimum of 3X load conditions than the previous year to be better equipped to handle the unprecedented spike of user loads.

4 Include Last-mile Load Testing Conditions: Including Last-mile Load Testing conditions would significantly help optimize the applications’ performance multi-fold by leveraging the right tools/infrastructure.

5 Performance Testing the Payment Gateways: In earlier days, stubs were used to test the performance gateways and it used to have its own challenge once it goes live. So, using Virtualization software etc., to cover end-to-end non-functional aspects would help simulate the performance issues in the right way.

Employ Chaos Engineering Mechanisms: As per Gremlinthe leading Chaos engineering tool vendor, “The heterogenous application architecture & infrastructures are posing a much greater challenge than ever. We all depend on these systems more than ever, yet failures have become much harder to predict. The outages hurt customers trying to shop, transact business, and get work done”. It is a known fact that one outage can cost a company millions of dollars.

Hence employing Chaos Engineering mechanisms goes a long way and it is a preventive medicine. In a nutshell, Chaos Engineering is a systematic and disciplined approach to identify failures before they become outages. They help fix failures before they end up in the news.

Recommended Performance Testing Types

  • Load
  • Stress
  • Spike
  • Performance Tests in Production

Key Metrics

  • Orders per minute
  • Page Views / hour
  • Sessions / hour
  • Errors
  • Response Time (Average. & 90th percentile)

Major website outages:

Salesforce – Source – –

AWS Outage:

fifty ecommerce brands tracked by web optimization software company Yottaa experienced site outages or performance issues during the cyber 5, or Thanks giving Day through Cyber Monday.

Contact Details:

Sravanthi Naga,

Performance Practice Head (Senior Manager)


Watch On-Demand Webinar

    Leave A Comment

    Related Post