Multi-Marketplace E-commerce Platform with Automated Order Processing at 100,000 Orders per Hour During Mega Sales

1. Business Context & Challenges

During Mega Sale events (11.11, 12.12), Shopee, TikTok Shop and Lazada flood sellers with massive order volumes within seconds. An Gia Consumer's legacy monolithic system processed orders one by one over synchronous HTTP, creating severe bottlenecks:

1.1. Webhook endpoint crashes and lost order data

When Shopee and TikTok Shop fired webhooks simultaneously at up to 15,000 requests per second, the legacy API gateway exhausted its connection pool, returning 502 Bad Gateway and 504 Gateway Timeout errors and losing thousands of customer orders.

1.2. Warehouse operations stalled by slow pick list and waybill printing

Fetching tracking numbers from carriers (GHN, Viettel Post) was so slow that the 200 workers at the central warehouse waited hours to print waybills, risking late deliveries and store downgrade penalties from the marketplaces.

1.3. Soaring yet inefficient cloud server costs

Without auto-scaling based on actual load, the company paid hundreds of millions of VND to rent high-spec servers for an entire month just to cover 3 peak sale days.

2. SUDO's Tailor-Made Solution Architecture

Sudo fully re-architected the order ingestion and distribution layer using lightweight Golang services and Apache Kafka message queues:

2.1. Golang webhook gateway handling millions of requests

A Webhook Ingestion microservice written in Go with asynchronous goroutines returns HTTP 200 to the marketplaces in just 2ms and immediately pushes order data into the Kafka queue.

2.2. Smart Kafka topic partitioning to prevent order conflicts

Kafka partitioning by store ID and warehouse ID lets hundreds of workflows run in parallel without resource contention or duplicate orders.

2.3. Automatic pod scaling on Kubernetes (K8s HPA)

The infrastructure runs on a Kubernetes cluster with HPA (Horizontal Pod Autoscaler). When order volume surges, processing pods scale automatically from 4 to 40 within 15 seconds and are released automatically once the peak window ends.

2.4. Batching for carrier API integration

Instead of requesting a waybill number for each order individually, the system groups orders into batches of 50-100 and sends them to the carrier in a single network call, making waybill printing 4 times faster.

3. Measured Results & Before-and-After Comparison

After going live through the two biggest Mega Sale events of the year, the system delivered impressive results:

• Peak order processing capacity: A smooth 100,000+ orders/hour (8.3 times the previous ceiling of 12,000 orders/hour).
• Orders dropped due to webhook server errors: 0%, preserving 100% of Mega Sale order data.
• Time to print waybills for 50,000 orders: Reduced from 6-8 hours to just 1.5 hours (4 times faster).
• Cloud infrastructure budget: 45% Cloud cost savings thanks to real-time elastic pod scaling.

4. What the Client's Leadership Says

"During the recent 11/11 Mega Sale, we processed a record of more than 350,000 orders without a single minute of downtime or a single dropped order. Our warehouse team could print waybills continuously from the moment the sale opened, helping An Gia achieve a 99.6% on-time delivery rate and rank No. 1 in the FMCG category on the marketplace. Sudo's solution delivers value you can measure in real revenue."

Ms. Le Hoang Yen, E-commerce Operations Director, An Gia Consumer

5. Strategic Recommendations for Mid-Sized & Large Enterprises

Based on An Gia Consumer's real-world challenge, Sudo's engineers recommend:

1. Multi-marketplace order processing must be asynchronous: Never let the order intake flow wait on waybill printing or calls to third parties.
2. Adopt Kubernetes auto-scaling: It is the most effective way to absorb sudden load spikes while keeping monthly Cloud costs under control.
3. Optimize batching: Group requests when communicating with shipping partners and payment gateways to avoid external bottlenecks.