In a world with ever-growing DDoS attacks, L7 attacks give even the most experienced engineers headaches. Now imagine if instead of following easy-to-detect patterns, bots mimicked customer behavior. That’s exactly what Shopify sees every day during flash sales.
For small stores that release tiny numbers of sought-after products, those products are often resold for a huge profit, creating a situation where, for bad actors, it’s advantageous to buy as many products as quickly as possible. During flash sales, when milliseconds matter, bots buy faster than humans. These bots’ constant search for new products created a constant load on Shopify’s infrastructure and SREs—until the company decided to create an automated system to detect and block nearly all bot traffic on its load balancers.
Felix Glaser offers an overview of this system and shares the challenges Shopify faced differentiating between bots and humans. Bots act as headless browsers or browser extensions, rotate their user agent to appear as NATed users, and mimic human browsing as best they can. When the stakes are real customers unable to complete their checkouts, misclassification isn’t an option.
Join in to learn how Shopify used simple statistics, heuristics, and some reasonable thresholds to block bots on its online stores and cut down up to 50% of its traffic and developed a bot detection software that doubles as a robust DDoS protection system.
Felix Glaser is a production engineer at Shopify, where he works on networking and security-related applications. Previously, Felix ran and sold his own startup, room.me, which matched roommates. In his free time, he organizes and plays CTFs for fun.
©2017, O'Reilly Media, Inc. • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • firstname.lastname@example.org