Everything open source
May 16–17, 2016: Training & Tutorials
May 18–19, 2016: Conference
Austin, TX

Load balancing is impossible

Tyler McMullen (Fastly)
2:40pm–3:20pm Wednesday, 05/18/2016
Location: Ballroom E Level: Intermediate
Average rating: ****.
(4.36, 11 ratings)

Prerequisite knowledge

Attendees should have basic knowledge of distributed systems, networking, and monitoring.


Load balancing is something most of us assume is a solved problem. But the idea that load balancing is “solved” could not be further from the truth. If you use multiple load balancers, the problem is even worse. Most of us use “random” or “round-robin” techniques, which have certain advantages but are highly inefficient. Others use more complex algorithms like “least-conns,” which can be more efficient but have horrific edge cases. “Consistent hashing” is a very useful technique but only applies to certain problems.

There are several factors that exist both in theory and practice that make efficient load balancing an exceptionally hard problem, including Poisson request arrival times, exponentially distributed response latency, and oscillations when sharing data between multiple load balancers. Luckily, there are techniques and algorithms that have been developed that can make life better. Tyler McMullen explains some of the ways that we can do better than “random,” “round-robin,” and naive “least-conns,” even with distributed load balancers.

Photo of Tyler McMullen

Tyler McMullen


Tyler McMullen is CTO of Fastly, where he is responsible for the system architecture and leads the company’s technology vision. As part of the founding team, Tyler built the first versions of Fastly’s instant purging system, API, and real-time analytics. Previously, Tyler worked on text analysis and recommendations at Scribd. A self-described technology curmudgeon, Tyler has experience in everything from web design to kernel development and loathes all of it. Especially distributed systems.

Comments on this page are now closed.


Picture of Sergey Stepanov
Sergey Stepanov
06/07/2016 2:44am CDT

Hi Tyler, can you please, share the slides? I attended your talk, but did not record those balancing algorithms which you have mentioned, except round robin and the JSQ (I guess).
My email is serg.stepanov@yahoo.com. Thank you!