Apache Traffic Server - HTTP Proxy Server on the Edge

Operations Ballroom CD
Average rating: ***..
(3.21, 14 ratings)

Apache Traffic Server is a fast, scalable and feature rich HTTP proxy and caching server. Traffic Server was originally a commercial product from Inktomi corporation, and has been actively used inside Yahoo! for many years, as well as by many other large web sites. As of 2009, Traffic Server is an OpenSource project under the Apache umbrella, and is rapidly being developed and improved on by an active community.

This talk will explain the details behind the Traffic Server technology; What makes it fast? Why is it scalable? And how is it different compared to other HTTP proxy servers? We will also delve into details as to how a large web site can utilize this power to create services with exceptional end-user experience.

In a nutshell, Traffic Server is designed using a hybrid processing model, combining an event driven engine, with a multi-threaded process approach. This allows Traffic Server to scale on modern multi-core systems, taking advantge of available CPUs. Each thread (typically 2-3 per CPU) runs its own event driven state machine, which implies that each thread handles many client connections, i.e. an m-to-n model. Limiting the number of worker threads avoids the resource problems associated with thousands or tens of thousands of threads.

Being an HTTP proxy server and cache, there are many potential use cases, e.g.

  • Forward proxy, e.g. inside a corporate firewall
  • Reverse proxy, a.k.a. HTTP accelerator
  • Intelligent traffic and connection management
  • Layer 7 (HTTP) routing, e.g. URL hashing

In this presentation, we will focus on three such use cases at Yahoo!, all which rely entirely on Traffic Server:

CDN: CDNs, or Content Delivery Network, are sometimes seen as an expensive and complicated infrastructure to own and operate yourself. We will show how a large Web site can deploy and manage its own CDN, using Traffic Server as the engine delivering hundreds of thousands of objects per second.

Connection manager: TCP is a reliable network protocol, but it comes at a price. Establishing a new TCP connection requires a 3-way handshake, which takes at least a full round-trip to complete. The longer the latency between the client and the server, the longer this handshake will take. TCP also uses a congestion control mechanism called slow-start, which further introduces latency by effectively throttling traffic on a new connection. Using Traffic Server deployed globally, we can eliminate a significant portion of the latency introduced by TCP and HTTP. Reducing this latency is obviously beneficial to the users, giving faster page-load times.

Layer 7 routing: For a large site, with terrabytes (or more) of data, and thousands or millions of users, having good data and cache locality can be vital. The closer the data is to the web server, the faster the pages can be rendered. Using Traffic Server, we can route each request to the optimal backend server, and these routing decisions are done on the fly. This is similar to how hardware Server Load Balancers (SLB) can be configured and deployed. Using commodity hardware and Apache Traffic Server, we can achieve the same (or better) results at a fraction of the cost compared to hardware SLBs.

Photo of Leif Hedstrom

Leif Hedstrom

Apple

Leif Hedstrom is a software engineer at Yahoo!, where among other things, he has designed and implemented several edge services actively used by many large sites, such as Y! News and Flickr. Lately he has pushed for Open Sourcing the HTTP proxy server used internally at Yahoo!, and since the fall of 2009, the Traffic Server project is now in the incubator stage with the Apache Software Foundation. Prior to Yahoo!, Leif has worked at several internet companies such as Netscape Communications and Infoseek Corporation.

Leif is an avid scuba diver, snowboarder, dog lover and family man. And of course, he’s a huge computer nerd.

Comments on this page are now closed.

Comments

Picture of Ryan Witt
Ryan Witt
06/23/2010 9:41am PDT

Really liked the in-depth look at the internals. Perhaps it wasn’t everyone’s cup of tea, but very useful for those seriously considering deploying traffic server.

For Velocity China sponsorship information for companies outside China, contact Yvonne Romaine at yromaine@oreilly.com.

  • Google
  • Strangeloop
  • Yahoo! Inc.
  • Dyn Inc.
  • Facebook
  • Schooner Information Technology
  • Tilera
  • AlertSite
  • AppDynamics
  • Aptimize
  • CDNetworks
  • Circonus
  • Cloudscaling
  • Clustrix
  • Coradiant
  • Dell
  • DTO Solutions
  • MaxiScale
  • Neustar
  • Nokia
  • NorthScale, Inc.
  • Shopzilla
  • Splunk
  • Virident
  • Zoompf
  • Neustar

For information on exhibition and sponsorship opportunities at the conference, contact Yvonne Romaine at yromaine@oreilly.com

Download the Velocity Sponsor/Exhibitor Prospectus

Download the Media & Promotional Partner Brochure (PDF) for information on trade opportunities with O'Reilly conferences or contact mediapartners@ oreilly.com

For media-related inquiries, contact Maureen Jennings at maureen@oreilly.com

To stay abreast of conference news and to receive email notification when registration opens, please sign up for the Velocity Conference bulletin (login required)

View a complete list of Velocity contacts