Reflecting a Year After Migrating to Apache Traffic Server

Nick Berry (LinkedIn)
Operations, Mission City Ballroom B4
Presentation: external link
Average rating: ***..
(3.62, 16 ratings)

LinkedIn started May 2003, I started August 2011. Over 8 years of cruft and confusion piled up before we even considered moving to Apache Traffic Server. This talk will focus on the journey and what we learned along the way:

What LinkedIn is doing with ATS to affect change across the entire stack with a infrastructure tier

  • cookie-based routing
  • DDoS proctection
  • Decentralized load balancing
  • Pushing logic into ATS, away from ~400 individual services
  • Multi-colo design including quick failover

Buy-in from management and service owners

  • Not losing a bet to David Henke
  • Dispelling myths from Devs

Building automation and tooling early

  • Evolving from hand-edits + scp to salt-based deployments with step-by-step execution scripts
  • Generating configs and dashboards for ATS and HA Proxy
  • Verification, validation
  • Treating the application like an appliance

Bizarre scenarios how users are querying the site

  • POST requests with no content-length and no body
  • Clients using CONNECT then sending request
  • Securing logs due to API users sending sensitive GET queries

Bizarre scenarios how the site is set up

  • /profile != /profile/
  • managing headless redirects

Metrics and monitoring (not much ATS-specifics shared/talked about in the community)

  • ATS-specific
  • Plugins
  • C apps

Patches we contributed:

  • Mapping requests based on incoming port
  • File descriptor bug if fd >32
  • Connection handling/locking
  • Fixing multiple core dumps for less popular features

Details on plugins we’ve developed and highlight some to be open-sourced either by Velocity or at Velocity.

  • Google PageSpeed: html minification, image lazy loading below the fold
  • Security: DDoS, Content-Security-Policy header
  • cookie-based routing
  • boom (http response override)
  • host override (cookie-based routing to specific origins/hosts)

Presentation

Photo of Nick Berry

Nick Berry

LinkedIn

Nick currently manages the Site Reliability Engineering team responsible for Presentation Infrastructure, Growth and Identity at LinkedIn. Before that, he was responsible for Yahoo’s global homepages.

Favorite quote: “Good judgment comes from experience, and often experience comes from bad judgment.”

Sponsors

Sponsorship Opportunities

For information on exhibition and sponsorship opportunities at the conference, contact Gloria Lombardo at (203) 381-9245 or glombardo@oreilly.com

Media Partner Opportunities

For media partnerships, contact mediapartners@ oreilly.com

Press and Media

For media-related inquiries, contact Maureen Jennings at maureen@oreilly.com

Contact Us

View a complete list of Velocity contacts