Skip to main content

Assembly at the Edge

Gamaiel Zavala (Yahoo!), Pushkar Sachdeva (Yahoo!)
Performance
Mission City Ballroom B1
Average rating: ***..
(3.17, 6 ratings)
Slides:   1-ZIP 

Yahoo! has historically suffered from high latency on the server due to it’s service oriented architecture. Pages often consist of a mix of static and personalized content, with the latter taking up the bulk of response time. This situation often resulted in the slowest service blocking the rest of the page. The architecture laid out below largely addresses these concerns by splitting the page assembly into two parts.

Yahoo! sites were already set up to be very modular. Each individual piece of the page can be executed standalone. This modularity allowed us to designate specific portions of the page as personalized and therefore uncacheable. The rest of the page is deemed cacheable.

The entire page renders on the origin server as normal but also outputs Edge Side Include (ESI) tags where personalized content should appear. The result is then cached at the edge (which sits between the user and origin) using Apache Traffic Server (ATS) so no additional hops are necessary to build the page layout.

The cache is determined by the URL and various headers sent from ATS, including device and other criteria. The header names are then included in the response’s Vary header in order to cache multiple variations.

ATS then parses the cached response to locate each ESI tag and fans out back to the origin for each one. The first chunk of the page is streamed out until an include is encountered. Once the request for that include finishes it will be flushed along with any other markup up to the next include. Such functionality did not exist in ATS prior and has since been submitted back to the open source branch.

In addition, includes can be grouped so that they can flush out of order as they complete, thereby eliminating unnecessary latency introduced by a single slow response. The origin may respond with HTML markup for blocking includes or in the case of a grouped include, script which injects the module into the correct location within the page. The end result is a partial assembly at the origin for any cacheable content and a final assembly at the edge to plug in the personalized uncacheable content. Altogether these changes yielded large gains in time to first byte and overall page load time.

Gamaiel Zavala

Yahoo!

Gamaiel Zavala is a Frontend Engineer at Yahoo!. He enjoys all types of code and strives to understand the big picture, from protocols and packets to the myriad of technologies that power web applications. He also enjoys playing guitar, wood working, and hobbyist electronics. Gamaiel currently resides in Ventura, California with his wife and two children.

Photo of Pushkar Sachdeva

Pushkar Sachdeva

Yahoo!

Pushkar Sachdeva is a Senior Software Engineer at Yahoo!. His interests are Computer Networks, HTTP Proxy and Caching.