Performance Measurement and Case Studies at MSN

Paul Roy (Microsoft), Alex Polak (Microsoft)
Web Performance Ballroom EFGH
Average rating: ****.
(4.00, 14 ratings)

MSN is on a mission to be the world’s fastest portal. Driving this mission requires a paradigm shift in how we measure performance and its impact. In this session we describe why existing metrics used at MSN and the industry in general are deficient, and need to evolve from an internal system view to a human view – namely, to represent rendering and responsiveness. We describe the requirements and gaps in this space, and offer a Call to Action to browser makers, tool makers, and the performance community in general to address these gaps. We also describe the range of performance measurement systems used at MSN, spanning both synthetic and real user environments.

Effectively measuring performance and its impact also requires assessing business impact. We describe how MSN uses A/B testing to assess the impact of performance changes on business metrics such as Page Views and Searches. A/B testing is critical, as business impact is the ultimate truth of whether a change is worthwhile.

We also will share the results and insights of several case studies, including the quantified impact on business metrics of implementing specific performance improvements and best practices. Lastly, we will discuss the techniques we’re using to drive the performance mission inside of MSN.

Photo of Paul Roy

Paul Roy


Paul Roy leads the MSN Performance & Reliability team. Paul has 25 years of software industry experience as a Developer, Architect, and Engineering Manager in the areas of distributed operating systems and internet systems. His passion is leading teams in the development and optimization of high performance systems. Prior to joining Microsoft in 1997, Paul worked at Silicon Graphics, Open Software Foundation, and Apollo Computer.

Photo of Alex Polak

Alex Polak


Alex Polak is a Technical Lead on the MSN Performance & Reliability team where he leads the work on front-end optimization and analysis. Alex has worked at Microsoft for 10 years and has expertise in web performance, network protocols, and test methodologies.

Comments on this page are now closed.


Picture of Aaron Peters
Aaron Peters
06/20/2011 2:28pm PDT

Well done case study. Velocity needs these kind of talks.

Picture of Jake Brutlag
Jake Brutlag
06/15/2011 7:45am PDT

Good call to arms on better measurement of rendering and responsiveness. Excellent case studies demonstrating the business value of recommended techniques like refactoring/deferring JS and inlining images with data URIs.

Picture of Aaron Peters
Aaron Peters
06/15/2011 7:04am PDT

Nice presentation. Fine structure and well delivered. We need more ‘web performance helps the business’ case studies at Velocity. Good stuff.

Picture of Steve Souders
Steve Souders
05/19/2011 8:04am PDT

MSN was the first major website I saw using advanced script loading techniques. They continue to be performance leaders today, especially in the area of JavaScript performance. I’m looking forward to hearing Paul and Alex distill MSN’s performance experiments into a clear set of best practices we can all takeaway.

  • Keynote Systems
  • Cisco
  • Google
  • Neustar
  • Betfair
  • Cotendo
  • Rackspace Hosting
  • Akamai
  • Apica
  • dynaTrace
  • Equinix
  • Facebook
  • New Relic
  • Opscode
  • Yahoo! Inc.
  • AppDynamics
  • Aptimize
  • Blaze
  • CDNetworks
  • Cedexis
  • Citrix Systems
  • Compuware Corporation
  • Dyn Inc.
  • F5 Networks
  • Heroku
  • Percona
  • Quest Software
  • Schooner Information Technology
  • SiteSpect
  • Splunk
  • Strangeloop
  • WatchMouse
  • Zeus Technology
  • Neustar

Sponsorship Opportunities

For information on exhibition and sponsorship opportunities at the conference, contact Yvonne Romaine at

Download the Velocity Sponsor/Exhibitor Prospectus

Contact Us

View a complete list of Velocity contacts