Skip to main content

Performance and Maintainability with Continous Experimentation

Seth Walker (Etsy)
Performance
Mission City Ballroom B1
Average rating: ****.
(4.11, 19 ratings)

Etsy developed its capability for continuous delivery for operational reasons, but as our analytical capabilities improved it also served as the foundation for continuous experimentation. Etsy product teams shipped over 400 experiments in 2013 – many of these were failures. This is a good thing, as it allows us to focus on the changes that are winners, but it can present challenges to the performance and front-end infrastructure teams as we adapt to optimizing for “throwaway-ability”. Seth will talk about some of the tools and techniques we’ve developed to manage these challenges, facilitating rapid experimentation while maintaining high performance.

  • Simultaneous experiments increase the surface area required for synthetic testing. We increasingly rely on instrumenting code for a sample of users for collecting both performance data and data on how code is being used in production.
  • We’ve developed a front-end codebase explorer that allows developers to see the dependencies tree of modular static assets bundles and between assets and templates. This tool also surfaces presumed-unused code and static analysis errors and warnings.
  • We’ve added a heads up display for Etsy admin that shows performance information, currently running experiments, and more.
  • We’ve instituted a weekly rotation for non-performance-team engineers to audit upcoming launches and experiments for performance opportunities.

Seth will also talk about how the performance team itself uses our experiment infrastructure to validate changes and help build a case for performance optimization.

Photo of Seth Walker

Seth Walker

Etsy

Seth Walker is championing performance and front-end ops at Etsy.