Building a Better Web
June 19–20, 2017: Training
June 20–22, 2017: Tutorials & Conference
San Jose, CA

Automated performance testing in preproduction with CI and OSS tools

Billy Hoffman (Rigor)
4:25pm–5:05pm Thursday, June 22, 2017
Performance Matters
Location: 210 AE
Secondary topics:  Build and automation, Designing for performance, Tools
Average rating: ****.
(4.00, 2 ratings)

Who is this presentation for?

  • Frontend developers and DevOps engineers

Prerequisite knowledge

  • A basic understanding of the concepts of build automation or frontend performance optimization techniques

What you'll learn

  • Understand why you should do performance testing in preproduction
  • Learn how to integrate open source performance tools into your build/CI systems
  • Get practical advice on how to get started


It would be wildly irresponsible to deploy a release without extensive functional testing. Finding out you broke the checkout process after you released would be terrible. Ideally, you’d want to automatically test the entire app, and fail a build if certain critical use cases don’t function properly. At a minimum, you’d want transparency into what functional bugs and regression exist so you can make a cost-benefit analysis as to whether you still want to ship.

Since performance metrics can directly impact business goals like conversion rate as much as functionality can, why on earth would you ship a build without transparency about whether performance metrics are better or worse or performance issues were introduced or regressed? The answer is you shouldn’t. And if functional testing has taught us anything, it’s that integrating and automating testing as part of a build or CI system is critical to ensuring that consistent and repeatable insight.

Billy Hoffman explains how to integrate open source performance testing tools into your workflow, exploring practical tips and step-by-step examples. Along the way, Billy shares the lessons he’s learned helping organizations get these tools into CI and success stories about how this can reduce regressions and accelerate development while remaining confident about performance.

Photo of Billy Hoffman

Billy Hoffman


Billy Hoffman is the CTO at Rigor, where he focuses on delivering best-in-class performance software. From hacking into sites as a penetration tester to optimizing performance for the Fortune 500, Billy has over a decade of deep technical experience into the inner workings of complex web applications. Billy was the lead researcher at the web security software company SPI Dynamics, where he developed and automated new techniques to exploit web vulnerabilities and spoke extensively at conferences such as Black Hat and RSA about his findings. Following SPI’s acquisition by Hewlett-Packard, Billy managed HP’s web security research group. Billy founded the web performance company Zoompf, where he developed the industry’s first site-wide performance analysis product that tells you the root cause of your performance issues. He has spoken at industry conferences such as Velocity and was a contributing author for O’Reilly’s Web Performance Daybook. Billy holds a BS in computer science from the Georgia Institute of Technology. Outside of work, Billy enjoys refinishing rooms in his house and spending time with his wife, Jill, and their three children.

Comments on this page are now closed.


Anand Mattah Chenna Kesavalu | PERFORMANCE ENGINEER
07/06/2017 12:53am PDT

Please upload slides from this session