Nothing exercises the limits of software like turning it over to the end users. While in-house testing is invaluable, the end users can often find the neglected edge cases or subtle race conditions in a more reliable manner. However, once the software is in the end user’s hands, postmortem crash information is usually lost. Even if it is retrieved, the users were not using the version instrumented with debug information. Because of those problems, valuable trends about crashes can go unnoticed, and opportunities for improved software robustness are forfeited.
By taking the Google Breakpad project and enhancing it with Mozilla’s own Socorro project, Mozilla has imbued its latest generation of products with the ability to overcome these problems. The client library component of Google Breakpad lives in the application to collect machine independent crash dump data. When a crash occurs, it captures the state of the application (frames, threads, modules, etc) and transmits that information to a waiting Socorro Server. The Socorro Server can handle multiple simultaneous submissions and will filter and schedule them for further analysis. The Socorro Server can be scaled simply by adding more instances, which allows the Mozilla installation to accept more than a million crash reports a day. Socorro Server uses Google Breakpad libraries to generate useful stack traces from crash reports. The data ends up in a PostgreSQL database overseen by the Socorro UI, which is a PHP web application that provides information on specific crash reports and trends.
In this talk, the three primary Mozilla developers in charge of the three parts of the project will discuss the architecture, deployment, and use of the Socorro/Google Breakpad system. While the path has not been entirely smooth, this Open Source project has proven invaluable to Mozilla.
Trapped at the triple point between a geek, a hippie and a biker, Lars is a self proclaimed Cowboy Programmer at the Mozilla Corporation. Unintentionally specializing in programming as performance art, Lars frequently jumps into projects on the Thursday prior to a Monday deadline. Lars is proud of being the only member of the Web Development team that does absolutely no Web development.
Lars prefers Python, PostgreSQL and Harleys, but is versed in C++, MySQL and Subarus.
Ted is a platform engineer with the Mozilla Corporation. He is the owner of the build system as well as the crash reporting modules. Ted has been involved with Mozilla for nearly seven years as a community contributor and now a full-time employee. He enjoys working on software that makes users’ and developers’ lives easier.
Austin is a Seattle based coder on the Mozilla web developer team. With a decade of frontend and backend experience at companies including Amazon.com and Rhapsody America, he is passionate about user focused development and web programming.
For information on exhibition and sponsorship opportunities at the conference, contact Sharon Cordesse at firstname.lastname@example.org
Download the OSCON Sponsor/Exhibitor Prospectus
For media-related inquiries, contact Maureen Jennings at email@example.com
To stay abreast of conference news and to receive email notification when registration opens, please sign up for the OSCON newsletter (login required)
View a complete list of OSCON contacts