Error Detection and Correction with MySQL Replication

Replication and Scale-Out
Location: Ballroom A
Average rating: ****.
(4.20, 5 ratings)

MySQL replication has been a critical part of scaling Facebook’s storage infrastructure.  However, it brings with it the fear of divergent replicas. A divergence could be introduced by lost or reordered queries, queries not identically executed on all replicas, or other errors in MySQL.  Once a divergence emerges, that divergence may become amplified by future replicated statements.

This session discusses a tool developed at Facebook to detect divergences, identify inconsistent data, and repair defects without having to take any replicas offline.  We address challenges posed asynchronous replication that complicate the use of table checksumming tools like Maatkit. We also aim to minimize interference with existing database load and inter-replica network traffic to make the tool suitable for online use.  

Finally, we’ll talk about results achieved using this tool.  

Photo of Daniel Peek

Daniel Peek


Daniel Peek works on distributed storage issues at Facebook.  Dr. Peek holds a doctorate in Computer Science from the University of Michigan, Ann Arbor.  

Comments on this page are now closed.


Juergen Heerdegen
04/20/2011 3:13am PDT

I attended this talk and found it interesting. I’d like to discuss it further with my team, who were not able to be there. Will the slides be available?

  • EnterpriseDB
  • Amazon Web Services
  • Clustrix
  • Continuent
  • Facebook
  • HTI Consultoria e Tecnologia
  • Monty Program
  • Percona
  • Rackspace Hosting
  • Schooner Information Technology
  • SkySQL
  • Xeround

Sponsorship Opportunities

For information on exhibition and sponsorship opportunities at the conference, contact Yvonne Romaine at

Media Partners Opportunities

For media partnerships, contact mediapartners@

Press & Media

For media-related inquiries, contact Maureen Jennings at

O'Reilly MySQL Conference Bulletin

To stay abreast of conference news and to receive email notification when registration opens, please sign up for the O'Reilly MySQL Conference Bulletin (login required).

Contact Us

View a complete list of O'Reilly MySQL Conference Contacts