Building a distributed time series database on PostgreSQL
Who is this presentation for?Data engineers, data architects, developers
Time series data tends to accumulate very quickly, across DevOps, IoT, industrial and energy, finance, and other domains. To drive real-time decisions and data science, software developers often seek to wrangle this large volume of data into a variety of database systems.
Michael Freedman discusses the five objectives for scaling a database for time series workloads—total storage volume, insert rate, query concurrency, query latency, and fault-tolerant replication—and how these objectives have different needs.
You’ll take a technical dive into how TimescaleDB leveraged its chunk-based architecture to go from a primary-replica system on PostgreSQL to a scale-out distributed time series database that can scale to tens of millions of metrics per second, store petabytes of data, and process queries even faster via better parallelization. Michael describes how this architecture, compared to a traditional sharded system, enabled a much broader set of capabilities that you’ll want for time series workloads (e.g., both scale up and scale out, elasticity without data movement, partitioning flexibility, and age-based data retention, tiering, and reordering).
- General knowledge of databases and distributed systems
What you'll learn
- Learn about scaling a database for time series workloads
TimescaleDB | Princeton University
Michael J. Freedman is the cofounder and CTO of TimescaleDB and a full professor of computer science at Princeton University. His work broadly focuses on distributed and storage systems, networking, and security, and his publications have more than 12,000 citations. He developed CoralCDN (a decentralized content distribution network serving millions of daily users) and helped design Ethane (which formed the basis for OpenFlow and software-defined networking). Previously, he cofounded Illuminics Systems (acquired by Quova, now part of Neustar) and served as a technical advisor to Blockstack. Michael’s honors include a Presidential Early Career Award for Scientists and Engineers (given by President Obama), the SIGCOMM Test of Time Award, a Sloan Fellowship, an NSF CAREER award, the Office of Naval Research Young Investigator award, and support from the DARPA Computer Science Study Group. He earned his PhD at NYU and Stanford and his undergraduate and master’s degrees at MIT.
Leave a Comment or Question
Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?
Join the conversation here (requires login)
Premier Diamond Sponsors
Premier Exhibitor Plus
For conference registration information and customer service
For more information on community discounts and trade opportunities with O’Reilly conferences
For information on exhibiting or sponsoring a conference
For media/analyst press inquires