Everything open source
May 16–17, 2016: Training & Tutorials
May 18–19, 2016: Conference
Austin, TX

Static and dynamic data with Postgres, Kafka, and Bottled Water

Chad Bailey (Heroku )
11:05am–11:45am Thursday, 05/19/2016
Sponsored
Location: Meeting Room 14
Average rating: ****.
(4.25, 4 ratings)

Prerequisite knowledge

Attendees should be familiar with basic web app architecture and/or have experience with running web apps on platforms like Heroku.

Description

Data stores like Postgres excel at storing and querying structured data, but many modern web apps have needs beyond what SQL can easily and quickly provide. Developers often find themselves writing the same data to multiple stores for searching, caching, or data warehousing. Meanwhile, tools like Kafka are giving developers a different way to think about their data. Instead of queries of static data, Kafka provides a real-time stream of app events that can be consumed for analysis and storage. Kafka emerged out of work at LinkedIn and is currently in use by companies like Netflix, PayPal, Spotify, and Uber.

But what if you could view your Postgres database as a series of events, Kafka-style? That’s the goal of Bottled Water, an open source tool that has emerged from Confluent. Bottled Water uses features added in Postgres 9.4 to capture change events in your database and push them to a Kafka cluster for real-time analysis and processing.

Heroku’s Chad Bailey walks attendees through building an end-to-end demo app to show how this all comes together. Chad demonstrates setting up a Kafka cluster and simple Ruby app to consume the Kafka event stream. Then, he’ll create a simple Rails app with a Postgres database to demonstrate Bottled Water. The demo itself is deceptively simple, but the real power comes from how you put the combination of static and dynamic data to work in your own apps.

This session is sponsored by Heroku.

Photo of Chad Bailey

Chad Bailey

Heroku

Chad Bailey left a career as a rocket scientist to build web apps, and he’s been doing just that for the better part of a decade. Chad currently works on the support team at Heroku, where he helps customers run their databases on one of the largest production Postgres fleets in the world.