Mar 15–18, 2020

Building live dashboards on Amazon DynamoDB using Rockset

Kshitij Wadhwa (Rockset), Dhruba Borthakur (Rockset)
2:35pm3:15pm Wednesday, March 18, 2020
Location: LL21A

Who is this presentation for?

Data engineers, data architects, developers

Level

Intermediate

Description

Rockset is a serverless search and analytics engine delivered as a cloud service. One of the company’s use cases is to support SQL on NoSQL data from Amazon DynamoDB and perform real-time analytics. There’s an increasing need for operational analytics and real-time business reporting in organizations, which requires the ability to search transactional data, run aggregations, and join the data with other datasets. Organizations using DynamoDB find it hard to run complex queries for analytics as it’s not designed for scan operations or fetching a large volume of data.

This led the company to architect and integrate DynamoDB and maintain an up-to-date indexed version of DynamoDB in Rockset. Kshitij Wadhwa and Dhruba Borthakur explore how Rockset maintains a real-time sync with DynamoDB and provides fast and complex SQL support over it.

You’ll follow along on the project’s journey and see the challenges the company faced and how it solved them. Kshitij and Dhruba explain the limitations of DynamoDB for building live dashboards, designing a system that bulk loads and then streams updates from DynamoDB in a fault-tolerant fashion, maximizing read throughput from DynamoDB without interfering with existing workloads, continuously monitoring DynamoDB streams without adding cost to the user’s DynamoDB bill while keeping data latency below one second, and building interactive and live dashboards on DynamoDB.

Prerequisite knowledge

  • A basic understanding of Amazon DynamoDB, SQL, and real-time analytics

What you'll learn

  • Learn about Rockset, an approach powerful SQL analytics on Amazon DynamoDB without ETL
Photo of Kshitij Wadhwa

Kshitij Wadhwa

Rockset

Kshitij Wadhwa is a software engineer at Rockset, where he works on the platform engineering team. Previously, Kshitij was an engineer at NetApp on the filesystem and protocols team in Cloud Backup Service. Kshitij holds a master degree in computer science from North Carolina State University.

Photo of Dhruba Borthakur

Dhruba Borthakur

Rockset

Dhruba Borthakur is cofounder and CTO at Rockset, a company building software to enable data-powered applications. Previously, Dhruba was the founding engineer of the open source RocksDB database at Facebook and one of the founding engineers of the Hadoop file system at Yahoo; an early contributor to the open source Apache HBase project; a senior engineer at Veritas, where he was responsible for the development of VxFS and Veritas SanPointDirect storage system; the cofounder of Oreceipt.com, an ecommerce startup based in Sunnyvale; and a senior engineer at IBM-Transarc Labs, where he contributed to the development of Andrew File System (AFS), a part of IBM’s ecommerce initiative, WebSphere. Dhruba holds an MS in computer science from the University of Wisconsin-Madison and a BS in computer science BITS, Pilani, India. He has 25 issued patents.

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)

Contact us

confreg@oreilly.com

For conference registration information and customer service

partners@oreilly.com

For more information on community discounts and trade opportunities with O’Reilly conferences

Become a sponsor

For information on exhibiting or sponsoring a conference

pr@oreilly.com

For media/analyst press inquires