Building live dashboards on Amazon DynamoDB using Rockset
Who is this presentation for?Data engineers, data architects, developers
Rockset is a serverless search and analytics engine delivered as a cloud service. One of the company’s use cases is to support SQL on NoSQL data from Amazon DynamoDB and perform real-time analytics. There’s an increasing need for operational analytics and real-time business reporting in organizations, which requires the ability to search transactional data, run aggregations, and join the data with other datasets. Organizations using DynamoDB find it hard to run complex queries for analytics as it’s not designed for scan operations or fetching a large volume of data.
This led the company to architect and integrate DynamoDB and maintain an up-to-date indexed version of DynamoDB in Rockset. Kshitij Wadhwa and Dhruba Borthakur explore how Rockset maintains a real-time sync with DynamoDB and provides fast and complex SQL support over it.
You’ll follow along on the project’s journey and see the challenges the company faced and how it solved them. Kshitij and Dhruba explain the limitations of DynamoDB for building live dashboards, designing a system that bulk loads and then streams updates from DynamoDB in a fault-tolerant fashion, maximizing read throughput from DynamoDB without interfering with existing workloads, continuously monitoring DynamoDB streams without adding cost to the user’s DynamoDB bill while keeping data latency below one second, and building interactive and live dashboards on DynamoDB.
- A basic understanding of Amazon DynamoDB, SQL, and real-time analytics
What you'll learn
- Learn about Rockset, an approach powerful SQL analytics on Amazon DynamoDB without ETL
Kshitij Wadhwa is a software engineer at Rockset, where he works on the platform engineering team. Previously, Kshitij was an engineer at NetApp on the filesystem and protocols team in Cloud Backup Service. Kshitij holds a master degree in computer science from North Carolina State University.
Dhruba Borthakur is cofounder and CTO at Rockset, a company building software to enable data-powered applications. Previously, Dhruba was the founding engineer of the open source RocksDB database at Facebook and one of the founding engineers of the Hadoop file system at Yahoo; an early contributor to the open source Apache HBase project; a senior engineer at Veritas, where he was responsible for the development of VxFS and Veritas SanPointDirect storage system; the cofounder of Oreceipt.com, an ecommerce startup based in Sunnyvale; and a senior engineer at IBM-Transarc Labs, where he contributed to the development of Andrew File System (AFS), a part of IBM’s ecommerce initiative, WebSphere. Dhruba holds an MS in computer science from the University of Wisconsin-Madison and a BS in computer science BITS, Pilani, India. He has 25 issued patents.
Leave a Comment or Question
Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?
Join the conversation here (requires login)
Premier Diamond Sponsors
Premier Exhibitor Plus
For conference registration information and customer service
For more information on community discounts and trade opportunities with O’Reilly conferences
For information on exhibiting or sponsoring a conference
For media/analyst press inquires