Sep 23–26, 2019

Lessons learned from scaling the tech stack of a modern analytics platform

Tom O'Neill (Sisense)
3:45pm4:25pm Thursday, September 26, 2019
Location: 1E 07/08

Who is this presentation for?

Data engineers, data architects, data scientists, data analysts




In a given day, analysts write more than 300,000 new lines of SQL in Periscope Data, on top of the millions of lines already executing on the platform. As the average analyst on the platform has increased their workload to use it for more than 20+ hours per week, Periscope has scaled up dramatically with dozens of terabytes of RAM supporting its 2,000+ teams as part of Sisense.

In this session, Tom O’Neill, Chief Cloud Officer, Sisense, will discuss lessons learned from scaling up Periscope Data to support incredibly large volumes of data and queries from its data teams. He’ll highlight the process of migrating from Heroku to Kubernetes and discovering new ways to leverage its power, plus other developments and changes to the tech stack, including warehouse and data processing tools and partnerships, that have allowed Periscope users to delve deeper into new data science and machine learning analysis.

Prerequisite knowledge

Familiarity with or interest in ETL concepts, data warehousing, and programming languages like SQL, Python and R

What you'll learn

Explore the challenges behind building a platform to support data science and analysis at massive scale.
Photo of Tom O'Neill

Tom O'Neill


Tom O’Neill is Chief Cloud Officer of Sisense, a leading independent platform for analytics builders. He was the co-founder and CTO of Periscope Data, which merged with Sisense in May 2019. Before Periscope Data, Tom worked on the machine learning algorithms behind the search results at Microsoft Bing. He has a bachelor’s degree in computer science from the University of Rochester.

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)

Contact us

For conference registration information and customer service

For more information on community discounts and trade opportunities with O’Reilly conferences

For information on exhibiting or sponsoring a conference

Contact list

View a complete list of Strata Data Conference contacts