Sep 23–26, 2019
Please log in

Lessons learned from scaling the tech stack of a modern analytics platform

Scott Castle (Sisense)
3:45pm4:25pm Thursday, September 26, 2019
Location: 1E 07/08
Average rating: ****.
(4.00, 1 rating)

Who is this presentation for?

  • Data engineers, data architects, data scientists, and data analysts




In a given day, analysts write more than 300,000 new lines of SQL in Periscope Data, on top of the millions of lines already executing on the platform. As the average analyst on the platform has increased their workload to use it for more than 20+ hours per week, Periscope has scaled up dramatically with dozens of terabytes of RAM supporting its 2,000+ teams as part of Sisense.

In this session, Scott Castle, General Manager at Sisense and former VP of Product at Periscope Data, will discuss lessons learned from scaling up Periscope Data to support incredibly large volumes of data and queries from its data teams. He’ll highlight the process of migrating from Heroku to Kubernetes and discovering new ways to leverage its power, plus other developments and changes to the tech stack, including warehouse and data processing tools and partnerships, that have allowed Periscope users to delve deeper into new data science and machine learning analysis.

Prerequisite knowledge

  • Familiarity with or interest in ETL concepts, data warehousing, and programming languages like SQL, Python, and R

What you'll learn

  • Explore the challenges behind building a platform to support data science and analysis at massive scale
Photo of Scott Castle

Scott Castle


Scott Castle is GM of the Data Business at Sisense and served as VP of Product for Periscope Data prior to its merger with Sisense in 2019. At Periscope Data, Scott oversaw product strategy, planning, design and delivery. He brings over 20 years of experience in software development and product management at leading technology companies including Adobe, Electric Cloud, and FileNet. Scott has a B.A. degree from the University of Massachusetts, Amherst, and a master’s degree from UC Irvine, both in computer science.

  • Cloudera
  • O'Reilly
  • Google Cloud
  • IBM
  • Cisco
  • Dataiku
  • Intel
  • Io-Tahoe
  • MemSQL
  • Microsoft Azure
  • Oracle Cloud Infrastructure
  • SAS
  • Arcadia Data
  • BMC Software
  • Hazelcast
  • SAP
  • Amazon Web Services
  • Anaconda
  • Esri
  •, Inc.
  • Kyligence
  • Pitney Bowes
  • Talend
  • Google Cloud
  • Confluent
  • DataStax
  • Dremio
  • Immuta
  • Impetus Technologies Inc.
  • Keyence
  • Kyvos Insights
  • StreamSets
  • Striim
  • Syncsort
  • SK holdings C&C

    Contact us

    For conference registration information and customer service

    For more information on community discounts and trade opportunities with O’Reilly conferences

    For information on exhibiting or sponsoring a conference

    For media/analyst press inquires