The need for gleaning answers from data in real time from unbounded data streams is moving from nicety to a necessity. Netflix is a data-driven company, and Netflix Keystone SPaaS processes over a trillion messages a day with at-least-once processing semantics in the cloud to derive insights. To ease extracting intelligence from this unbounded stream easily, Netflix is building stream-processing-as-a-service (SPaaS) infrastructure so that the user can focus on extracting value and not have to worry about building a scalable, fault-tolerant infrastructure.
Monal Daxini explores how Netflix is building a scalable, self-serve, fault-tolerant, multitenant stream processing service by leveraging open source software like Kafka, Samza, Flink, Mesos, and Docker, how it is evolving the service in flight without downtime, and lessons learned along the way.
Monal Daxini is an engineering manager at Netflix, where he is building a scalable and multitenant event processing pipeline and leads the infrastructure for stream processing as a service. He has worked on Netflix’s Cassandra and Dynamite infrastructure and was instrumental in developing the encoding compute infrastructure for all Netflix content. Monal has 15 years of experience building distributed systems at organizations like Netflix, NFL.com, and Cisco.
©2017, O'Reilly Media, Inc. • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • email@example.com
Apache Hadoop, Hadoop, Apache Spark, Spark, and Apache are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries, and are used with permission. The Apache Software Foundation has no affiliation with and does not endorse, or review the materials provided at this event, which is managed by O'Reilly Media and/or Cloudera.