Presented By O'Reilly and Cloudera
Make Data Work
Sept 29–Oct 1, 2015 • New York, NY

Streaming in the extreme

Jim Scott (NVIDIA)
2:55pm–3:35pm Wednesday, 09/30/2015
IoT & Real-time
Location: 3D 02/11 Level: Intermediate
Average rating: ***..
(3.50, 6 ratings)
Slides:   1-PDF 

Have you ever heard of Kafka? Are you ready to start streaming all the events in your business? What happens to your streaming solution when you outgrow your single data center? What happens when you are at a company that is already running multiple data centers and you need to implement streaming across data centers? What about when you need to scale to a trillion events per day?

I will discuss technologies like Kafka that can be used to accomplish real-time, lossless messaging that works in both single and multiple globally dispersed data centers. I will also describe how to handle the data coming in through these streams in batch processes as well as real-time processes.

Finally, we will discuss why a streaming-only implementation can deliver a better experience with equivalent results to the Lambda Architecture.

Photo of Jim Scott

Jim Scott


Jim Scott is the head of developer relations, data science, at NVIDIA. He’s passionate about building combined big data and blockchain solutions. Over his career, Jim has held positions running operations, engineering, architecture, and QA teams in the financial services, regulatory, digital advertising, IoT, manufacturing, healthcare, chemicals, and geographical management systems industries. Jim has built systems that handle more than 50 billion transactions per day, and his work with high-throughput computing at Dow was a precursor to more standardized big data concepts like Hadoop. Jim is also the cofounder of the Chicago Hadoop Users Group (CHUG).