Building an intelligent big data app in 30 minutes

Claudiu Barbura (Ubix), David Talby (Atigeo)
Hadoop & Beyond
Location: 212
Average rating: ***..
(3.31, 13 ratings)
Slides:   1-PPTX 

Real, production-grade big data apps require integration between data ingestion components, Beyond Hadoop technologies, data science & modeling tools, publishing data to low-latency, high-throughput REST API’s backed by SQL or NoSQL stores, a visualization / application layer – as well as monitoring, instrumentation, security and administration tools. Adding intelligence to such an application also requires a broad set of machine learning algorithms, the ability to train & measure experiments in parallel, the ability to publish models as front-end low-latency robust API’s, and having online measurements in place. All this results in man-months of work on an average project to glue these pieces together into a production system. In this talk we’ll describe some of the main gaps we’ve had to address building dozens of such systems over the years, the design patterns & reference architecture we’ve come to adopt, and some handy tools to automate common tasks. We’ll demonstrate the challenges by building an end-to-end scalable, intelligent app during the session

Photo of Claudiu Barbura

Claudiu Barbura

Ubix

Claudiu is Atigeo’s Senior Director of Engineering, Platform Services, and oversees agile engineering teams in the US and Romania while also acting as Lead Architect in building and operating xPatterns, an enterprise-class, Big Data Analytics platform. Claudiu has 17 years of industry experience in various roles, with a strong passion for Software Architecture leveraging industry best patterns and practices and contributing with a significant level of innovation. His experience spans across the Open Source, Big Data and Microsoft’s Windows/.Net technology stacks

Photo of David Talby

David Talby

Atigeo

David Talby has extensive experience in building & operating web-scale search and business platforms, as well as building world-class, agile, distributed teams. Previously he was with Microsoft’s Bing group where he was a Principal Group Manager leading all business operations for Bing Shopping in the US and Europe, managing teams in Seattle, India and several European countries. Prior to Microsoft, David worked at Amazon both in Seattle and in the UK where he built and ran distributed teams which helped scale Amazon’s financial systems. David is a veteran of the Israeli Air Force and he holds a PhD in Computer Science along with two masters degrees in Science and Business Administration respectively, all of which were obtained at the Hebrew University of Jerusalem. In addition, David has taught courses on Object Oriented Design; Software Design; Distributed algorithms; advanced programing and system analysis; operating systems; systems programming and marketing at the Hebrew University of Jerusalem and Hadassah College of Technology, Jerusalem. David has 23 published papers & patents to date