Increasing complexity of learning algorithms and Deep Neural Networks, combined with size of data and parameters, has made it challenging to exploit existing large-scale data processing pipelines for training and inference.
In this talk we walk the pathway of employing different tools and frameworks, ranging from Spark for pre-processing, to Deep Learning Frameworks for training and inference. We aim to target, the nuances in the data sets, in terms of pre-processing, training and inference as it relates to algorithm/optimization techniques, frameworks and scale.
Across our work in the field, we encounter various kinds of production pipelines.
Leveraging a typical “Big Data Production Pipeline” for learning and inference presents all kinds of challenges and opportunities, especially in terms of mapping data sets to optimal algorithm and/or architecture.
We guide you through the source code (on sample data sets): ingestion, pre-processing, training, inference and deployment across data sets as employed in production at scale.
Vartika Singh is a solutions consultant at Cloudera. Previously, Vartika was a data scientist applying machine-learning algorithms to real-world use cases, ranging from clickstream to image processing. She has 10 years of experience designing and developing solutions and frameworks utilizing machine-learning techniques.
Jeff Shmain is a principal solutions architect at Cloudera. He has 16+ years of financial industry experience with a strong understanding of security trading, risk, and regulations. Over the last few years, Jeff has worked on various use-case implementations at 8 out of 10 of the world’s largest investment banks.
Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?
Join the conversation here (requires login)
©2018, O’Reilly UK Ltd • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • email@example.com