There are many challenges in serving DNNs at scale today. As algorithms become more complex requiring more processing power, trade-offs have to be made to serve accurate results quickly in a cost-effective way.
First, we’ll describe Microsoft Project Brainwave, a DNN-serving platform powered by Intel field-programmable gate array (FPGA) chips. This was motivated by the Recurrent Neural Networks (RNNs) required by the Bing team to process text and return query results. We’ll talk about why Intel FPGAs help us adapt to the fast-changing AI landscape by enabling us to reconfigure the chips with software, instead of having to fab chips and deploy them to datacenters every time there is a refresh. We’ll highlight the innovations in Project Brainwave, from a new data type to utilizing spatial compute so there is no need for a large batch size to get high throughput and fast results.
Second, we’ll talk about Azure Machine Learning, an end-to-end machine learning platform accessible as a set of services by a Python SDK or command-line interface. Using familiar languages such as Python and frameworks such as TensorFlow, we’ll describe how to train a computer vision model and deploy it to an FPGA directly from a Jupyter Notebook. No knowledge of Verilog or VHDL is required. Performance for models such as ResNet 50 in under 2 ms can be achieved, five times faster than published benchmarks.
Finally we’ll show a demo of running through a Jupyter notebook and deploying an image classification model to run on an FPGA. This model can be deployed on an FPGA running in the Azure cloud or on an Azure IOT Edge device. The audience can see how they can easily do this themselves for fast and cost-effective AI.
Ted Way is a senior program manager on the Microsoft Azure Machine Learning engineering team. He’s passionate about telling the story of how AI will empower people and organizations to achieve more. He currently works on bringing machine learning to the edge and hardware acceleration of AI.
He received BS degrees in electrical engineering and computer engineering, MS degrees in electrical engineering and biomedical engineering, and a PhD in biomedical engineering from the University of Michigan – Ann Arbor. His PhD dissertation was on “spell check for radiologists,” a computer-aided diagnosis (CAD) system that uses image processing and machine learning to predict lung cancer malignancy on chest CT scans.
He has been invited as a keynote speaker for two Microsoft partner conferences, and has twice received the Microsoft Executive Briefing Center’s Distinguished Speaker Award, awarded to only 5 out of over 1,000 speakers.
Aishani Bhalla is a software engineer on the Azure Machine Learning team. At Microsoft, she’s helping people operationalize models to build intelligence into every application, accelerated on FPGAs with Project Brainwave. She has a BS in Computer Science from the University of Buffalo.
Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?
Join the conversation here (requires login)
©2019, O'Reilly Media, Inc. • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • email@example.com