Presented By
O’Reilly + Intel AI
Put AI to Work
April 15-18, 2019
New York, NY

Fast (and cheap) AI accelerated on FPGAs

Ted Way (Microsoft Corporation), Aishani Bhalla (Microsoft)
4:55pm5:35pm Wednesday, April 17, 2019
Implementing AI
Location: Rendezvous
Secondary topics:  Computer Vision, Edge computing and Hardware, Platforms and infrastructure

Who is this presentation for?

Data scientists and IT pros who want to accelerate their DNN processing speed



Prerequisite knowledge

Audience members should have beginner knowledge of various DNNs and the problems they can solve, such as ResNet, Inception, VGG, and DenseNet for image classification and models such as SSD-VGG for bounding boxes. A basic understanding of Python and TensorFlow would also be helpful but not required.

What you'll learn

Learn how Project Brainwave accelerates AI by using FPGAs Learn how Python, TensorFlow, Azure ML, and Project Brainwave can be used to deploy a DNN onto an FPGA


There are many challenges in serving DNNs at scale today. As algorithms become more complex requiring more processing power, trade-offs have to be made to serve accurate results quickly in a cost-effective way.

First, we’ll describe Microsoft Project Brainwave, a DNN-serving platform powered by Intel field-programmable gate array (FPGA) chips. This was motivated by the Recurrent Neural Networks (RNNs) required by the Bing team to process text and return query results. We’ll talk about why Intel FPGAs help us adapt to the fast-changing AI landscape by enabling us to reconfigure the chips with software, instead of having to fab chips and deploy them to datacenters every time there is a refresh. We’ll highlight the innovations in Project Brainwave, from a new data type to utilizing spatial compute so there is no need for a large batch size to get high throughput and fast results.

Second, we’ll talk about Azure Machine Learning, an end-to-end machine learning platform accessible as a set of services by a Python SDK or command-line interface. Using familiar languages such as Python and frameworks such as TensorFlow, we’ll describe how to train a computer vision model and deploy it to an FPGA directly from a Jupyter Notebook. No knowledge of Verilog or VHDL is required. Performance for models such as ResNet 50 in under 2 ms can be achieved, five times faster than published benchmarks.

Finally we’ll show a demo of running through a Jupyter notebook and deploying an image classification model to run on an FPGA. This model can be deployed on an FPGA running in the Azure cloud or on an Azure IOT Edge device. The audience can see how they can easily do this themselves for fast and cost-effective AI.

Photo of Ted Way

Ted Way

Microsoft Corporation

Ted Way is a senior program manager on the Microsoft Azure Machine Learning engineering team. He’s passionate about telling the story of how AI will empower people and organizations to achieve more. He currently works on bringing machine learning to the edge and hardware acceleration of AI.

He received BS degrees in electrical engineering and computer engineering, MS degrees in electrical engineering and biomedical engineering, and a PhD in biomedical engineering from the University of Michigan – Ann Arbor. His PhD dissertation was on “spell check for radiologists,” a computer-aided diagnosis (CAD) system that uses image processing and machine learning to predict lung cancer malignancy on chest CT scans.

He has been invited as a keynote speaker for two Microsoft partner conferences, and has twice received the Microsoft Executive Briefing Center’s Distinguished Speaker Award, awarded to only 5 out of over 1,000 speakers.

Photo of Aishani Bhalla

Aishani Bhalla


Aishani Bhalla is a software engineer on the Azure Machine Learning team. At Microsoft, she’s helping people operationalize models to build intelligence into every application, accelerated on FPGAs with Project Brainwave. She has a BS in Computer Science from the University of Buffalo.

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)