Portability and performance in embedded deep learning: Can we have both?

Description

In recent years, there has been lots of work done on low-precision inference that shows that by training for quantization, large gains in energy efficiency can be achieved. On the other hand, we have seen embedded runtime packages like TensorFlow Lite and Caffe2Go emerge that offer portability over a number of platforms. Cormac Brick look at the challenge presented by this choice and asks, “Why can’t we have both?” Cormac explains how big this gap truly is, using state-of-the-art methods for both and specifically trained networks to show performance over a range of popular vision applications. ­He then covers best-in-class design techniques for developing portable networks to maximize performance on different compute architectures variety of network architectures and shares industry challenges and progress needed to close the portability performance gap.