Presented By O’Reilly and Intel AI
Put AI to work
8-9 Oct 2018: Training
9-11 Oct 2018: Tutorials & Conference
London, UK

Efficient neural network training on Intel Xeon-based supercomputers

Ananth Sankar (Intel), Valeriu Codreanu (SURFsara), Damian Podareanu (SURFsara), Colin Healy (Dell EMC)
14:35–15:15 Thursday, 11 October 2018
Location: King's Suite - Sandringham
Secondary topics:  Edge computing and Hardware, Platforms and infrastructure

What you'll learn

  • Understand best-known methods for neural network training
  • Learn real-world use cases for models efficiently trained at large scale
  • Explore results from tests performed on Stanford's CheXNet project

Description

SURFSara and Intel collaborated as part of the Intel Parallel Computing Center initiative to advance the state of large-scale neural network training on Intel Xeon CPU-based servers. The team evaluated data and model parallel approaches and synchronous versus asynchronous SGD methods with popular neural networks such as ResNet50 using large datasets on the TACC (Texas Advanced Computing Center) and Dell HPC supercomputers.

Ananth Sankar, Valeriu Codreanu, Damian Podareanu, and Colin Healy share insights on several best-known methods, including CPU core, memory pinning, and hyperparameter tuning that the team developed to demonstrate top-one/top-five state-of-the-art accuracy at scale. Ananth, Valeriu, Damian, and Colin then explore real-world problems that can be solved by utilizing models efficiently trained at large scale and present tests performed at Dell EMC on CheXNet, a Stanford University project that extends a DenseNet model pretrained on the large-scale ImageNet dataset to detect pathologies in chest X-ray images, including pneumonia. They highlight improved time to solution on extended training of this pretrained model and investigate whether various storage and interconnect options lead to more efficient scaling.

Photo of Ananth Sankar

Ananth Sankar

Intel

Ananth Sankaranarayanan is the head of AI solutions engineering in the AI Products Group at Intel Corporation, where he is responsible for coengineering AI platforms, accelerating AI performance on Intel products, and scaling solutions to worldwide customers and partners across cloud service providers, enterprise, HPC, and communication service providers. Ananth has held a number of engineering leadership roles at Intel since he first started in 2001. Previously, he led Intel’s big data analytics Solutions team. He received the Intel Achievement Award for delivering Intel’s first production high-performance computing capability and more than 30 divisional recognition awards. Ananth holds a BE in computer science and engineering and an MBA in information systems. He has been awarded two patents and has authored several technical publications.

Photo of Valeriu Codreanu

Valeriu Codreanu

SURFsara

Valeriu Codreanu is the PI of the Intel Parallel Computing Center at SURFsara, focusing on optimizing deep learning techniques using the Intel ecosystem as well as extending the use of these techniques to other scientific domains. Previously, he was an HPC consultant at SURFsara, focusing on machine learning and a postdoctoral researcher at both Eindhoven and Groningen Universities, working on GPU computing, computer vision, and embedded systems in the scope of several EU-funded projects. Valeriu holds an MSc in electrical engineering and a PhD in computer architecture from the Polytechnic University of Bucharest.

Photo of Damian Podareanu

Damian Podareanu

SURFsara

Damian Podareanu is a co-PI for the Intel Parallel Computing Center at SURFsara and an HPC consultant for the Deep Learning for HPC and Efficient Deep Learning projects. Since 2017, he’s also been leading the Quantum Computing and Quantum Internet project. Damian focuses on optimizations and efficient scaling of machine learning algorithms. Previously, he was an AI researcher. Damian studied mathematics and computer science at the University of Bucharest, high-performance computing at the Polytechnic University of Bucharest, and artificial intelligence at the University of Groningen.

Photo of Colin Healy

Colin Healy

Dell EMC

Qualified in Industrial Computing and Engineering. Working 25 years with GPU’s in Application and Compute across Poweredge Server & Precision Workstation

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)