SURFSara and Intel collaborated as part of the Intel Parallel Computing Center initiative to advance the state of large-scale neural network training on Intel Xeon CPU-based servers. The team evaluated data and model parallel approaches and synchronous versus asynchronous SGD methods with popular neural networks such as ResNet50 using large datasets on the TACC (Texas Advanced Computing Center) and Dell HPC supercomputers.
Ananth Sankar, Valeriu Codreanu, Damian Podareanu, and Colin Healy share insights on several best-known methods, including CPU core, memory pinning, and hyperparameter tuning that the team developed to demonstrate top-one/top-five state-of-the-art accuracy at scale. Ananth, Valeriu, Damian, and Colin then explore real-world problems that can be solved by utilizing models efficiently trained at large scale and present tests performed at Dell EMC on CheXNet, a Stanford University project that extends a DenseNet model pretrained on the large-scale ImageNet dataset to detect pathologies in chest X-ray images, including pneumonia. They highlight improved time to solution on extended training of this pretrained model and investigate whether various storage and interconnect options lead to more efficient scaling.
Ananth Sankaranarayanan is a senior director of AI technical acceleration and scaling at Intel, where he’s responsible for leading a worldwide team that enables customers and partners to build AI solutions with the maximum benefit of Intel hardware and software capabilities. Previously, he led the creation of the big data analytics solutions team and championed multiple high-growth industry-first solutions in intelligent transportation, healthcare, retail, and financial services segments and drove them to a worldwide scale. Ananth won the “Intel Achievement Award,” the highest employee recognition, for his transformative work in the creation of high-performance computing (top 500 supercomputer) production capability to accelerate silicon design and manufacturing. Ananth earned his bachelor’s of engineering in computer science and his master’s degree in business administration from the City University of Seattle. Ananth is a strong supporter of improving education, and as a portion of his volunteer work, he coaches middle and high school robotics and STEM teams who have won national-level recognitions for engineering and teamwork. Ananth holds two patents and has coauthored more than 10 publications and a book, AI for Autonomous Networks, the proceeds of which go to the Girls Who Code nonprofit organization.
Valeriu Codreanu is the PI of the Intel Parallel Computing Center at SURFsara, focusing on optimizing deep learning techniques using the Intel ecosystem as well as extending the use of these techniques to other scientific domains. Previously, he was an HPC consultant at SURFsara, focusing on machine learning and a postdoctoral researcher at both Eindhoven and Groningen Universities, working on GPU computing, computer vision, and embedded systems in the scope of several EU-funded projects. Valeriu holds an MSc in electrical engineering and a PhD in computer architecture from the Polytechnic University of Bucharest.
Damian Podareanu is a co-PI for the Intel Parallel Computing Center at SURFsara and an HPC consultant for the Deep Learning for HPC and Efficient Deep Learning projects. Since 2017, he’s also been leading the Quantum Computing and Quantum Internet project. Damian focuses on optimizations and efficient scaling of machine learning algorithms. Previously, he was an AI researcher. Damian studied mathematics and computer science at the University of Bucharest, high-performance computing at the Polytechnic University of Bucharest, and artificial intelligence at the University of Groningen.
Qualified in Industrial Computing and Engineering. Working 25 years with GPU’s in Application and Compute across Poweredge Server & Precision Workstation
©2018, O’Reilly UK Ltd • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • firstname.lastname@example.org