Presented By O'Reilly and Cloudera
Make Data Work
Dec 4–5, 2017: Training
Dec 5–7, 2017: Tutorials & Conference

Fusing a deep learning platform with a big data platform

YONGLIANG XU (StarHub), Masatake Iwasaki (NTT DATA)
1:45pm2:25pm Thursday, December 7, 2017
Average rating: *****
(5.00, 1 rating)

Who is this presentation for?

  • Developers and technical managers

Prerequisite knowledge

  • Basic knowledge of Hadoop-like distributed computing infrastructures

What you'll learn

  • Gain insights the application of a reference architecture for distributed deep learning from a telco perspective


SmartHub, the analytics division of StarHub, and NTT DATA, a global IT innovator in Japan with committers to Hadoop and Spark, have embarked on a partnership to design next-generation architecture to power the data products that will help generate new insights. YongLiang Xu and Masatake Iwasaki explain how deep learning and other analytics models can coexist on the same platform to address opportunities and challenges in initiatives such as smart cities.

Deep learning is the next key-enabler to transform data into actionable analytics products. However, big data platforms using technologies such as Hadoop and Spark remain the backbone for analytic applications. Therefore integrating big data platforms with deep learning technologies like TensorFlow is crucial to support the development of cutting-edge data analytics products.

YongLiang and Masatake present a reference architecture that incorporates distributed deep learning with an existing big data platform through frameworks such as Intel BigDL and TensorFlowOnSpark. This architecture creates an environment in which deep learning workloads can coexist with other existing analytics workloads and continue to leverage the same real-time data pipeline and monitoring frameworks within the platform.

Topics include:

  • An overview of the reference architecture that incorporates distributed deep learning with existing big data platforms
  • Key considerations on why SmartHub and NTT DATA have chosen the above architecture
  • Case studies on the integration and deployment of deep learning models on the integrated architecture
  • Future works and plans



YongLiang Xu is the lead data architect for SmartHub, the analytics division of StarHub, where he is responsible for transforming and architecting the next generation of big data architecture. His work includes reengineering SmartHub’s big data platform for real-time processing to support real-time machine learning and experimenting with new Apache projects and optimizing the big data platform for streamlined and seamless performance. Previously, YongLiang was a software engineer at DSO National Laboratories, Singapore, where he developed solutions based on big data technologies.

Photo of Masatake Iwasaki

Masatake Iwasaki


Masatake Iwasaki is a software engineer at NTT DATA, where he works on OSS professional services, including consulting, system integration, and technical support of open source software
such as Hadoop, Spark, and Storm for enterprise systems. He is also a committer for Apache Hadoop and Apache HTrace (incubating).