Sep 23–26, 2019

Yu Zhou
Software Development Engineer, Microsoft

Yu Zhou is a software development engineer on the Azure big data team at Microsoft, where he develops innovative big data solutions, including distributing computing systems and streaming computing. He earned his master of science degree in EE from Beijing University of Posts and Telecommunications and his bachelor of science degree in EE from Hunan University.

Sessions

4:35pm–5:15pm Thursday, September 26, 2019

Location: 1E 09

Using Spark to speed up the diagnosis performance for big data applications

Data Engineering and Architecture

Ruixin Xu (Microsoft), Long Tian (Microsoft), Yu Zhou (Microsoft)

Ruixin Xu, Long Tian, and Yu Zhou explore an experiment run using Spark and Jupyter notebooks as a replacement for existing IDE-based tools for internal DevOps. The Spark-based solution improved the diagnosis performance significantly, especially for a complex job with a large profile, and leveraging the Jupyter notebooks brings the benefit of fast iteration and easy knowledge share. Read more.