Engineering the Future of Software
April 2–3, 2017: Training
April 3–5, 2017: Tutorials & Conference
New York, NY

Migrating an architecture across batches and streams

Michelle Brush (Cerner Corporation)
2:15pm–3:05pm Wednesday, April 5, 2017
Distributed systems
Location: Beekman Parlor
Level: Intermediate
Average rating: ****.
(4.60, 5 ratings)

Who is this presentation for?

  • Software architects, senior software engineers, and engineering executives

Prerequisite knowledge

  • Experience making major design decisions for at least one large system

What you'll learn

  • Explore how Cerner migrated from batch to streams
  • Understand how changing your architectural paradigm impacts everything that surrounds your system
  • Learn advice to mitigate the impact to your system

Description

Cerner’s Population Health platform, HealtheIntent, brings together the world’s healthcare data to derive the complete picture of a person’s health. This means normalizing and standardizing healthcare data from a variety of source systems, including electronic medical records, physician ordering systems, and even payer billing and claims systems.

The HealtheIntent platform grew up as a batch system. Most of the processing occurred hourly. Some data even landed only daily or monthly. While there were some technical and operational challenges, the batch world was (mostly) forgiving and (mostly) predictable. Then everything changed.

Over the last year, Cerner has shifted its Hadoop data processing pipeline to a streaming one using Storm and Kafka. The change was born out of a need to have a near-real-time system, so the decision to shift the architecture was easy. In terms of the impact to the design and the code, on the surface the change seemed straightforward, but in the end, it required that the team rethink everything: support, monitoring, alerting, performance measurement, and even team structure and associate onboarding. Michelle Brush shares the lessons learned executing a large-scale architectural migration and provides advice on how to minimize the impact.

Photo of Michelle Brush

Michelle Brush

Cerner Corporation

Michelle Brush is a math geek turned computer geek with 15 years of software development experience. Michelle has developed algorithms and data structures for search, compression, and data mining in embedded as well as enterprise systems. In her current role as an engineering director for Cerner Corporation, Michelle leads teams that develop the platform for ingesting stream and batch data specific to Cerner’s Population Health solutions. She is also the chapter leader for the Kansas City chapter of Girl Develop It and one of the conference organizers for Midwest.io.

Comments on this page are now closed.

Comments

Junsik Shim | SENIOR ENGINEER
04/19/2017 3:55pm EDT

When will the slides be posted?