Presented By
O’Reilly + Cloudera
Make Data Work
29 April–2 May 2019
London, UK
Please log in

Transforming a financial services data infrastructure for the modern era by building a PCI DSS-compliant data platform from the ground up on AWS

Eoin O'Flanagan (NewDay), Darragh McConville (Kainos)
11:15–11:55 Thursday, 2 May 2019
Data Engineering and Architecture
 Location: Capital Suite 10/11
Secondary topics:  AI and Data technologies in the cloud, Data Integration and Data Pipelines, Financial Services, Security and Privacy
Average rating: ****.
(4.86, 7 ratings)
Download slides (PDF)

Who is this presentation for?

  • Data engineers and architects

Level

Intermediate

Prerequisite knowledge

  • A basic understanding of data warehousing, ETL, and cloud technologies

What you'll learn

  • Gain an overview of a PCI cloud-based data lake and an architecture utilizing batch and real-time workloads across PaaS, container orchestration, and serverless

Description

Eoin O’Flanagan and Darragh McConville explain how NewDay built a high-performance contemporary data processing platform from the ground up on AWS. Join in to explore the company’s journey from a traditional legacy onsite data estate to an entirely cloud-based PCI DSS-compliant platform.

Eoin and Darragh detail the platform’s design principles, in the context of scale, security, speed of delivery, and proportional costs, share the company’s approach to maximizing security and performance while minimizing cost, and outline how it has prioritized the use of PaaS services and how it combined container orchestration, AWS Lambdas, and AWS DBaaS to minimize the use of IaaS. They also cover how it handles batch and real-time workloads with Kafka Streams and discuss the end-user tooling the company provides to its platform consumers. Along the way, you’ll discover how NewDay keeps the data of over six million credit card customers secure.

Photo of Eoin O'Flanagan

Eoin O'Flanagan

NewDay

Eoin O’Flanagan is a lead data engineer at NewDay, where, for the past couple of years, he has worked as part of NewDay’s digital transformation, specifically in bringing in and enabling new data capabilities. He previously worked at data analytics firm Dunnhumby, where he held roles across data, IT, and architecture.

Photo of Darragh McConville

Darragh McConville

Kainos

Darragh McConville is a solution architect specializing in data engineering at Kainos, where he’s Kainos’s lead architect for NewDay’s AWS data platform. He’s been working with data-intensive systems for over a decade and was the founder of Kainos’s data and analytics capability. He enjoys working with talented people and like every engineer, loves a technical challenge. In his spare time, he’s usually up a mountain or in a squash court but lately has developed an unhealthy fascination with unsolved crimes.