Data lakes and distributed systems: The truths and myths

Jesus Jackson (eGlobalTech)

2:15pm–3:05pm Tuesday, February 25, 2020

Location: Beekman Parlor

Data & Security, Distributed systems

Secondary topics: Anti-Pattern, Best Practice, Overview

Average rating:

(4.00, 4 ratings)

Download slides (PPTX)

Who is this presentation for?

Software developers, architects, infrastructure engineers, directors, CTOs, and CDOs

Level

Beginner

Description

In his time designing and deploying large-scale data lakes and distributed systems for clients including Fortune 100 companies and federal agencies, Jesus Jackson has learned many hard truths (and techniques that worked well across all of his projects) as well as some solutions that rarely ever work (the myths).

Join in to explore these great techniques and approaches for designing a data lake, such as how to integrate data across multiple indices in real time to provide a performant user experience when searching across datasets, securely manage highly sensitive data using cell-level security and encryption, implement real data governance and provenance to really understand your data and how it’s being disseminated across the lake, and more. Jesus also dives into the myths—explaining why they rarely work and providing alternatives—along with a few (humorous) war stories.

Prerequisite knowledge

General knowledge of distributed systems (useful but not required)

What you'll learn

Learn how to design and build a highly available, distributed system
Identify the dos and don'ts of distributed systems
Discover techniques and approaches that work well and those that don't

Jesus Jackson

eGlobalTech

Jesus Jackson is the senior director of technology strategy at eGlobalTech and head of eGT Labs, eGlobalTech’s tech innovation hub; he leads product development at the company. He specializes in big data, distributed systems, microservices, data science, and full stack development and integration, with expertise in leading, building, and deploying user-centric and scalable products to millions of users. Jesus holds a BS in computer science and mathematics from the University of Maryland, College Park.