Presented By O'Reilly and Cloudera
Make Data Work
March 13–14, 2017: Training
March 14–16, 2017: Tutorials & Conference
San Jose, CA

FireEye's journey migrating 25 TB of RDBMS data to Hadoop

Ganesh Prabhu (FireEye), Vivek Agate (FireEye), Alex Rivlin (FireEye)
4:20pm5:00pm Wednesday, March 15, 2017
Enterprise adoption
Location: 230 A Level: Beginner

Who is this presentation for?

  • Engineers and project managers considering big data implementations

Prerequisite knowledge

  • General familiarity with RDBMS

What you'll learn

  • Understand the approach, plan, tools, and technologies useful for big data implementation

Description

FireEye has a consolidated RDBMS data platform used for research and analytics. It’s currently 25 TB in size with more than 200 tables, and it’s growing at rate of 25 to 30 GB per day. The platform, which holds transaction tables that are terabytes in size each with a billion+ rows, has seen exponential growth (from 1 TB in 2013 to 25 TB in 2016) in the last three years. FireEye realized that it had to start exploring new avenues for the platform since a traditional RDBMS platform was unable to scale to the current volume of data.

Ganesh Prabhu, Alex Rivlin, and Vivek Agate present FireEye’s journey migrating data from RDBMS to Cloudera’s big data Hadoop platform. The migration was completed with a very lean team of two engineers and a single administrator in a very short period of time. Ganesh, Alex, and Vivek cover the plan’s details, migration approach, and tools implemented to help with the migration and explore the challenges faced along the way. If you are are looking at embarking on this RDBMS to Hadoop journey, join in to learn from FireEye’s experience.

Photo of Ganesh Prabhu

Ganesh Prabhu

FireEye

Ganesh Prabhu is a staff software engineer at FireEye with 20+ years of RDBMS and engineering experience.

Photo of Vivek Agate

Vivek Agate

FireEye

Vivek Agate is a staff software engineer at FireEye with 8+ years of experience in software design and development in various Java technologies.

Photo of Alex Rivlin

Alex Rivlin

FireEye

Alex Rivlin leads the team responsible for dynamic threat intelligence at FireEye, which includes a crowdsourced malware exchange across FireEye customers and a malware analytics platform supporting FireEye’s research. For two decades, Alex developed novel analytical capabilities for high-tech companies in projects such as semiconductor failure analysis, supply chain optimization, pricing, and cybersecurity. Previously, Alex worked at Altera (currently part of Intel), where he was hired to develop analytical platform for semiconductor test. His very first project earned a US patent for optimization of bulk loading of data to RDBMS. He later joined supply chain optimization project, where he was responsible for operational analytics. One of his developments allowed real-time reallocation of materials, a feature not available in any commercial packages. Alex also spent time at Flextronics, where he was in charge of project management for implementation of global procurement solution and master data management.

Comments on this page are now closed.

Comments

Eric Park | ARCHITECT
03/27/2017 3:06pm PDT

Are the slides for this talk online somewhere? There are some links I’d like to follow!