Migrating from a traditional legacy BI system is a challenge. ETL processes have historically consumed much of the effort and many of the resources of traditional BI projects. At the same time, organizations have developed many ETL systems over the years, which often have insufficient documentation and are difficult to manage.
Enric Biosca offers an overview of the eAGLE accelerator, which speeds up migration processes from legacy ETL to big data implementations by enabling auditing, lineage, and translation of legacy code for big data. Along the way, Enric demonstrates how graphing and automatic translation technologies help companies reduce their migration times.
The eAGLE methodology is based on four pillars:
eAGLE uses graph technologies to model and analyze ETL code, which enables the use of KPIs and metrics to measure and quantify ETL code and allows users to visualize the processes in an advanced way to recognize impact, dependencies, data lineage, etc., explore the code, and discover hidden relationships, bottlenecks, and other inefficiencies. Generating a metadata model of the objects of the ETLs allows you to improve your knowledge of your ETLs as well as the objects in your data warehouse (particularly their relationships). Volume and complexity metrics allow an analytical view of the processes.
By analyzing ETLs using these technologies and methodologies, companies can better manage their migration strategy and save time analyzing the code. Performing a classification and translation accelerates the migration as much as 30% in total dedicated effort.
Enric Biosca is a data and analytics manager at the everis Data Innovation Center (eDIN). Enric is a computer engineer with more than 10 years of experience in data and information architecture. He holds a master’s degree in data science from the Universitat de Barcelona.
14 years of professional experience in IT consulting specializing in the areas of Artificial Intelligence,Big Data, Cloud, Business Intelligence, Technical Architectures and IT Strategy. Knowledge of the life cycle of complex transformation projects from conception to implementation.
Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?
Join the conversation here (requires login)
©2018, O’Reilly UK Ltd • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • firstname.lastname@example.org