Leading Change in Data Engineering

Neil Martin (comparethemarket.com), Rob Siwicki (comparethemarket.com)
Hadoop Platform
Location: 114
Average rating: ***..
(3.00, 1 rating)
Slides:   1-PDF 

Compare the Market’s Data Architect, Rob Siwicki and Senior Project Manager, Neil Martin will present the lessons learned whilst delivering a successful yet complex multifaceted project to reinvigorate the organization’s data infrastructure. In the process they will illustrate how Hadoop has played a pivotal role in being able to deliver right time high quality data products and how the platform has initiated a step change in agility within the data engineering team.

The challenges of the legacy data infrastructure and the organisational context will be outlined, followed by the actions and techniques used to deliver the new IT enablers, engage stakeholders and ultimately generate a positive and sustainable change in both technology and people in an fast paced e-commerce environment – an environment where units of work are frenetically traded between teams written on post-it-notes and exchanged and tracked on Kanban boards.

Such actions include:

- The development of a technology vision supporting key 7 strategic themes:
- Data Definitions and Governance
- Operational Analytics
- Tactical Data Delivery
- Next Generation Infrastructure
- Data Interfaces
- Customer Relationship Management
- Personalisation

- The creation of a guiding coalition of stakeholders

- Communicating the technology vision

- Establishing a sense of urgency

- Building short term quick wins and confidence with the help of the Cloudera team

- Eliminating obstacles to success

The advantages of Hadoop will be presented with the outcomes of the current phase of the implementation and expected business benefits. The analysis of outcomes will be followed by a prediction of the new challenges the data team will face as the next two years unfold and the continued implementation of the technical strategy unlocks greater potential for the use of Hadoop and CTM’s data.

Compare the Market’s Data Architect, Rob Siwicki and Senior Project Manager, Neil Martin will present the lessons learned whilst delivering a successful yet complex multifaceted project to reinvigorate the organization’s data infrastructure. In the process they will illustrate how Hadoop has played a pivotal role in being able to deliver right time high quality data products and how the platform has initiated a step change in agility within the data engineering team.

The challenges of the legacy data infrastructure and the organisational context will be outlined, followed by the actions and techniques used to deliver the new IT enablers, engage stakeholders and ultimately generate a positive and sustainable change in both technology and people in an fast paced e-commerce environment – an environment where units of work are frenetically traded between teams written on post-it-notes and exchanged and tracked on Kanban boards.

Such actions include:

- The development of a technology vision supporting key 7 strategic themes:
- Data Definitions and Governance
- Operational Analytics
- Tactical Data Delivery
- Next Generation Infrastructure
- Data Interfaces
- Customer Relationship Management
- Personalisation

- The creation of a guiding coalition of stakeholders

- Communicating the technology vision

- Establishing a sense of urgency

- Building short term quick wins and confidence with the help of the Cloudera team

- Eliminating obstacles to success

The advantages of Hadoop will be presented with the outcomes of the current phase of the implementation and expected business benefits. The analysis of outcomes will be followed by a prediction of the new challenges the data team will face as the next two years unfold and the continued implementation of the technical strategy unlocks greater potential for the use of Hadoop and CTM’s data.

The talk will provide insight into how to achieve coordinated technological change in a highly agile IT organization; an organisational function that supports one of the UK’s most recognisable brands. Discover valuable lessons learned and begin to understand how your organization may want to take first steps in its engagement proving and implementing Big Data technology.

Neil Martin

comparethemarket.com

Neil Martin is a Senior Project Manager at comparethemarket.com, part of the BGL Group in Peterborough, England. He has a 17 year career in project management across the financial services and utilities industries, having spent time at Pearl Assurance and EON. Neil has been part of the BGL Group for 7 years, delivering new insurance schemes for several well know global banks, leading the change programme in Legal Services and most recently delivering the data roadmap in comparethemarket.
He resides in Peterborough and is a keen runner. Neil can be contacted at neil.martin@comparethemarket.com.

Rob Siwicki

comparethemarket.com

Rob Siwicki is a Data Architect at comparethemarket.com. Rob’s career has allowed him to deliver business improvements through enterprise architecture and the delivery of data intensive systems in multiple industries including investment banking, financial services, high-tech manufacturing, consulting and retail ecommerce. Rob holds both technology and business masters degrees and likes to apply both skillsets when creating solutions. Rob is a keen sportsman and competes in Ironman Triathlon. Rob can be contacted via linked in uk.linkedin.com/in/robsiwicki