Billions of Visa cards are used around the world to make payments. Each payment transaction has a story. Getting payments from point A to point B is complex, and the resulting data Visa captures reflects this. The scale and complexity of that data is a direct manifestation of the number, variety, and complexity of payment transactions processed by the Visa network.
As enterprises go, Visa is among the more cautious. Visa systems have stringent availability requirements to ensure payments never fail, designed around proven ideas and technologies. Visa is almost never an early adopter of technologies. Instead, it waits for technologies to harden and prove themselves.
Nandu Jayakumar explores the adoption of big data practices at Visa and explains how Visa is transforming the way it manages data: database appliances are giving way to Hadoop and HBase; proprietary ETL technologies are being replaced by Spark; and enterprise warehouse data models will be complemented by flexible data schemas. Due to the nature of its business, regulatory compliance and secure management of data are central tenets at Visa. Many open source and less mature technologies tend to fall short in these aspects. Embracing big data technologies has also required a culture change in the engineering teams and the ultimate users of data in the company. Nandu also discusses Visa’s experience with Impala supporting a large reporting setup.
Nandu Jayakumar is a software architect and engineering leader at Oracle. Before that he was responsible for the long-term architecture of data systems and was Senior Director of data platform development at Visa. Previously, as a senior leader of Yahoo’s well-regarded data team, Nandu built key pieces of Yahoo’s data processing tools and platforms over several iterations, which were used to improve user engagement on Yahoo websites and mobile apps. He also designed large-scale advertising systems and contributed code to Shark (SQL on Spark) during his time there. Nandu holds a bachelor’s degree in electronics engineering from Bangalore University and a master’s degree in computer science from Stanford University, where he focused on databases and distributed systems.
©2016, O'Reilly Media, Inc. • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • email@example.com
Apache Hadoop, Hadoop, Apache Spark, Spark, and Apache are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries, and are used with permission. The Apache Software Foundation has no affiliation with and does not endorse, or review the materials provided at this event, which is managed by O'Reilly Media and/or Cloudera.