The PayPal data ecosystem is fairly large, with 250+ PB of data on a polygot ecosystem of data stores transacting in 200+ countries, supported by some of the largest installations of Oracle, Hortonworks, and Aerospike clusters. Given this massive scale and complexity, discovering and accessing the right datasets, being able to secure this data at the desired latency, and a frictionless development environment for data analysts and data scientists became imperative.
Subhadra Tatavarti and Chen Kovacs explain how PayPal’s data platform team is helping solve this problem holistically with a combination of a self-service data integration platform—which consists of a data integration layer that moves data at scale, a data framework layer using Gimel, a single unified API that can be used to access data stored on any data store supported, and a frictionless IDE environment that brings all these together—and PayPal’s customized Jupyter notebooks environment known as PPNotebooks. PayPal Notebooks takes the versatility and power of Jupyter and enhances it for enterprises with features like one-click access to any Hadoop environment, built-in scheduling using Apache Airflow, collaboration and sharing with seamless integration with GitHub, and native publishing to Tableau.
Subhadra Tatavarti leads strategy and product for data platforms and infrastructure at PayPal. Her team manages and propels the data platforms that power PayPal’s core customers, processing over 250 PB of data, and builds products that cater to over 5,000 PayPal developers, analysts, and data scientists—with the goal to not just enable this community but also drive efficiency, reduce friction, and reduce time to market, which in turn drives PayPal’s growth. Subhadra is an experienced leader of large organization-wide transformations that drive innovation and accelerate business delivery.
©2019, O'Reilly Media, Inc. • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • firstname.lastname@example.org