These days everyone is excited about big data and fast data. Capital One has embraced this new generation of technology with open arms. However, as Edward Heinlein was fond of reminding us, “TANSTAFL—There ain’t no such thing as a free lunch.”
For many years, there’s been a very real battle around the standard operating model of software. Tech giants like Oracle and IBM have traditionally built massively expensive enterprise-ready products, while the open source community provides free, albeit usually inferior, software. For a product to be enterprise-ready, it must guarantee complete reliability alongside performance and flexibility. There are notable successes in the open source world such as Linux and Open SSH/SSL, but the realm of distributed stream computing has lacked comparable solutions.
Capital One set out to find whether we could build or find enterprise-ready technology in the open source world to tackle difficult streaming problems that also provides equivalent performance, durability, and availability as a mainframe computer. Ilya Ganelin details Capital One’s attempt to answer this question in a rigorous and complete way, not just by making a prototype or discovering exciting new tools, but by creating an open source-based, enterprise-ready product that can transparently replace an enormously expensive proprietary solution. Ilya presents Capital One’s novel solution for real-time decisioning on Apache Apex.
Ilya Ganelin is a roboticist turned data engineer. After a few years building self-discovering robots at the University of Michigan and another few years working on embedded DSP software with cell phones and radios at Boeing, he landed in the world of big data at the Capital One Data Innovation Lab. Ilya is an active contributor to the core components of Apache Spark and a committer to Apache Apex with the goal of learning what it takes to build a next-generation distributed computing platform. Ilya is an avid bread maker, cook, skier, and race-car driver.
©2016, O'Reilly Media, Inc. • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • firstname.lastname@example.org
Apache Hadoop, Hadoop, Apache Spark, Spark, and Apache are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries, and are used with permission. The Apache Software Foundation has no affiliation with and does not endorse, or review the materials provided at this event, which is managed by O'Reilly Media and/or Cloudera.