The Financial Industry Regulatory Authority (FINRA) acquires approximately 30 billion market events per day that are ingested, aggregated, and analyzed for the purpose of performing surveillance of the US markets. FINRA uses cloud computing and leverages big data technology to process an ever-increasing volume of financial data.
FINRA’s Market Regulation Surveillance Program runs hundreds of surveillance algorithms and patterns daily against hundreds of terabytes of market data to detect market manipulation, compliance breeches, and other potentially illegal activities. FINRA is leveraging a Hadoop architecture running on Amazon Web Services (AWS) cloud infrastructure. This architecture provides elastic capacity so that FINRA can easily handle dynamic workloads while benefiting from operational economies of scale.
This talk will focus on how FINRA is successfully migrating a large portfolio of SQL-oriented batch analytics jobs from an in-house proprietary MPP database appliance platform to a cloud based SQL over the Hadoop platform, consisting of Hive, Amazon EMR, and S3. This presentation covers the following:
Jaipaul Agonus is a director in the Market Regulation Technology Department at FINRA. Jaipaul is a big data engineering leader with nearly 18 years of IT industry experience, specializing in big data analytics and cloud-based solutions. He’s currently involved in building next-generation big data market analytic platforms with machine learning, advanced visualization, and contextual access across applications.
Comments on this page are now closed.
©2015, O'Reilly Media, Inc. • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • email@example.com
Apache Hadoop, Hadoop, Apache Spark, Spark, and Apache are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries, and are used with permission. The Apache Software Foundation has no affiliation with and does not endorse, or review the materials provided at this event, which is managed by O'Reilly Media and/or Cloudera.