Presented By O'Reilly and Cloudera
Make Data Work
Feb 17–20, 2015 • San Jose, CA

Making Big Data Usable in Market Regulation

Scott Donaldson (FINRA)
4:50pm–5:30pm Thursday, 02/19/2015
Business & Industry
Location: LL21 B

FINRA Market Regulation’s mission is to detect, discipline, and deter market manipulation, and to ensure market integrity on the U.S. equities, fixed income, and options markets. FINRA acquires approximately 30 billion market events per day that are reconstructed for detection and analysis. A key component in this analysis is an Order Lifecycle, which is the graph of all equity market events – from the orders submitted by individuals, institutions, or firms, through the fragmented mesh of exchanges, dark pools and over-the-counter trading systems, and to the final order executions.

The business impact of big data solutions largely depends on the usability and relevance of the analytics provided to users. FINRA has successfully adapted an agile SDLC with continuous user engagement in delivering a solution capable of interrogating over a trillion-record database. This approach achieved a successful product launch and is ensuring reliable operations as the solution becomes ingrained into the regulatory specialists’ daily procedures.

In this presentation, Scott Donaldson of FINRA Market Regulation Technology will focus on: challenges faced with existing data warehouse appliance solutions not capable of meeting the Market Regulation Department’s business objectives; the architecture solution whose fast, interactive style allowed regulatory analysts to conduct more robust regulatory reviews with altogether new query capabilities; lessons learned in the process of making big data more usable by providing a rich user experience to ensure a successful rollout; and the next phase in overhauling FINRA’s market surveillance and analysis platforms.

Photo of Scott Donaldson

Scott Donaldson


Scott is Senior Director at FINRA’s Market Regulation Technology. Scott leads the data and analytics teams responsible for the surveillance of U.S. equities, options and fixed income markets.