Under reasonable circumstances, how much can you expect to lose? The financial statistic Value at Risk (VaR) seeks to answer this question. Since its development on Wall Street soon after the stock market crash of 1987, VaR has been widely adopted across the financial services industry. Some organizations report the statistic to satisfy regulations, some use it to better understand the risk characteristics of large portfolios, and others compute it before executing trades to help make informed and immediate decisions.
Estimating VaR can be computationally intensive. As a flexible processing framework with the ability to both scale up to large amounts of data and leverage vast compute resources, Apache Spark is a compelling platform for undertaking financial risk calculations. At Cloudera, we’ve assisted several organizations in using Spark to compute VaR and other financial statistics.
In this talk, we’ll walk through a basic VaR calculation with Spark. The calculation employs the widely used Monte Carlo method, which is useful for modeling portfolios with non-normal distributions of returns. It simulates thousands or millions of random market scenarios, and uses a model to predict the response of the portfolio to each scenario. The talk, which will cover Spark design patterns in time series analysis, visualizing data, and Monte Carlo simulation, aims to give a feel for what it is like to approach financial modeling with Spark.
Sandy Ryza is a senior data scientist at Clover Health. He was previously at Cloudera doing engineering and data science. He is an author of O’Reilly’s Advanced Analytics with Spark, as well as a Spark committer and member of the Hadoop project management committee. He graduated Phi Beta Kappa from Brown University.
©2015, O'Reilly Media, Inc. • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • firstname.lastname@example.org
Apache Hadoop, Hadoop, Apache Spark, Spark, and Apache are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries, and are used with permission. The Apache Software Foundation has no affiliation with and does not endorse, or review the materials provided at this event, which is managed by O'Reilly Media and/or Cloudera.