In today’s world, with cyber incidents reported almost daily, security teams are increasingly turning to data for answers. Data-driven approaches can prove immensely valuable in providing visibility to support decision making and drive action across the entire cybersecurity lifecycle. Data platforms to support security operations teams in their reactive efforts to detect and respond to security incidents have existed for a long time, from SIEMs historically and Apache Metron to Spot. In addition, a whole new class of traditionally underserved stakeholders and use cases is emerging from security executives who need strategic decision support to deliver proactive initiatives that measure and mitigate cyber risks.
Designing successful data solutions for the cybersecurity domain can be a daunting task. The diversity of problems to be solved for various stakeholders in and around a security function leads to an array of complex and potentially competing data and analysis requirements. This complexity initially arises from the need to collect and prepare data of any type from wherever it resides and however it’s exposed. The data must be then stored in a way that can accommodate a range of access patterns. Finally, interfaces must exist to promote wide accessibility to allow the range of platform users to analyze the data and consume insights, taking these users’ varying data analysis skill levels into account.
Once you’ve understood your users and their needs, you face the challenge of navigating the vast sea of data technologies vying for your attention to arrive at a solution. But with the proliferation of open source and proprietary technology options, each with their own trade-offs, how do you deliver a scalable and flexible data platform that will serve your security organization for years to come?
Charaka Goonatilake explores the key drivers that influence the architecture of a cyber data platform and explains how to deliver on these requirements using open source big data technologies like Spark and the Hadoop ecosystem. Charaka walks you through real-life lessons learned and the successes and failures experienced while building and evolving data platforms.
Charaka Goonatilake is CTO at Panaseer, where he designs and delivers big data solutions that enable chief information security officers and their teams to gain visibility into the true state of security within their business to improve cyber hygiene and reduce cyber risk exposure. Charaka has been immersed in big data technologies since the very early days of Hadoop and has hands-on experience using Hadoop in the enterprise to produce data-driven insights. Over the past eight years, across Panaseer and BAE Systems Applied Intelligence, Charaka has architected and engineered Hadoop-based data platforms for a range of cybersecurity use cases, from security analytics for threat detection to threat intelligence management and cybersecurity risk management.
©2018, O’Reilly UK Ltd • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • firstname.lastname@example.org