Making Open Work
May 8–9, 2017: Training & Tutorials
May 10–11, 2017: Conference
Austin, TX

The index as a first-class citizen

Matthew Jaffee (Pilosa)
11:00am11:40am Wednesday, May 10, 2017
Sponsored
Location: Meeting Room 13 (Sponsored)
Average rating: ****.
(4.50, 2 ratings)

What you'll learn

  • Explore Pilosa, an open source distributed, sparse bitmap index that exists as an acceleration layer over existing data stores

Description

A database is traditionally responsible for both storing and indexing data so that it can be kept safely and accessed quickly. Modern at-scale software architecture has increasingly tended toward breaking things apart (e.g., microservices), which databases have, with few exceptions, resisted.

Matthew Jaffee offers an overview of Pilosa, an open source distributed, sparse bitmap index that exists as an acceleration layer over existing data stores, which is being successfully used in production to accelerate queries that were otherwise impractical due to high latency and excess memory use. Pilosa can be used to speed up certain queries to existing databases, or make joining data from multiple stores much faster.

Matthew covers some background on databases and indexes and discusses the pros and cons of separating the index from the storage before diving into a general overview of Pilosa and a demonstration of how it can be used to reduce latency and enhance data exploration.

This session is sponsored by Pilosa.

Matthew Jaffee

Pilosa

Matthew Jaffee is a lead slothware engineer at Pilosa. His job experience ranges from writing some of the most inscrutable Javascript and CSS you’ll ever read to using GPUs to inspect high rate network traffic. Previously, Matthew was the lead platform engineer at Umbel, where he worked on the streaming data ingest, storage, and query code that supported the product. He holds an MS in computer science with a focus on networking and distributed systems.