A database is traditionally responsible for both storing and indexing data so that it can be kept safely and accessed quickly. Modern at-scale software architecture has increasingly tended toward breaking things apart (e.g., microservices), which databases have, with few exceptions, resisted.
Matthew Jaffee offers an overview of Pilosa, an open source distributed, sparse bitmap index that exists as an acceleration layer over existing data stores, which is being successfully used in production to accelerate queries that were otherwise impractical due to high latency and excess memory use. Pilosa can be used to speed up certain queries to existing databases, or make joining data from multiple stores much faster.
Matthew covers some background on databases and indexes and discusses the pros and cons of separating the index from the storage before diving into a general overview of Pilosa and a demonstration of how it can be used to reduce latency and enhance data exploration.
This session is sponsored by Pilosa.
©2017, O'Reilly Media, Inc. • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • firstname.lastname@example.org