Presented By O'Reilly and Cloudera
Make Data Work
5–7 May, 2015 • London, UK

Scale out databases for CERN use cases

11:45–12:25 Wednesday, 6/05/2015
Hadoop Platform
Location: King's Suite - Sandringham
Average rating: ****.
(4.50, 4 ratings)
Slides:   1-PPTX 

Prerequisite Knowledge

Basic understanding of shared-nothing systems like Hadoop and relational databases

Description

Data generation rates are expected to grow very fast for some database workloads going into the second run of the Large Hadron Collider and beyond. In particular this is expected for data coming from controls, logging, and monitoring systems. Storing, administering, and accessing big data sets in a relational database system is in certain cases very demanding on the technology and therefore on cost. Thus, there is high interest in the CERN database community to find alternative solutions to relational database systems for storing and querying big data volumes with fast and scalable data access time. Scale-out database engines are an emerging and rapidly developing area. Recently a technical solution that has attracted attention is Cloudera Impala with columnar storage provided by Parquet on top of the Hadoop Distributed File System. This solution has the additional benefit of offering SQL as the main data access interface, which makes it easy to integrate with existing client applications. In this presentation we will discuss the results of our tests with the Cloudera Impala data querying engine, including tests of data loading and integration with existing data sources, notably Oracle databases. We will report on query performance tests done with various data sets of interest at CERN, especially the accelerator log database.

Photo of Zbigniew Baranowski

Zbigniew Baranowski

CERN

Zbigniew Baranowski is a database systems specialist and a member of a group which provides and supports central database services at CERN.

Comments on this page are now closed.

Comments

Igar Nakhvat
15/05/2015 21:36 BST

Hi! Can you send me your presentation? My email is i_nakhvat@wargaming.net Many thanks!