Unifying Your Data Management Platform with Hadoop: Batch and Real-time Machine Data Ingest, Alerts, and Analytics

Hadoop has evolved significantly in recent years, today serving as a unified platform for near-real-time (NRT) and batch workflows, such as querying, analysis and alerting for logs and machine data.

In this session, we’ll dive into the details of using SolrCloud and Cloudera Impala together to serve search queries, by integrating Flume to stream events into Solr, Impala and HBase. Such a system supports the definition and extraction of data fields that can be incorporated into the user experience at scale. It also allows definition of complex alerts over large datasets, again scaling with the right technologies underneath. Finally, we will explain in detail the front-end layer that gives users a consistent view across data in Solr, Impala and HBase. The beauty of such a system lies in its scalability, low total cost of ownership, and flexibility to allow the same data to be used for building out other features and products.