Lean Big Data for Mobile

Data: Big Data
Location: B118-119
Average rating: *****
(5.00, 3 ratings)

Lean Big Data for Mobile – Assembling a lean data architecture for large-scale mobile device information on a cost conscious budget with a small team

The explosive growth of smart phones around the world is resulting in the next wave of big data. Capturing and analyzing this data requires lean operations, an ability to quickly react to data and scale-capable systems that can grow rapidly with the business.

This talk will cover lessons learned in building transactional, near real time and long haul batch processing systems specifically designed to handle data from millions of mobile devices using exclusively open source software, Python, Java and Scala. Specifically, the talk will cover:

1) Database Polygamy – evolving Urban Airship’s data real time and data wharehousing systems with PostgreSQL, MongoDB, Cassandra, and HBase
2) Staged architecture – move data in tiers of messages using Kafka and Beanstalk
3) Dodging the cloud – building for failure so that you can survive when it happens
4) Choosing tools – how to find the right tool for the right job and how to balance new technologies with increased complexity
5) Remember the business – ad-hoc capabilities and how to get people feedback quickly (Pig, Cascading)
6) Mobile challenges – issues unique to working with data in mobile

Erik Onnen

Urban Airship

Erik is a Senior Engineer and Analytics team lead at Urban Airship, the leading provider of Push, In-App Purchase and Subscription services for mobile applications. He specializes in distributed systems at scale. Prior to joining Urban Airship, Erik was a platform lead for Jive Software where he lead the development of the activity and recommendation engine services.

Comments on this page are now closed.


Mitsuharu Hadeishi
07/26/2011 8:51am PDT

One of the best technical talks I’ve attended in a while. Huge amount of practical info and war stories, tons of technologies covered.