We describe a case study in aircraft fleet management, where we needed a database that would store data about all the different parts and subunits of an aircraft. A single aircraft already contains some 6,000,000 parts, not counting components.
The particular structure of the queries arising naturally from day-to-day processes quickly led to the insight that no single data model was sufficient to ensure satisfactory performance. Whichever data model we tried, there was always at least one important query that would either take ages to complete or be unbearably complicated, or indeed both.
After a lot of frustration, the happy end could be achieved by using a multi-model database, which is a document store, a graph database, and a key/value store – all in one engine – held together by a common and flexible query language. Suddenly, all pieces fell into place and all required queries could be performed not only quickly, but also conveniently!
In this approach, parts and subunits are represented by JSON documents, which are the vertices of a graph, and the edges of the graph describe the containment. In this way, using graph queries, one can quickly find out, for example, what the smallest replaceable component containing a certain broken part is. As a further example, using document queries and secondary indexes, one can quickly find all documents corresponding to parts or subunits that need maintenance in the next week.
In this talk we tell the story of this case study, and explain why it was a real eye-opener that taught us a lesson about data modeling in general as well as database engines in particular, that will help us in a large class of other use cases.
Max Neunhöffer is a mathematician turned database developer. In his academic career he has worked for 16 years on the development and implementation of new algorithms in computer algebra, mainly for the open source system GAP. During this time he has juggled a lot with mathematical big data like group orbits containing trillions of points. Recently he has returned from St Andrews in Scotland to Germany, has shifted his focus to NoSQL databases, and now helps to develop ArangoDB.
©2015, O’Reilly UK Ltd • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • firstname.lastname@example.org
Apache Hadoop, Hadoop, Apache Spark, Spark, and Apache are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries, and are used with permission. The Apache Software Foundation has no affiliation with and does not endorse, or review the materials provided at this event, which is managed by O'Reilly Media and/or Cloudera.