Orchestrating the Cloud
Location: Pavilion 9 - 10 Audience level:
Constraints are everywhere when working with high throughput tools, be they video encoding, indexing, image processing, parsing or complex scientific analysis. As scale increases so network, disk I/O, CPU performance and utilization all become larger and larger barriers to to actually getting work done.
This talk introduces the use of elastic, scalable cloud approaches as a set of productivity tools for constrained, highly flexible domains. In the same way as the abstractions in Ruby and Rails have come to reduce complexity and speed development in software, so infrastructure abstractions can help those dealing with web scale data overcome these constraints, and return triumphantly to the Important Stuff.
Through real world examples, we’ll explore how Ruby and Rails can play a central part in the orchestration, management and monitoring of scalable computation at petabyte scale. Over the course of the session, we’ll aim to cover:
- The productivity tax of web scale applications
- Hard constraints of big data: maintaining availability at scale
- Architectures and models for high throughput systems
- Addressing data as a programmable resource
- Orchestrating cloud architectures with Ruby
- Data on Rails: Managing data sets and workflows
- Fighting the data deluge: building a genome with Mission Control
This talk will be of interest to anyone looking to perfect distributed computation of any kind, or to scale their existing apps up as their businesses grow. Join us, and the sequencing team from the Wellcome Trust Sanger Institute, to demonstrate how cloud approaches can be applied at the razor’s edge of data throughput using open source Ruby code.
People planning to attend this session also want to see:
Wellcome Trust Sanger Institute
Matt Wood heads up software development at the Wellcome Trust Sanger Institute, where he is responsible for the software that drives the Institute’s world class sequencing facility, along with the next generation of DNA sequencing technology.
Matt’s interests lie in adding value to biological data through scientific software: after completing his PhD, he’s built grid services in Nottingham, web-scale search engines in New York City and genes in Hinxton.
Matt is an advocate of agile development with Scrum, sensible semantics and, of course, the duck billed platypus.