Build resilient systems at scale
May 27–29, 2015 • Santa Clara, CA

Systems thinking – patterns for scaling unstable systems

Siddharth Ram (Intuit)
1:45pm–2:25pm Friday, 05/29/2015
Location: Ballroom F
Average rating: ****.
(4.60, 5 ratings)
Slides:   external link

Prerequisite Knowledge

Basic understanding of distributed computing and SOA.

Description

Scale and failure can be seen as inversely proportional. If you want to scale massively, you have to accept failures in the system, and design for components in your system to fail. SOA requires us to embrace the concepts of distributed computing: that everything around us will fail, and we need to design for failure cases, not happy paths.

There is a set of standard patterns used at Intuit for scaling in this environment. Loose coupling is enabled using patterns like service decoupling. Horizontal duplication and taking advantage of read/write ratios enable theoretically infinite capacity. Formulaic splits of data and a shared nothing approach enable management of a failure’s blast radius.

This presentation will cover patterns used at Intuit that allow us to manage the characteristics of instability. Right technology patterns, right cultural changes, and right mindsets have moved a large organization from a scaled-up, shared everything to a scaled-out, shared nothing architecture.

The presentation will treat Intuit’s journey as a case study — a large cultural, technological, and organizational change in progress at a 30-year-old software company. It will uncover the pitfalls and the patterns that must be thought through in designing for failure. It will include results from a large cultural shift at Intuit.

Topics covered will include:

  • How systems drift into failure
  • Focusing on interconnects as failure points rather than elements
  • How to loosely couple services
  • Scaling your system using formulaic splits and horizontal duplication
  • Addressing the fallacies of distributed computing
  • Applying the service decoupling pattern
  • Solving for polyglot environments using sidecars
  • End-to-end ownership

Conclusions:
Don’t let failures happen to you — take charge by planning resilience and assuming failures. Lead your organization to a place where you embrace failures rather than fight them. You will be more productive, sleep better, and have greater availability by doing so.

This session is sponsored by Intuit.

Photo of Siddharth Ram

Siddharth Ram

Intuit

Siddharth Ram is chief architect of the Small Business group at Intuit. He is responsible for the technology and architecture of a group with 1200+ engineers, covering products for small business — accounting, payments, payroll, and front office — across the globe. This includes products like QuickBooks, DemandForce, and QuickBase

Prior to Intuit, Siddarth was at a number of startups including his own. At Qualcomm, he worked on layer 2/layer 3 software for telecom networks and managed a large distributed computing environment. He has a Bachelors degree in Electrical Engineering from the National Institute of Technology, India and a Masters in Electrical Engineering from Stony Brook University.

Comments on this page are now closed.

Comments

Picture of Siddharth Ram
Siddharth Ram
05/29/2015 12:24pm PDT

Vinaya, thanks for the feedback. I ran out of time to present that material – I will plan on a blog post to share real life situations

Vinaya Nadig
05/29/2015 9:11am PDT

I would have loved it more, if we had a few more examples of some scenarios where some actions were taken to ‘minimize unnecessary complexity’ and also ‘how to plan for failures’. Examples of what was existent and how it was improved to brace failures, would have been really interesting.

Picture of Siddharth Ram
Siddharth Ram
05/29/2015 8:33am PDT

Hi Ben,

The deck can be found at http://www.slideshare.net/SiddharthRam2/scaling-unstable-systems-velocity-2015

Cheers
Siddharth

Ben Vanderhoff
05/29/2015 7:31am PDT

Can you post your slide deck to slideshare?