4–7 Nov 2019

Extensible Autonomous Transactions In The World of Microservices

Divya Nagar (Nexmo, The Vonage API Platform), Nicola Giacchetta (Nexmo, The Vonage API Platform)
15:5516:40 Thursday, 7 November 2019
Location: M4/M5
Secondary topics:  Case Study, Framework-focused

Who is this presentation for?

Software Engineers, Software Architectss

Level

Intermediate

Description

Over the last decade or so, there has been a strong movement toward a flexible style of building large systems that have lately been called microservices. When you move to microservices, the data distribution becomes the biggest problem. How to move data transactionally in a distributed architecture of hundreds of microservices while keeping the system in a consistent state. how do we provide services with the data they require correctly in minimal time. This talk describes the similar problem we faced while scaling the services at Nexmo and how are we solving it.

Other ways to solve these problems are either keep a huge shared global database, which is the violation of microservice fundamentals or create an internal Death star in which failure in a single machine you didn’t even know existed can crash your entire system. These solutions are already in place and eventually becomes more and more difficult to modify and scale.

In this talk, I will present a solution which we are using at Nexmo by deconstructing the database and creating an application independent scale and data aware layer. By using this the application can run independently without worrying about the data.

This talk will present the following Data handling patterns:

- Replica management for Consistent & Partition Tolerant in data stores

- Command and Query Responsibility Separation to handle high throughout writes and virtual buckets based replication to manage read hot-spotting on a subset of the data corpus

- Data pre-compute and time-travel to create timeline consistent view of data across multiple distributed systems

- Data sourcing from Database replication mechanisms, local transactions and async relaying to an analytics platform

- By-passing network bottlenecks like centralized load-balancers by using data-placement-aware smart local proxies/sidecars

Prerequisite knowledge

We will try to cover the basics as much as possible but It would be good to have some knowledge of basic distributed systems and microservices

What you'll learn

By implementing the practices and architectural choices described in this talk audience will be able to decouple the applications from the data layer and other data dependencies to reduce cascading system failures. They will also be able to understand the data patterns and how to use them according to their system requirements.
Photo of Divya Nagar

Divya Nagar

Nexmo, The Vonage API Platform

I am Divya Nagar currently working at Nexmo Limited as a Software Engineer. I work with awesome people who are building a scale aware layer at Nexmo to create a highly available infrastructure which can scale. Apart from that. My primary interests are data-centric distributed systems and databases.

Photo of Nicola Giacchetta

Nicola Giacchetta

Nexmo, The Vonage API Platform

I am Nicola Giacchetta currently working at Nexmo Limited as a Senior Software Engineer. I work and manage a team who works on the scale and infra maturity arm of the company.

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)

Contact us

confreg@oreilly.com

For conference registration information and customer service

partners@oreilly.com

For more information on community discounts and trade opportunities with O’Reilly conferences

Become a sponsor

For information on exhibiting or sponsoring a conference

Contact list

View a complete list of O'Reilly Software Architecture contacts