4–7 Nov 2019

Architecting a distributed system to monitor internet performance on a global scale

Sergio Freitas (ThousandEyes)
11:0011:45 Thursday, 7 November 2019
Location: Hall A5
Secondary topics:  Overview

Who is this presentation for?

  • Software engineers, system architects, site reliability engineers, and CTOs

Level

Intermediate

Description

ThousandEyes provides a platform for collecting, storing, visualizing, and alerting on network performance-related metrics against network targets globally to help its users measure and optimize the impact of dynamic network conditions on services they operate or leverage internally. The platform continuously collects internet data in real time from thousands of global vantage points, processing hundreds of millions of events per hour.

Sergio Freitas explains the progression of the company’s architecture that seamlessly adapted to increasing nonfunctional requirements like performance, volume, and increasingly complex functional requirements without compromising core tenets of simplicity and maintainability. This was possible due to the successful consolidation of various distributed architectural patterns into a unique macroarchitecture that can evolve with changing business needs and new technical challenges. You’ll discover the lessons learned and trade-offs made during this journey of architectural evolution from self-managed bare-metal servers to virtual machines and finally leveraging microservices in a container orchestration platform. Sergio focuses on microservice subarchitecture to scale horizontally without compromising on throughput, latency, and complexity; highly performant and low-latency event-driven architecture to maintain mission-critical service-level agreements (SLAs) at scale with non-trivial data enrichment pipelines; scalable and robust infrastructure to deploy data collection thousands of agents in more than 350 locations worldwide; and seamless, efficient, and reliable storage of structured and unstructured data using a combination of NoSQL and relational databases.

Prerequisite knowledge

  • Experience designing, building, or running large systems (useful but not required)

What you'll learn

  • Discover how to architect a large system that's able to process millions of events per hour on a global scale, including common pitfalls of using microservices and real-time message processing and streaming
Photo of Sergio Freitas

Sergio Freitas

ThousandEyes

Sérgio Freitas is a lead software engineer at ThousandEyes focused on unearthing performance issues on the internet, the cloud, and the networks the world runs on. He’s a software engineer at heart and two-time CTO with extended experience developing and running large scale systems.

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)

Contact us

confreg@oreilly.com

For conference registration information and customer service

partners@oreilly.com

For more information on community discounts and trade opportunities with O’Reilly conferences

Become a sponsor

For information on exhibiting or sponsoring a conference

pr@oreilly.com

For media/analyst press inquires