Architecting a distributed system to monitor internet performance on a global scale

Description

ThousandEyes provides a platform for collecting, storing, visualizing, and alerting on network performance-related metrics against network targets globally to help its users measure and optimize the impact of dynamic network conditions on services they operate or leverage internally. The platform continuously collects internet data in real time from thousands of global vantage points, processing hundreds of millions of events per hour.

Sergio Freitas explains the progression of the company’s architecture that seamlessly adapted to increasing nonfunctional requirements like performance, volume, and increasingly complex functional requirements without compromising core tenets of simplicity and maintainability. This was possible due to the successful consolidation of various distributed architectural patterns into a unique macroarchitecture that can evolve with changing business needs and new technical challenges. You’ll discover the lessons learned and trade-offs made during this journey of architectural evolution from self-managed bare-metal servers to virtual machines and finally leveraging microservices in a container orchestration platform. Sergio focuses on microservice subarchitecture to scale horizontally without compromising on throughput, latency, and complexity; highly performant and low-latency event-driven architecture to maintain mission-critical service-level agreements (SLAs) at scale with non-trivial data enrichment pipelines; scalable and robust infrastructure to deploy data collection thousands of agents in more than 350 locations worldwide; and seamless, efficient, and reliable storage of structured and unstructured data using a combination of NoSQL and relational databases.