September 19–20, 2016: Training
September 20–22, 2016: Tutorials & Conference
New York, NY

The modern Linux tracing landscape

Sasha Goldshtein (Sela Group)
2:25pm–3:05pm Thursday, 09/22/2016
Measuring the right things Cloud, DevOps Beekman Audience level: Intermediate
Average rating: ****.
(4.67, 6 ratings)

Prerequisite knowledge

  • Experience administering on or developing applications for Linux
  • What you'll learn

  • Understand why static and dynamic tracing are extremely powerful and useful tools for diagnosing issues in the field or in development
  • Description

    The Linux kernel has multiple “tracers” built in, with various degrees of support for aggregation, dynamic probes, parameter processing, filtering, histograms, and other features. Starting from the venerable ftrace, introduced in kernel 2.6, all the way through eBPF, which is still under development, there are many options to choose from when you need to statically instrument your software with probes, or diagnose issues in the field using the system’s dynamic probes. (Modern tools include SystemTap, SysDig, ktap, perf, bcc, and others.) Sasha Goldshtein reviews the modern tracing landscape—ftrace, perf_events, kprobes, uprobes, eBPF—uncovering what insight into system activity these tools can offer. Sasha then explores specific examples using tracing tools for diagnostics: tracing a memory leak using low-overhead kmalloc/kfree instrumentation, diagnosing a CPU caching issue using perf stat, probing network and block I/O latency distributions under load, and merely snooping user activities by capturing terminal input and output.

    Photo of Sasha Goldshtein

    Sasha Goldshtein

    Sela Group

    Sasha Goldshtein is the CTO of Sela Group, a Microsoft C# MVP and Azure MRS, a Pluralsight author, and an international consultant and trainer. Sasha’s consulting work revolves mainly around distributed architecture, production debugging, and mobile application development. Sasha is the author of Introducing Windows 7 for Developers (Microsoft Press) and Pro .NET Performance (Apress). He is also a prolific blogger and the author of numerous training courses, including .NET Debugging, .NET Performance, Android Application Development, and Modern C++.