July 20–24, 2015
Portland, OR

Plate spinning: Modern concurrency in Python

Luciano Ramalho (ThoughtWorks)
11:30am–12:10pm Thursday, 07/23/2015
Scale D137/138
Average rating: ****.
(4.00, 8 ratings)
Slides:   1-PDF 

Prerequisite Knowledge

Understanding Python generators is a prerequisite for following this talk. Previous experience with event-oriented programming in any form will be useful but is not a requirement.

Description

Traditionally the concurrency story in Python was not great: although threads can be trusted to improve the throughput in I/O bound programs, in the standard CPython interpreter the GIL (Global Interpreter Lock) prevents any gains with CPU-bound tasks. This started to change with the multiprocessing module introduced in Python 2.6 and 3.0, which implements nearly the same API as the threading module but using independent processes, thus sidestepping the GIL.

The concurrent.futures package added in Python 3.2 introduced the concept of a Future to represent a pending task, and the very high level Executor API which lets you easily delegate tasks to threads or processes for asynchronous (i.e. non-blocking) execution.

Then Python 3.3 made it possible to transparently delegate processing from one coroutine to another using the new yield from construct. This enables asynchronous programming without callbacks or threads in user-level code, avoiding callback hell and making multi-step asynchronous algorithms look like simple sequential code. On top of yield from and the Future idea, Guido van Rossum built the asyncio library, implementing a pluggable event loop and an asynchronous networking library.

This talk will give a high-level overview of these tools, showing simple yet practical examples of their use.

Photo of Luciano Ramalho

Luciano Ramalho

ThoughtWorks

Luciano Ramalho is the author of Fluent Python. Ramalho was a Web developer before the Netscape IPO in 1995, and switched from Perl to Java to Python in 1998. Since then he worked on some of the largest news portals in Brazil using Python, and taught Python web development in the Brazilian media, banking and government sectors. He has spoken multiple times at OSCON, PyCon, PythonBrasil, FISL and RuPy. Ramalho is a member of the Python Software Foundation and co-founder of Garoa Hacker Clube, the first hackerspace in Brazil. He is a managing partner at Python.pro.br, a training company.