Python Guru Series ?????? - Part 5: Asynchronous Programming in Python

Hello everyone, in previous post, we discussed methods for implementing concurrency programming in Python. Concurrency programming not only helps us bypass the Global Interpreter Lock (GIL) but also improves application performance.

Continuing the Python Guru series today, let's explore asynchronous programming in Python.

1. A Brief Overview of Synchronous and Asynchronous Programming

In traditional synchronous programming, each task is executed sequentially. This means that one task must be completed before the next one can start.

Asynchronous programming in Python is a technique that allows you to write code capable of handling multiple tasks concurrently without blocking the main thread of the program. This is particularly useful in I/O-bound tasks such as API requests, database queries, file reading/writing, etc., where waiting for these tasks to complete can degrade application performance.

2. Asynchronous Programming in Python

2.1 Coroutine

Starting with Python 3.4+ and the introduction of Asyncio, Python's standard library for asynchronous programming, writing asynchronous applications in Python has become easier than ever.

A coroutine is a key concept when talking about asynchronous programming in Python. In a simple context, coroutines are functions that can pause and resume execution at various points. They allow the program to execute code asynchronously by pausing the execution of tasks at await points and resuming them once the work is complete.

Python uses the async def syntax to define an asynchronous function (a coroutine) and the await keyword to pause and wait for the result of tasks (python coroutines).

Await is used to wait for a coroutine to complete without blocking the main thread. When encountering the await keyword, the event loop pauses the current coroutine and switches to another task (the syntax and concept are quite similar to NodeJS).

2.2 Event Loop

The event loop is the brain behind coordinating the activities of all coroutines in a Python program.

Key Operations of the Event Loop:

• Scheduling: The event loop schedules coroutines and other asynchronous tasks for execution.
• Executing: The event loop executes coroutines until they reach await points, then pauses them and switches to other tasks.
• Handling I/O: The event loop can wait for I/O operations (such as file reading/writing or network connections) to complete without blocking the main thread.

Imagine the event loop as a master chef who orchestrates the entire kitchen, ensuring that all diners receive the dishes they ordered.

After receiving orders from the waitstaff, the chef organizes the kitchen staff, including sous chefs and assistants (workers), to prepare the dishes requested by the customers.

When a dish is ready, the chef signals the waitstaff to deliver the dish to the exact table where the customer is waiting, allowing them to enjoy their meal while waiting for other dishes to be served. This process continues until all tables have been served. Everyone is happy, with food to enjoy while waiting for more.

2.3 FastAPI

FastAPI leverages the application of coroutines and the event loop to create a framework that enables the deployment of high-performance Python web applications.

When you launch a FastAPI application, an event loop is created. All incoming requests are handled within this event loop.

Since all requests run on the same event loop, FastAPI can efficiently handle thousands of concurrent requests without creating multiple threads or processes. This maximizes the benefits of asynchronous programming, particularly in I/O-bound tasks like database connections, external API calls, or file processing.

3. Common Mistakes

When working with frameworks like FastAPI or with asyncio, there are common errors you need to avoid to ensure your code runs correctly and efficiently.

3.1 Not using await in an async function

Error: Declaring an async function but not using await to wait for the result of another coroutine.

Result: The function will not execute asynchronously and may cause errors or not behave as expected.

Solution: Ensure that you use await when calling a coroutine from an async function.

3.2 Blocking the event loop with synchronous tasks

Error: Executing blocking (synchronous) tasks in an async function, such as calling synchronous I/O functions or running CPU-intensive tasks.

Result: The event loop is blocked, reducing the performance and benefits of asynchronous programming.

Solution: Use asynchronous versions of functions or run blocking tasks in a separate thread or process.

3.3 Forgetting to Close the Event Loop

Error: Not closing the event loop after completing tasks.

Result: The event loop may not be properly released, leading to resource leaks.

Solution: Use asyncio.run() (which automatically closes the event loop) or ensure the loop is closed manually by calling loop.close().

4. Summary

Today, we've discussed how to write asynchronous code in Python, covering the use of Asyncio, Coroutines, Eventloop, and some common mistakes encountered in asynchronous programming.

Thank you for reading. Hope you gain something meaningful.

Once again, I'm Phan, a curious and dedicated developer.

Happy coding!