Leveraging AsyncIO and Decorators to Boost Python Performance

Python is a versatile language offering many powerful constructs to its users. Two of these constructs are asyncio, a library to write single-threaded concurrent code using coroutines, multiplexing I/O access over sockets and other resources, running network clients and servers, and other related primitives, and decorators, which are a powerful tool allowing us to "wrap" a function or a class method with additional functionality. In this article, we will explore these two concepts in depth, and show how they can be used to significantly boost the performance of your Python code.

The Basics of AsyncIO

AsyncIO, introduced in Python 3.4, is a library that allows writing single-threaded concurrent code using coroutines, multiplexing I/O access over sockets and other resources, running network clients and servers, and other related primitives.

The most important aspect of AsyncIO is its ability to handle asynchronous I/O operations. Normally, when your program needs to communicate with an external resource such as a web server, it has to wait for the response before it can continue processing. This is known as synchronous or blocking I/O.

On the other hand, asynchronous I/O, as provided by AsyncIO, allows the program to continue processing other tasks while waiting for the response. This way, the overall execution time can be significantly reduced, especially for I/O-bound tasks.

The Power of Decorators

Python decorators are a powerful tool that allows us to "wrap" a function or a class method with additional functionality. This can be used to modify the behavior of the function or method, or to add some preprocessing and postprocessing around the original call.

Decorators are typically used for tasks like logging, enforcing access control and authentication, instrumentation and timing functions, rate-limiting, and caching; and many more use-cases.

The decorator pattern works by taking a function as an argument and returning a new function that includes the original behavior, but augmented with the additional behavior. This new function can be used in the place of the original function.

Unleashing the Power of AsyncIO with Decorators

The combination of AsyncIO and decorators can provide some significant performance improvements. We can create a decorator that transforms a synchronous function or method into an asynchronous one, and then use it with AsyncIO to perform concurrent I/O operations.

Here's an example of a decorator that turns a synchronous method into an asynchronous one:

def _asyncify(func)
? ? @functools.wraps(func)
? ? def wrapper(self, *args, **kwargs):
? ? ? ? loop = asyncio.get_event_loop()
? ? ? ? return loop.run_in_executor(self.executor, func, self, *args, **kwargs)
? ? return wrapperdef _asyncify(func)        

The decorator is then used in a factory function asyncify that creates a new class with all methods converted to asynchronous ones:

def asyncify(max_workers:int):
    """Decorator that converts all the methods of a class into async methods.
    """
    def decorator(cls: typing.Type):
        attrs:typing.Dict[str, typing.Any] = {}
        class_executor = ThreadPoolExecutor(max_workers=max_workers)
        attrs['executor'] = class_executor
        for attr_name, attr_value in cls.__dict__.items():
            if isinstance(attr_value, types.FunctionType) and attr_name.startswith('__') is False:
                attrs[attr_name] = _asyncify(attr_value)
            else:
                attrs[attr_name] = attr_value
        return type(cls.__name__, cls.__bases__, attrs)
    return decorator        

With this decorator, you can easily convert a class like the requests.Session class into an asynchronous class. Here's an example of how you can use it:

@asyncify(max_workers=5)
class AsyncSession:
    """A class that wraps the requests.Session class and converts all the methods into async methods.
    """
    session: Session
    def __init__(self):
        self.session = Session()
        
    def fetch(self, url:str, method:str='GET', **kwargs) -> typing.Union[str, dict, bytes]:
        """Fetches the content of the url and returns it.
        """
        response = self.session.request(method, url, **kwargs)
        headers  = response.headers
        content_type = headers.get('content-type')
        assert isinstance(content_type, str)
        if 'json' in content_type:
            return response.json()
        if content_type.startswith('text'):
            return response.text
        return response.content)        

Performance Comparison

To demonstrate the potential benefits of using AsyncIO and decorators in Python, let's run a comparison between an asynchronous HTTP client and a synchronous one. For this comparison, we will use the AsyncClient class defined above, and a simple requests.Session client:

sync_client = Session()
client = AsyncSession()

async def test():
    return await client.fetch('https://www.google.com')
    
def sync_test():
    return sync_client.get('https://www.google.com').text

async def main()
? ? start = time.time()
? ? await asyncio.gather(*[test() for _ in range(100)])
? ? print(f'Async: {time.time() - start}')
? ? start = time.time()
? ? for _ in range(100):
? ? ? ? sync_test()
? ? print(f'Sync: {time.time() - start}')? ??
? ??
if __name__ == '__main__':
? ? asyncio.run(main())        

When run, the output shows a significant speedup for the asynchronous client:

Async: 2.10167288780212
Sync: 7.9019665718078614        

In this example, the asynchronous client is almost 4 times faster than the synchronous one. This shows the power of AsyncIO and decorators when used together.

Conclusion

AsyncIO and decorators are powerful tools in Python. Used separately, they can improve code organization and performance. But used together, as we've shown in this article, they can provide significant performance boosts and make your code more efficient. Whether you're developing a web service, a data processing pipeline, or any other kind of I/O-bound application, these tools can help you write faster, more efficient Python code.