Let’s first look at the spinner example from the book Fluent Python, chapter 18. The below Python program runs a loading spinner while it simulates a time-consuming job.
import asyncio import itertools import sys @asyncio.coroutine def spin(message): write, flush = sys.stdout.write, sys.stdout.flush for char in itertools.cycle('|/-\\'): status = char + ' ' + message leng = len(status) write(status) flush() write('\x08' * leng) try: yield from asyncio.sleep(.1) except asyncio.CancelledError: break write(' ' * leng + '\x08' * leng) @asyncio.coroutine def heavy_work(): # pretend waiting a long time for I/O yield from asyncio.sleep(5) return 42 @asyncio.coroutine def supervisor(): spinner = asyncio.async(spin('thinking ...')) print('spinner object: ', spinner) result = yield from heavy_work() spinner.cancel() return result def main(): loop = asyncio.get_event_loop() result = loop.run_until_complete(supervisor()) loop.close() print('answer: ', result) if __name__ == '__main__': main()
asyncio constructs concurrency with coroutines driven by an event loop.
Please note that the latest version of Python when the book has been written was 3.4, when
await keywords were not there yet. Here goes 3.5 version of the spinner example.
async def spin(message): write, flush = sys.stdout.write, sys.stdout.flush for char in itertools.cycle('|/-\\'): status = char + ' ' + message leng = len(status) write(status) flush() write('\x08' * leng) try: await asyncio.sleep(.1) except asyncio.CancelledError: break write(' ' * leng + '\x08' * leng) async def heavy_work(): # pretend waiting a long time for I/O await asyncio.sleep(5) return 42 async def supervisor(): spinner = asyncio.ensure_future(spin('thinking ...')) print('spinner object: ', spinner) resut = await heavy_work() spinner.cancel() return result if __name__ == '__main__': main() # main is the same
asyncio arrived, there was not a strict distinction between a generator and a coroutine. Sometimes one could tell “this is a coroutine” because it contains the
= yield syntax (implying coroutines are for data pipe-lining), but it does not apply always.
A traditional coroutine focuses to deal with data as its consumer sends data (or exceptions) to it. It naturally controls its states via
= yield syntax, e.g., it runs until it hits
yield and waits for a consumer to
send data. In other words, it acts in a concurrent manner.
With the advent of Python
asyncio as standard library, PEP 3156 made the distinction of
@asyncio.coroutine-decorated coroutine for the suitable use of coroutines with
asyncio. And this coroutine is called generator-based coroutine after the invention of native coroutines.
As coroutines have done crucial roles in asynchronous works for Python, Python 3.5 introduced native coroutines with brand-new
await keywords, to make the significant difference of coroutines, especially with
So, as the new keywords imply, native coroutines seem to take more weights on this waiting behavior for asynchronous handling in a single thread.
According to PEP 492,
The following syntax is used to declare a native coroutine:
async def read_data(db): pass
Key properties of coroutines:
async deffunctions are always coroutines, even if they do not contain await expressions.
- It is a
yield fromexpressions in an
- Internally, two new code object flags were introduced:
CO_COROUTINEis used to mark native coroutines (defined with new syntax).
CO_ITERABLE_COROUTINEis used to make generator-based coroutines compatible with native coroutines (set by `types.coroutine() function).
- Regular generators, when called, return a generator object; similarly, coroutines return a coroutine object.
StopIterationexceptions are not propagated out of coroutines, and are replaced with
RuntimeError. For regular generators such behavior requires a future import (see PEP 479).
- When a native coroutine is garbage collected, a
RuntimeWarningis raised if it was never awaited on.
The behavior of existing generator-based coroutines in
Differences of native coroutines from generators:
- Native coroutine objects do not implement
__next__methods. Therefore, they cannot be iterated over or passed to
tuple()and other built-ins. They also cannot be used in a
for..inloop. An attempt to use
__next__on a native coroutine object will result in a
- Plain generators cannot
yield fromnative coroutines: doing so will result in a
- Generator-based coroutines (for
asynciocode must be decorated with
yield fromnative coroutine objects.
Falsefor native coroutine objects and native coroutine functions.
The spinner example shows the basics: How
asyncio starts an event loop, drives coroutines (or tasks) asynchronously, and then gets the results at the proper moment.
The event loop is the central execution device provided by asyncio. It provides multiple facilities, including:
- Registering, executing and cancelling delayed calls (timeouts).
- Creating client and server transports for various kinds of communication.
- Launching subprocesses and the associated transports for communication with an external program.
- Delegating costly function calls to a pool of threads.
asyncio has more and much more than the spinner. Check the references including great slides and essays.