[FIXED] In Python async is it possible to provide an async generator with the context of if's caller?

Issue

I have a 'chain' of async tasks, i.e. task which gather/await other tasks with an arbitrary depth. Now I want an 'upper' task (i.e. one that awaits 'lower' tasks, which might then await yet lower tasks) to provide one of the 'lower' tasks with some contextual information, e.g. some logging object which might not yet exist when the lower task gets created.

async def compute(args):
    #
    # Here we don't know (yet) when or by whom this function get's awaited
    #
    logger = global_loggers[find_context(asyncio.current_task())]
    .. do stuff ..
    logger.write("something that goes somewhere")
    yield stuff
    logger.write("something that goes somewhere")
    yield stuff
    logger.write("something that goes somewhere")
    ..


async def some_weired_scheduler():
    #
    # this function doesn't make sense, it's just here to make the `await` queue deeper
    ..
    tasks_for_later.append(asyncio.gather(compute(<args1>), compute(<args2>)))
    ..

async def logging_executor():
    #
    # here we invoke the compute-generators and we want to 'inject' some runtime
    # environment, in this case it's just a logger
    #
    logger = MyLogger()
    global_loggers[find_context(asyncio.current_task())] = logger

    for t in tasks_for_later:
        await t

Is there some built in way to accomplish something like this? Is there something like a 'task stack' where I can see what task awaits what another yields or returns? Doesn't Quart do something like this with it's global request object?

Solution

"Is there something like a 'task stack' where I can see what task awaits what another yields or returns?"

Yes. It is called "contextvars" - https://docs.python.org/3/library/contextvars.html

It does exactly what you need - including, in a case where you need just one value to be known in a "subtask in the same call stack", you could create "loggers" as a contextvars.ContextVar in itself.

Note that several of the calls in asyncio allow you to pass a context - which will hold its own copy of all existing contextvars - explicitly. It is a bit awkward to use in that one can only "switch" a context by making a function call - either a call that will acept a context as parameter explicitly, or by using the context.run method.

(Unfortunately, your example code is not a stand-alone script, and I failed to understand how you would orchestrate the calls to some_weird_scheduler and logging_executor - so that I could make it a working script with contextvars)

Answered By - jsbueno