[FIXED] How can I ensure either one specific task, or the rest, can run at a time with asyncio?

Issue

I have a system that processes events. Every now and then a configuration change may happen that means events must stop being processed while the system is reconfigured. For example (simplified and abstracted):

class System:
    # this will be running multiple times concurrently
    async def process_events(self, event):
        # this line must not be run while `reconfigure_event_processing` is running
        handler = await lookup_event_handler(self.handler_lookup_config, event)

        await handler.handle_event(event)

    async def reconfigure_event_processing(self, config):
        # this code must wait until nothing is using `lookup_event_processing_configuration`
        self.handler_lookup_config = config

The documentation of asyncio synchronisation primitives is quite confusing to me, so I want to know which of them, if any, solves this problem.

Solution

Coming back to this a long time later, I've realised that what I really needed was what is called a read-write lock. For this you should just use a tried and tested library, such as aiorwlock.

From Wikipedia:

An RW lock allows concurrent access for read-only operations, while write operations require exclusive access. This means that multiple threads can read the data in parallel but an exclusive lock is needed for writing or modifying data. When a writer is writing the data, all other writers or readers will be blocked until the writer is finished writing

Example usage is as follows:

from aiorwlock import RWLock

class System:
    def __init__(self):
        self.lock = RWLock()

    async def process_events(self, event):
        async with self.lock.reader_lock:
            handler = await lookup_event_handler(self.handler_lookup_config, event)

        await handler.handle_event(event)

    async def reconfigure_event_processing(self, config):
        async with self.lock.writer_lock:
            self.handler_lookup_config = config

---

My original solution can be seen in the revision history of this answer, but I'd strongly recommend simply using an existing library instead.

Answered By - pxeger