[FIXED] What is asyncio.PriorityQueue's default priority?

Issue

Does asyncio.PriorityQueue have a default priority? If I do not give a priority value with an entry, will it default to FIFO? Apologies if this is a duplicate, I have been searching for this answer everywhere and haven't found it yet.

I have a discord music bot that currently uses asyncio.Queue to hold song info, and am debating switching to a priority queue to add a "play next" option. I have struggled to find detailed descriptions of asyncio's priority queue implementation.

Solution

While the regular asyncio.Queue uses a deque to store the items under the hood, the asyncio.PriorityQueue uses a list with the heap queue implementation from the standard library. (source)

The heapq documentation should give you all the info you need to know about how this works, but in short there is no "default" priority configured anywhere. The heap methods will simply use the rich comparison operators (<=, >= etc.) on the items to maintain invariance.

That is why the PriorityQueue docs suggest using 2-tuple items with the first element ensuring the ordering.

Tuples support rich comparison by proxy of their elements. And that comparison is lazy. That means, when comparing two tuples tup1 and tup2, the first check compares tup1[0] and tup2[0]. And if that check returns a clear order, the other elements are not even checked.

Try this:

from asyncio import PriorityQueue

class Foo:
    pass

q = PriorityQueue()
q.put_nowait(Foo())
q.put_nowait(Foo())  # error
item = q.get_nowait()
print(item)

You will get an error as soon as you try to add a second Foo instance because there is no way to order objects of the class Foo.

But do this instead:

...
q = PriorityQueue()
q.put_nowait((2, Foo()))
q.put_nowait((1, Foo()))
item = q.get_nowait()
print(item)  # (1, <__main__.Foo object at 0x...>)

This will work and get you the tuple with the 1 as the first element because 1 < 2.

If you make the first elements ambiguous, you will again get an error:

...
q = PriorityQueue()
q.put_nowait((1, Foo()))
q.put_nowait((1, Foo()))  # error again

But as soon as you add a __lt__ method to Foo, the priority queue will accept multiple instances just fine (because the heapq functions will) and set up their priority corresponding to their ordering with the < operator:

from asyncio import PriorityQueue


class Foo:
    def __init__(self, value: int) -> None:
        self.value = value

    def __str__(self) -> str:  # just for easier demonstration
        return f"Foo({self.value})"

    def __lt__(self, other: object) -> bool:
        if not isinstance(other, Foo):
            raise NotImplementedError
        return self.value < other.value


q = PriorityQueue()
q.put_nowait(Foo(2))
q.put_nowait(Foo(1))
item = q.get_nowait()
print(item)  # Foo(1)

So in short, you simply need to ensure that whatever items you put into the PriorityQueue can be ordered and that their order reflects the priority you want to establish.

Answered By - Daniil Fajnberg