Issue
Problem
I have a few tasks that run continuously, but one of them occasionally needs to restart so this one is run in the background. How can exceptions from this background task be raised immediately? In the following example, the exception is not raised until the next attempt to restart, which can be very infrequent in real applications, so can undesirably go unnoticed for a long time.
Example
https://replit.com/@PatrickPei/how-to-raise-exceptions-in-python-asyncio-background-task
This example runs 3 tasks:
foo
, which raises no exceptionsbar
, which raises an exception after 6 iterationson_interval
, which restartsbar
every 5 seconds
import asyncio
task = None
i = 0
async def foo():
while True:
print("foo")
await asyncio.sleep(1)
async def bar():
while True:
global i
i += 1
if i > 4:
raise ValueError()
print("bar", i)
await asyncio.sleep(1)
async def on_interval(n):
while True:
await asyncio.sleep(n)
# Cancel task.
global task
print("Canceling bar")
task.cancel()
try:
await task
except asyncio.CancelledError:
pass
# Restart task.
print("Restarting bar")
task = asyncio.create_task(bar())
async def main():
# Start background task.
print("Starting bar")
global task
task = asyncio.create_task(bar())
# Start other tasks.
await asyncio.gather(
foo(),
on_interval(3),
)
if __name__ == "__main__":
asyncio.run(main())
Output
bar
iterates 4 times and raises an exception, which is not caught until the next restart, as shown by 3 foo
iterations after bar 4
. This is a problem when there is a lot of time in between restarts since exceptions go unnoticed for a long time.
Starting bar
bar 1
foo
bar 2
foo
bar 3
foo
Canceling bar
Restarting bar
bar 4
foo
foo
foo
Canceling bar
Traceback (most recent call last):
File "~/example.py", line 60, in <module>
asyncio.run(main())
File "~/.pyenv/versions/3.10.5/lib/python3.10/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "~/.pyenv/versions/3.10.5/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
return future.result()
File "~/example.py", line 53, in main
await asyncio.gather(
File "~/example.py", line 37, in on_interval
await task
File "~/example.py", line 22, in bar
raise ValueError()
ValueError
Attempts
- Started another task to check
asyncio.Task.exception
, but this is cumbersome because every background task needs another busy loop to help raise its exceptions. - Tried
asyncio.Task.add_done_callback
but since the background task is still not awaited until the next restart, it only logs the error and does not stop the other taskfoo
.
Solution
In python 3.11, using async with
the asynchronous context manager and asyncio.TaskGroup()
solves this problem simply.
import asyncio
i = 0
async def foo():
while True:
print("foo")
await asyncio.sleep(1)
async def bar():
while True:
global i
i += 1
if i > 14:
raise ValueError()
print("bar", i)
await asyncio.sleep(1)
async def on_interval(n):
while True:
async with asyncio.TaskGroup() as tg1:
print("Restarting bar")
task2 = tg1.create_task(bar())
await asyncio.sleep(n)
print("Canceling bar")
task2.cancel()
async def main():
async with asyncio.TaskGroup() as tg:
task1 = tg.create_task(foo())
task3 = tg.create_task(on_interval(3))
asyncio.run(main())
-------------------------
foo
Restarting bar
bar 1
foo
bar 2
foo
bar 3
Canceling bar
foo
Restarting bar
bar 4
+ Exception Group Traceback (most recent call last):
| File "C:\Users\ф\PycharmProjects\tkinter\rrr.py", line 42, in <module>
| asyncio.run(main())
| File "C:\Users\ф\AppData\Local\Programs\Python\Python311\Lib\asyncio\runners.py", line 190, in run
| return runner.run(main)
| ^^^^^^^^^^^^^^^^
| File "C:\Users\ф\AppData\Local\Programs\Python\Python311\Lib\asyncio\runners.py", line 118, in run
| return self._loop.run_until_complete(task)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "C:\Users\ф\AppData\Local\Programs\Python\Python311\Lib\asyncio\base_events.py", line 650, in run_until_complete
| return future.result()
| ^^^^^^^^^^^^^^^
| File "C:\Users\ф\PycharmProjects\tkinter\rrr.py", line 37, in main
| async with asyncio.TaskGroup() as tg:
| File "C:\Users\ф\AppData\Local\Programs\Python\Python311\Lib\asyncio\taskgroups.py", line 135, in __aexit__
| raise me from None
| ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
+-+---------------- 1 ----------------
| Exception Group Traceback (most recent call last):
| File "C:\Users\ф\PycharmProjects\tkinter\rrr.py", line 25, in on_interval
| async with asyncio.TaskGroup() as tg1:
| File "C:\Users\ф\AppData\Local\Programs\Python\Python311\Lib\asyncio\taskgroups.py", line 135, in __aexit__
| raise me from None
| ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
+-+---------------- 1 ----------------
| Traceback (most recent call last):
| File "C:\Users\ф\PycharmProjects\tkinter\rrr.py", line 17, in bar
| raise ValueError()
| ValueError
+------------------------------------
foo
Process finished with exit code 1
Answered By - Сергей Кох
0 comments:
Post a Comment
Note: Only a member of this blog may post a comment.