[FIXED] Calling .terminate() on asyncio subprocess raises ProcessLookupError

Issue

I have some non-async code that uses a subprocess...

import subprocess
import signal

p = subprocess.Popen(['/bin/true'], stdout=subprocess.PIPE)

# ... do something else here ...

# The process may or may not have finished yet.
# For the sake of this test, let us ensure a finish here
# by waiting for EOF on a pipe.
p.stdout.read()

p.terminate()

I tried migrating it to asyncio. However, the .terminate() call raises ProcessLookupError.

import asyncio
import asyncio.subprocess
import signal

async def main():
    p = await asyncio.create_subprocess_exec('/bin/true',
        stdout=asyncio.subprocess.PIPE)
    # ... do something else here ...
    # for the sake of this test, ensure a finish here
    await p.stdout.read()
    p.terminate()

asyncio.run(main())

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python3.8/asyncio/runners.py", line 43, in run
    return loop.run_until_complete(main)
  File "/usr/lib64/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "<stdin>", line 6, in main
  File "/usr/lib64/python3.8/asyncio/subprocess.py", line 141, in terminate
    self._transport.terminate()
  File "/usr/lib64/python3.8/asyncio/base_subprocess.py", line 149, in terminate
    self._check_proc()
  File "/usr/lib64/python3.8/asyncio/base_subprocess.py", line 142, in _check_proc
    raise ProcessLookupError()
ProcessLookupError

What is the error in this code? What am I doing wrong?

I tested on the following versions:

• python39-3.9.0-1.fc32.x86_64
• python3-3.8.5-5.fc32.x86_64

Solution

Solution: before you call .terminate(), use p.returncode to check whether the process has returned. The same also applies to calling .kill() or .send_signal().

if p.returncode is None:
    p.terminate()

This code is safe.[*] The process cannot be "reaped" between the check and the .terminate() call. The process can only be reaped while your async function is waiting (await statement).

[*] I lied, it's not safe. Looking at ThreadedChildWatcher, the Unix process could get reaped immediately. This looks like a very annoying race condition.

Discussion

In the non-async subprocess module, calling .wait() is what reaps the process and sets .returncode. If you have not called .wait(), .returncode will not be set. If a UNIX process exits but has not yet been reaped, it continues to exist as a "zombie".

In asyncio, the event loop reaps the process and sets .returncode. This could happen during any await statement in your function. The current documentation does not mention this. Reaping a Unix process means it no longer exists. There is nothing to send a signal to.

In theory, asyncio could be changed to allow the code in the question. However, there is a backwards-compatibility issue. By now, I suspect some programs rely on .returncode getting set without/before .wait(), despite it not being documented. In order to set .returncode, the Unix process must be reaped.

The most backwards-compatible change would probably be for asyncio to do the check itself. This would not help code that called os.kill() using p.pid. Such code is unlikely to be supported. (To start with, it could not be supported using portable Unix system calls, unless you removed or degraded FastChildWatcher).

Answered By - sourcejedi