[FIXED] Clear Subprocess.popen's output (so far)

Issue

I want to run a binary with subprocess. The binary acts similarly (but is different from) bash's tail -f <file name> which prints certain number of lines from an existing continuous stream, while also printing any new outputs as they appear.

I want to run the command with subprocess.Popen(), clear stdout so far after a bit of delay, then start reading only the new that appear after the delay lines. Eg something like this:

process = subprocess.Popen('<my command>', shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)

sleep(1)
# TODO clear all STDOUT so far here

# Now start reading the output line by line
for line in iter(process.stdout.readline, b""):
    ...

How can I clear the stream up to a certain point? I tried seek or truncate but those dont seem to work on the stream produces by popen's stdout.

Thanks in advance!

Update: I know I can readlines for a short duration before actually storing the data. But I'm assuming there should be a cleaner function available that I'm failing to find.

Solution

You can do a non-blocking read on the stream to consume everything output so far. This is pretty easy on Unix-like systems that support os.set_blocking():

# switch to non-blocking mode and read everything up to this point
os.set_blocking(process.stdout.fileno(), False)
process.stdout.read()
# go back to blocking mode so we can stop at EOF
os.set_blocking(process.stdout.fileno(), True)

For other platforms, it requires a bit more work. See this question: A non-blocking read on a subprocess.PIPE in Python

Answered By - yut23