[FIXED] In what condition could ensure_future not actually start the future?

Issue

I'm trying to asynchronously download a file in Python, using wget in a subprocess. My code looks like this:

async def download(url, filename):
    wget = await asyncio.create_subprocess_exec(
        'wget', url,
        'O', filename
    )
    await wget.wait()


def main(url):
    loop = asyncio.get_event_loop()
    future = asyncio.ensure_future(download(url, 'test.zip'), loop=loop)
    print("Downloading..")
    time.sleep(15)
    print("Still downloading...")
    loop.run_until_complete(future)
    loop.close()

What I'm trying to do is witness the printing of "Downloading.." then 15 seconds later "Still downloading...", all while the download of the file has started. What I'm actually seeing is that the download of the file only starts when the code hits loop.run_until_complete(future)

My understanding is that asyncio.ensure_future should start running the code of the download coroutine, but apparently I'm missing something.

Solution

When passed a coroutine, asyncio.ensure_future converts it to a task - a special kind of future that knows how to drive the coroutine - and enqueues it in the event loop. "Enqueue" means that the code inside the coroutine will be executed by a running event loop that schedules the coroutines. If the event loop is not running, then none of the coroutines will get a chance to run either. The loop is told to run by a call to loop.run_forever() or loop.run_until_complete(some_future). In the question the event loop is only started after the call to time.sleep(), so the beginning of the download is delayed by 15 seconds.

time.sleep should never be called in a thread that runs the asyncio event loop. The correct way to sleep is with asyncio.sleep, which yields the control to the event loop while waiting. asyncio.sleep returns a future that can be submitted to the event loop or awaited from a coroutine:

# ... definition of download omitted ...

async def report():
    print("Downloading..")
    await asyncio.sleep(15)
    print("Still downloading...")

def main(url):
    loop = asyncio.get_event_loop()
    dltask = loop.create_task(download(url, 'test.zip'))
    loop.create_task(report())
    loop.run_until_complete(dltask)
    loop.close()

The above code has a different problem. When the download is shorter than 15 seconds, it results in a Task was destroyed but it is pending! warning being printed. The problem is that the report task was never canceled when the download task finished and the loop closed, it was just abandoned. This happening often indicates a bug or a misunderstanding of how asyncio works, so asyncio flags it with a warning.

The obvious way to eliminate the warning is by explicitly canceling the task of the report coroutine, but the resulting code ends up being verbose and not very elegant. An simpler and shorter fix is to change report to await the download task, specifying a timeout for displaying the "Still downloading..." message:

async def dl_and_report(dltask):
    print("Downloading..")
    try:
        await asyncio.wait_for(asyncio.shield(dltask), 15)
    except asyncio.TimeoutError:
        print("Still downloading...")
        # assuming we want the download to continue; otherwise
        # remove the shield(), and dltask will be canceled
        await dltask

def main(url):
    loop = asyncio.get_event_loop()
    dltask = loop.create_task(download(url, 'test.zip'))
    loop.run_until_complete(dl_and_report(dltask))
    loop.close()

Answered By - user4815162342