Issue
The "traditional" way for a library to take file input is to do something like this:
def foo(file_obj):
data = file_obj.read()
# Do other things here
The client code is responsible for opening the file, seeking to the appropriate point (if necessary), and closing it. If the client wants to hand us a pipe or socket (or a StringIO
, for that matter), they can do that and it Just Works.
But this isn't compatible with asyncio, which requires a syntax more like this:
def foo(file_obj):
data = yield from file_obj.read()
# Do other things here
Naturally, this syntax only works with asyncio objects; trying to use it with traditional file objects makes a mess. The reverse is also true.
Worse, it seems to me there's no way to wrap this yield from
inside a traditional .read()
method, because we need to yield all the way up to the event loop, not just at the site where the reading happens. The gevent library does do something like this, but I don't see how to adapt their greenlet code into generators.
If I'm writing a library that handles file input, how should I deal with this situation? Do I need two versions of the foo()
function? I have many such functions; duplicating all of them is not scalable.
I could tell my client developers to use run_in_executor()
or some equivalent, but that feels like working against asyncio instead of with it.
Solution
This is one of the downsides of explicit asynchronous frameworks. Unlike gevent
, which can monkeypatch synchronous code to make it asynchronous without any code changes, you can't make synchronous code asyncio
-compatible without rewriting it to use asyncio.coroutine
and yield from
(or at least asyncio.Futures
and callbacks) all the way down.
There's no way that I know of to have the same function work properly in both an asyncio
and normal, synchronous context; any code that's asyncio
compatible is going to rely on the event loop to be running to drive the asynchronous portions, so it won't work in a normal context, and synchronous code is always going to end up blocking the event loop if its run in an asyncio
context. This is why you generally see asyncio
-specific (or at least asynchronous framework-specific) versions of libraries alongside synchronous versions. There's just no good way to present a unified API that works with both.
Answered By - dano
0 comments:
Post a Comment
Note: Only a member of this blog may post a comment.