Issue
In my flask application there are some REST end points which takes too much time to respond. When invoked, it mostly carries out some CRUD operations in database, some of which could be made asynchronous. I do not have any issue if it sends the response to the client and database inserts keep going on in the background. I wanted to use asyncio, but heard that flask does not support asyncio. In that event I am left with just the choice of threading. Any suggestions? I do not have the option of dumping flask. I do not want to use Celery, as it would be too big of a change.
When using threading at some places it works, at others it does not.Looks like it is not finding the application context.
RuntimeError: Working outside of application context.
This typically means that you attempted to use functionality that needed
to interface with the current application object in some way. To solve
this, set up an application context with app.app_context(). See the
documentation for more information.
In the thread my 1st line of code is - and it is here where it fails:
user: AuthUser = g.logged_in_user
@Edit In the view function I have to do multiple things - some of them are chained, so cannot be made asynchronous - they need to happen in order as the result returned from one database call is used in invoking the next call & the final result is used in composing the json output the method returns. There is only one database call which is independent from others & being insertion of @ 1K records is the greatest contributor in slowing down the api response.
This method if I place at the end it says that the psycopg2 connection has already closed - though I never closed the connection explicitly - only may be the view function has returned the json payload.
If I place the heavy method at the beginning of the view function it works.
@Edit2 From the view function I pass application context and user.Instead of passing the database connection to the thread, I connect to the PostgreSql database from inside the thread.
threading.Thread(target=set_responder_contacts,
args=(user, template_id,),
kwargs={'app':current_app._get_current_object()})
.start()
Code snippet from the function the thread invokes:
def set_responder_contacts(user: AuthUser, template_id: int, app=None):
with app.app_context():
try:
db = dbConn()
#database insertion code
Solution
asyncio
wouldn't directly help here.
If you trust your server process to remain up for the duration of the background function, then sure, just spin up a thread and do the background work there:
def heavy_work(some_id):
pass
@app.post(...)
def view(...):
some_id = create_thing(...)
threading.Thread(target=heavy_work, args=(some_id,)).start()
return "Okay (though processing in the background)"
There are caveats:
- As I alluded to earlier, if the WSGI server process is killed for some reason (for instance, a memory or request count limit is exceeded, or it outright crashes), the background operation will be taken with it.
- If the heavy operation is heavily CPU-bound, it may affect the performance of other requests being served by the same server process.
Answered By - AKX
0 comments:
Post a Comment
Note: Only a member of this blog may post a comment.