[FIXED] flask application with background threads

Issue

I am creating a flask application, for one request I need to run some long running job which is not required to wait on the UI. I will create a thread and send a message to UI. The thread will calculate and update the database. But, UI will see a message upon submit. Below is my implementation, but it is running the thread and then sending the output to UI which is not I prefer. How can I run this thread in the background?

@app.route('/someJob')
def index():
    t1 = threading.Thread(target=long_running_job)
    t1.start()
    return 'Scheduled a job'

def long_running_job
    #some long running processing here

How can I make thread t1 to run the background and immediately send message in return?

Solution

The best thing to do for stuff like this is use a message broker. There is some excellent software in the python world meant for doing just this:

• Celery (http://www.celeryproject.org/), and
• RQ (http://python-rq.org/).

Both are excellent choices.

It's almost never a good idea to spawn a thread the way you're doing it, as this can cause issues processing incoming requests, among other things.

If you take a look at the celery or RQ getting started guides, they'll walk you through doing this the proper way!

Answered By - rdegges