[FIXED] Flask, push application context for Flask-sqlalchemy to huey worker

Issue

I'm novice novice with the python ecosystem and in web development and I would like to build an application with the Flask framework.

This application will have to execute a background task. For this I chose to use the huey task queue.

The background task will have to perform some queries to a database. And for this I have chosen Flask-SQLAlchemy.

I managed to execute my task on the huey worker:

NFO:huey.consumer:MainThread:The following commands are available:
+ app.tasks.my_task
INFO:huey:Worker-1:Executing app.tasks.my_task: c5dd18bc-df2e-4380-9c1f-b597d2924ba2

But the following error occurs:

/huey/api.py", line 379, in _execute task_value = task.execute()
...
...
flask_sqlalchemy/__init__.py", line 1042, in get_app raise RuntimeError(
RuntimeError: No application found. Either work inside a view function or push an application context. See http://flask-sqlalchemy.pocoo.org/contexts/.

Here is my project structure:

app/
├── config.py
├── __init__.py
├── models.py
├── tasks.py
├── views.py
└─── foo.db

And here is my code:

#__init__.py
from flask import Flask
from app.config import db, Config, huey
from app.tasks import my_task

def create_app():
    app = Flask(__name__)
    app.config.from_object(Config)
    db.init_app(app)
    # register blueprints
    from app.views import main as main_blueprint
    app.register_blueprint(main_blueprint)
    return app

#config.py
from flask_sqlalchemy import SQLAlchemy
from huey import RedisHuey

huey = RedisHuey(__name__, host="localhost")
db = SQLAlchemy()

class Config:
    SQLALCHEMY_DATABASE_URI = "sqlite:///foo.db"
    SQLALCHEMY_TRACK_MODIFICATIONS = False

#tasks.py
from app.config import huey
from app.models import User

@huey.task()
#@huey.context_task(??)
def background_task():
    print("Database query:")
    User.query.get(1)  # Here is the problem
    return 1

#view.py
from flask import Blueprint
from app.tasks import my_task

main = Blueprint("main", __name__)

@main.route("/")
def index():
    background_task()  # running the registered background task
    return "hello view"

#models.py
from app.config import db

class User(db.Model):
    def __init__(self, username: str):
        self.username = username

    id = db.Column(db.Integer, primary_key=True)
    username = db.Column(db.String(64), unique=True, nullable=False)

I went through flask application context documentation:

https://flask.palletsprojects.com/en/2.1.x/appcontext/

And also huey documentation about shard resources:

https://huey.readthedocs.io/en/latest/shared_resources.html

I understand that I have to somehow provide the application context to the worker but I fail to connect the pieces together.

I also tried this

from flask import current_app

@huey.task()
def my_task():
    with current_app.app_context():
        print("Database query:")
        User.query.get(1)
        return 1

And it gave me this error:

/flask/globals.py", line 47, in _find_app raise RuntimeError(_app_ctx_err_msg)
RuntimeError: Working outside of application context.

Solution

It is better if you let Huey create a Flask app for its use. Organize your code as follows:

Add a second method that creates a Flask app for the specific use by Huey, similar to create_app

#__init__.py
from flask import Flask
from app.config import db, Config, huey
from app.tasks import my_task

def create_app():
    # ...
    return app

def create_huey_app():
    app = Flask('HUEY APP')
    app.config.from_object(Config)

    # only initialize stuff here that is needed by Huey, eg DB connection

    db.init_app(app)

    # register any blueprints needed
    # e.g. maybe it needs a blueprint to work with urls in email generation
    
    return app

Make all your tasks have a first parameter that is a callable:

#tasks.py
from app.config import huey
from app.models import User

# every task takes an app_factory parameter
@huey.task()
def background_task(app_factory):
    app = app_factory()
    with app.app_context():    
        User.query.get(1)
        return 1 

Then pass create_huey_app as the callable to each task instantiation:

#view.py
from flask import Blueprint
from app.tasks import my_task

main = Blueprint("main", __name__)

@main.route("/")
def index():
    # pass the Huey app factory to the task
    background_task(create_huey_app)  # running the registered background task
    return "hello view"

If you want to run the task locally when debugging:

@main.route("/")
def index():
    # pass the Huey app factory to the task
    if current_app.debug:
        background_task.call_local(create_huey_app)  # running the task directly
    else:
        background_task(create_huey_app)  # running the task in Huey
    return "hello view"    

Answered By - pjcunningham