[FIXED] Can't run Celery worker using Celery, Flask, and RabbitMQ: unable to load celery application

Issue

When I run celery -A app.celery worker --loglevel=INFO --pidfile='' I get back the following:

Usage: celery [OPTIONS] COMMAND [ARGS]...

Error: Invalid value for '-A' / '--app':
Unable to load celery application.
'nonetype' object has no attribute '_instantiate_plugins'

To the best of my understanding, in celery -A [name].celery... [name] should be the file where the Celery instance is created and held, which in my case is app.py. This is my first time working with Celery, so would love help here!

My file structure is as follows:

--app
   -- app.py
   -- celery_config
      -- __init__.py
      -- flask_celery.py

app.py

from flask import Flask
from celery_config.flask_celery import make_celery

# Create app
app = Flask(__name__)
app.config.from_envvar("APP_SETTINGS")

...

# Setup Celery
celery = make_celery(app)

flask_celery.py

from __future__ import absolute_import, unicode_literals
import os
from celery import Celery

def make_celery(app):
    celery = Celery(app.import_name)
    celery.conf.update(
        result_backend=app.config["CELERY_RESULT_BACKEND"],
        broker_url=app.config["CELERY_BROKER_URL"],
        timezone="UTC",
        task_serializer="json",
        accept_content=["json"],
        result_serializer="json"
    )
    TaskBase = celery.Task
    class ContextTask(TaskBase):
        abstract = True
        def __call__(self, *args, **kwargs):
            with app.app_context():
                return TaskBase.__call__(self, *args, **kwargs)
    celery.Task = ContextTask
    return celery

Solution

I figured it out.

Turns out since I was running the command from inside the app directory I needed to run celery -A celery worker --loglevel=INFO --pidfile='' rather than celery -A app.celery worker --loglevel=INFO --pidfile='' -- -A app searches for celery within the directory app, but I was already in that directory. I only realized this after finding this GitHub comment.

UPDATE: This was also not the answer, the issue was that I was expecting running the Celery worker to pick up my .env variables, which it doesn't do because it's not a Flask-specific package. I had to export my .env variables into my local environment because it was trying to instantiate the app's database without the DATABASE_URL variable. celery -A app.celery worker --loglevel=INFO --pidfile='' was the right command.

Answered By - Tati