[FIXED] When I enable uwsgi thread support and start the scheduler the api stops working

Issue

I'm new to python, flask, nginx and all that stuff.

I have a flask app that acts as API for a frontend. Also when the flask app is started I would like to start a scheduled task with APScheduler.

The problem is that when I enable uwsgi thread support and start the scheduler the api stops working (504 Gateway Time-out). But the scheduler works as seen in the logfile. When i remove the scheduler / thread support the api works but I obviously don't have the scheduler anymore. Somehow I suspect the scheduler prevents the flask app from being run properly?

As I am new to those technologies I'll post my setup below. If you need any more file infos please tell me.(The whole thing is run on a raspberry pi and the api is accessed from my pc over lan)

app.service

[Unit]
Description=uWSGI instance to serve app
After=network.target

[Service]
User=pi
Group=www-data
WorkingDirectory=/home/pi/flask
Environment="PATH=/home/pi/flask/appenv/bin"
ExecStart=/home/pi/flask/appenv/bin/uwsgi --ini app.ini --enable-threads

[Install]
WantedBy=multi-user.target

app.ini

[uwsgi]
module = wsgi:app

master = true
processes = 5

socket = /home/pi/flask/app.sock
chmod-socket = 660
vacuum = true

die-on-term = true

app.py

#!/usr/bin/env python3

from flask import Flask, request
from apscheduler.schedulers.background import BackgroundScheduler

import logging
logging.basicConfig(filename='logfile.log',level=logging.DEBUG)

from api.Controller import Controller
from Handler.Handler import Handler
from apscheduler.executors.pool import ThreadPoolExecutor, ProcessPoolExecutor

api_controller = Controller()
handler = Handler()

def startHandlerJob():
    handler.ExecuteAllSensors()

app = Flask( __name__ )


@app.route('/app')
def apiDefinition():
    return 'API Definition: GetHumidityValues, TODO'

@app.route( "/app/GetHumidityValues", methods=["GET"] )
def GetHumidityValues():
    logging.info("app.py: API-call GetHumidityValues")
    return api_controller.GetHumidityValues()


if (__name__ == "__main__"):
    app.run(host='0.0.0.0')

executors = {
    'default': ThreadPoolExecutor(20),
    'processpool': ProcessPoolExecutor(5)
}
job_defaults = {
    'coalesce': False,
    'max_instances': 1
}

scheduler = BackgroundScheduler(daemon=True, executors=executors, job_defaults=job_defaults)
scheduler.start()
scheduler.add_job(startHandlerJob,'cron', minute='*')

logfile.log

WARNING:apscheduler.scheduler:Execution of job "startHandlerJob (trigger: cron[minute=''], next run at: 2019-12-18 19:01:00 CET)" skipped: maximum number of running instances reached (1) DEBUG:apscheduler.scheduler:Next wakeup is due at 2019-12-18 19:02:00+01:00 (in 59.980780 seconds) DEBUG:apscheduler.scheduler:Looking for jobs to run WARNING:apscheduler.scheduler:Execution of job "startHandlerJob (trigger: cron[minute=''], next run at: 2019-12-18 19:02:00 CET)" skipped: maximum number of running instances reached (1) DEBUG:apscheduler.scheduler:Next wakeup is due at 2019-12-18 19:03:00+01:00 (in 59.979407 seconds)

systemctl status app

* app.service - uWSGI instance to serve app
   Loaded: loaded (/etc/systemd/system/app.service; enabled; vendor preset: enabled)
   Active: active (running) since Wed 2019-12-18 18:40:57 CET; 23min ago
 Main PID: 21129 (uwsgi)
    Tasks: 8 (limit: 2200)
   Memory: 22.9M
   CGroup: /system.slice/app.service
           |-21129 /home/pi/flask/appenv/bin/uwsgi --ini app.ini --enable-threads
           |-21148 /home/pi/flask/appenv/bin/uwsgi --ini app.ini --enable-threads
           |-21149 /home/pi/flask/appenv/bin/uwsgi --ini app.ini --enable-threads
           |-21150 /home/pi/flask/appenv/bin/uwsgi --ini app.ini --enable-threads
           |-21151 /home/pi/flask/appenv/bin/uwsgi --ini app.ini --enable-threads
           `-21152 /home/pi/flask/appenv/bin/uwsgi --ini app.ini --enable-threads

Dec 18 18:40:57 raspberrypi uwsgi[21129]: mapped 386400 bytes (377 KB) for 5 cores
Dec 18 18:40:57 raspberrypi uwsgi[21129]: *** Operational MODE: preforking ***
Dec 18 18:40:59 raspberrypi uwsgi[21129]: WSGI app 0 (mountpoint='') ready in 2 seconds on interpreter 0xa6f900 pid: 21129 (default app)
Dec 18 18:40:59 raspberrypi uwsgi[21129]: *** uWSGI is running in multiple interpreter mode ***
Dec 18 18:40:59 raspberrypi uwsgi[21129]: spawned uWSGI master process (pid: 21129)
Dec 18 18:40:59 raspberrypi uwsgi[21129]: spawned uWSGI worker 1 (pid: 21148, cores: 1)
Dec 18 18:40:59 raspberrypi uwsgi[21129]: spawned uWSGI worker 2 (pid: 21149, cores: 1)
Dec 18 18:40:59 raspberrypi uwsgi[21129]: spawned uWSGI worker 3 (pid: 21150, cores: 1)
Dec 18 18:40:59 raspberrypi uwsgi[21129]: spawned uWSGI worker 4 (pid: 21151, cores: 1)
Dec 18 18:40:59 raspberrypi uwsgi[21129]: spawned uWSGI worker 5 (pid: 21152, cores: 1)

The logfile shows that the scheduler is active. But when I try to use http:/raspberryipaddress/app the answer is a 504 Gateway Time-out response. When I remove the scheduler and disable thread support this call works as desired.

Any help is appreciated. Probably I am missing something obvious, because I am new to all those things. Thanks!

Solution

Okay thanks for the answers. I did not get it to work with your suggested solutions. Therefore I simply decided to run the scheduler as separate systemd service. That way the scheduling does not stop the flask api from working.

Answered By - LuPaw