[FIXED] Meraki API - Redirect stdout, stderr to websocket or file

Issue

I'm using a Flask route to call the Meraky python library and trying to redirect the stdout and stderr of each API call to a websocket or eventually a log file, here the code I'm using:

import meraki
from contextlib import redirect_stdout, redirect_stderr
import io
from flask import Flask
from flask_socketio import SocketIO, emit

async_mode = None

app = Flask(__name__)
app.config['SECRET_KEY'] = 'secret!'
socketio = SocketIO(app, async_mode=async_mode)

@app.route('/getOrganizations')
def getOrganizations():
    captured_output = io.StringIO()
    with redirect_stdout(captured_output), redirect_stderr(captured_output):
        try:
            API_KEY = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
            dashboard = meraki.DashboardAPI(API_KEY, output_log=False)

            organizations = dashboard.organizations.getOrganizations()
            socketio.emit('my_test',
                          {'data': captured_output.getvalue()})
            return {'organizations': organizations}

        except meraki.APIError as err:
            print('Error: ', err)
            return {'error': err}

If I restart the Flask server everything works fine the FIRST call and I get the desired output:

2022-05-05 17:02:32 meraki: INFO > Meraki dashboard API session initialized with these parameters: {'version': '1.15.0', 'api_key': '************************************9ea0', 'base_url': 'https://api.meraki.com/api/v1', 'single_request_timeout': 60, 'certificate_path': '', 'requests_proxy': '', 'wait_on_rate_limit': True, 'nginx_429_retry_wait_time': 60, 'action_batch_retry_wait_time': 60, 'retry_4xx_error': False, 'retry_4xx_error_wait_time': 60, 'maximum_retries': 2, 'simulate': False, 'be_geo_id': None, 'caller': None, 'use_iterator_for_get_pages': False} 2022-05-05 17:02:32 meraki: DEBUG > {'tags': ['organizations', 'configure'], 'operation': 'getOrganizations', 'method': 'GET', 'url': '/organizations', 'params': None} 2022-05-05 17:02:32 meraki: INFO > GET https://api.meraki.com/api/v1/organizations 2022-05-05 17:02:33 meraki: INFO > GET https://n392.meraki.com/api/v1/organizations 2022-05-05 17:02:34 meraki: INFO > organizations, getOrganizations - 200 OK

BUT in the subsequent calls nothing will be redirect to captured_output, it returns just nothing!

I've tried with different methods eg. sys.stdout, sys.stderr, with websocket or redirect to file, Flask, FastAPI...you name it! I was able to get the stdout/stderr only the first time after a server restart.

Has someone an idea?

Regards

Fabrizio

Solution

I was finally able to address the issue, the problem was how io.String() handles the buffer...but I ended up with a complete other solution. I was looking to catch the API execution logs and display it in the web Application console, the best solution is to redirect the stdout/stderr to a redis pub/sub server like this:

from rlog import RedisHandler

logging.basicConfig(
    level=logging.DEBUG,
    format='%(asctime)s %(name)12s: %(levelname)8s > %(message)s',
    datefmt='%Y-%m-%d %H:%M:%S',
)

logger = logging.getLogger()
logger.addHandler(RedisHandler(channel='live_log',host=redis_hostname, port=6379))

The RedisHandler forwards the stdout/stderr to redis, a websocket server then send all the output to its connected clients (the webapplication console). The result is a "live log" console that lets you follow the script execution without waiting for the API response. Final result can be seen here

Answered By - cyberdevnet