[FIXED] Python/QT Live Application Log in QML

Issue

I am trying to create a live log of all console output in the QML GUI. I want to emit a QT signal every time Python writes to stdout/stderr. I am using Python 3.8. Here is my code that performs logging of QML output and Python interpreter stderr by writing to a file and the console.

class Log(QObject, object):

    def __init__(self):
        logging.basicConfig(
            level=logging.DEBUG,
            format="%(asctime)s [%(levelname)s] %(message)s",
            handlers=[
                logging.FileHandler("debug.log", mode='w'),
                logging.StreamHandler()
            ]
        )
        qInstallMessageHandler(self.qt_message_handler)

    def qt_message_handler(self, mode, context, message):
        log_entry = "%s: %s (%s:%d, %s)" % (mode.name.decode(), message, context.file, context.line, context.file)
        logging.warning(log_entry)

Solution

Depends on what you are trying to catch:

Regular log messages? Use an own logging handler:

class MyLogHandler(logging.Handler):
    def emit(self, record):
        """Do whatever you need with the record"""
# somewhere appropriate
logging.getLogger().addHandler(MyLogHandler())

Output of print statement? You can catch and convert into log messages. I use somethin like this in my programs to catch leftover print() calls:

real_print = print
def _print(*stuff, sep=' ', end='\n', file=sys.stdout, flush):
    if not (file is sys.stdout or file is None):
        return real_print(*stuff, sep=sep, end=end, file=file, flush=flush)
    text = sep.join(str(thing) for thing in stuff)
    # replace by other logging call as desired
    # e.g. add file / lineno information
    logging.info(text)
__builtins__['print'] = _print

Unregulated output to stdout / stderr (C libraries): This is a bit more complicated. If possible, you can isolate the part of the code that outputs things into a subprocess. You can then take its output from the Subprocess object properties.

Otherwise, you can fiddle around with sys.stdout / sys.stderr, but I suspect this won't help with C libs (never tried myself).

In any case, be extra careful with error handling when doing log stuff. Vanishing log messages are a ticket for hours of debugging fun.

EDIT: If you are trying to create an IDE-like application, you should probably start another Python instance in a subprocess. Then you can catch its stdout/stderr easily; also "sandbox" and "gui" environments should be clearly separated.

Proof-of-concept:

import sys
import subprocess
# runs python --version
sp = subprocess.run([sys.executable, "--version"])
print(sp.stdout)

For "real" work you would of course omit the --version arg, and send input to sp.stdin somehow.

Answered By - Torben Klein