[FIXED] Python Logging does not work over modules

Issue

In my main.py file I initialize the logger as follows:

# File: main.py
import logging
logger = logging.getLogger(__name__)

import submodule

#...

def main():
    logger.debug(f'Test')

if __name__ == '__main__':
    logger.setLevel(logging.DEBUG)
    formatter = logging.Formatter('[%(levelname)s | %(name)s] %(message)s')

    # The handler h1 logs only debug and info to stdout
    h1 = logging.StreamHandler(sys.stdout)
    h1.setLevel(logging.DEBUG)
    h1.addFilter(lambda record: record.levelno <= logging.INFO)
    h1.setFormatter(formatter)

    # The handler h2 logs only warning, error and exception to stderr
    h2 = logging.StreamHandler()
    h2.setLevel(logging.WARNING)
    h2.setFormatter(formatter)

    logger.addHandler(h1)
    logger.addHandler(h2)

    main()

When I use the logger in the main script, everything works correctly. But when I use it in a submodule like this:

# File: submodule.py
import logging
logger = logging.getLogger(__name__)

#...

logger.info('Test 1')
logger.error('Test 2')

I should see the following output (or something like this):

[DEBUG | __main__] Test
[INFO | submodule] Test 1
[ERROR | submodule] Test 2

But I get this output:

[DEBUG | __main__] Test
Test 2

To me it looks like the logger in submodules did not use the config of the logger in the main module. How to fix that?

Solution

This is because

logger = logging.getLogger(__name__)

will return different loggers in the main module and the submodule (because __name__ is a different string). You are now configuring only the logger for the main module.

But loggers are hierarchial, so the easiest way forward is to configure the root logger with the handlers:

root_logger = logging.getLogger('')
...
root_logger.addHandler(h1)

Every other logger is a child of the root logger so messages will "bubble up" by default to the root and use the handlers and levels configured there.

Answered By - vaizki