[FIXED] How to use .yaml file for logging specific level only

Issue

I am dealing with the same problem as python logging specific level only & logging with filters the only exception being i am using an .yaml file.

I checked the documentation, it says:

The fileConfig() API is older than the dictConfig() API and does not provide functionality to cover certain aspects of logging. For example, you cannot configure Filter objects, which provide for filtering of messages beyond simple integer levels, using fileConfig(). If you need to have instances of Filter in your logging configuration, you will need to use dictConfig(). Note that future enhancements to configuration functionality will be added to dictConfig(), so it’s worth considering transitioning to this newer API when it’s convenient to do so.

I have checked install filter on logging level in python using dictConfig and Where is a complete example of logging.config.dictConfig?

My latest python file looks like:

import logging
import logging.config
import yaml

with open('logging.yaml', 'rt') as f:
    config = yaml.safe_load(f.read())
logging.config.dictConfig(config)

logger = logging.getLogger('module1')

# 'application' code
logger.debug('debug message')
logger.info('info message')
logger.warning('warn message')
logger.error('error message')
logger.critical('critical message')

The logging.yaml file looks like:

version: 1
formatters:
   simple:
     format: '{asctime} : {name} : {levelname} : {message}'
     style: '{'
handlers:
 console:
    class: logging.StreamHandler
    level: DEBUG
    formatter: simple
    stream: ext://sys.stdout
loggers:
   module1:
     level: DEBUG
     handlers: [console]
     propogate: no
root:
    level: DEBUG
    handlers: [console]

Not understanding how to modify logging.yaml file so that I can view only INFO in console and save CRITICAL, ERROR, WARNING to logfile.log.

Solution

OP Here.

I was hoping that I would be able to configure the filter by just modifying logging.yaml file. Unfortunately I had to create classes for filtering (in my case infoFilter and criticalFilter).

The following code log INFO in terminal and WARNING, ERROR or CRITICAL in file.

The Python Code:

import logging
import os
import logging.config
import yaml

DEFAULT_LEVEL = logging.DEBUG

class infoFilter(logging.Filter):
    def filter(self, log_record):
        return log_record.levelno == logging.INFO

class warningerrorcriticalFilter(logging.Filter):
    def filter(self, log_record):

        if log_record.levelno in {logging.WARNING, logging.ERROR, logging.CRITICAL}:
            return True
        else:
            return False

def log_setup(log_cfg_path='config.yaml'):
    if os.path.exists(log_cfg_path):
        with open(log_cfg_path, 'r') as f:
            try:
                config = yaml.safe_load(f.read())
                logging.config.dictConfig(config)
            except Exception as e:
                print('Error with file, using Default logging')
                logging.basicConfig(level=DEFAULT_LEVEL)

    else:
        logging.basicConfig(level=DEFAULT_LEVEL)
        print('Config file not found, using Default logging')

log_setup()

logger = logging.getLogger('dev')
logger.debug('debug message')
logger.info('info message')
logger.warning('warn message')
logger.error("error message")
logger.critical('critical message')

The config.yaml file:

version: 1

formatters:
  extended:
    format: '%(asctime)-20s :: %(levelname)-8s :: [%(process)d]%(processName)s :: %(threadName)s[%(thread)d] :: %(pathname)s :: %(lineno)d :: %(message)s'
  simple:
    format: "%(asctime)s :: %(name)s :: %(message)s"

filters:
  show_only_info:
    (): __main__.infoFilter
  show_only_warningerrorcritical:
    (): __main__.warningerrorcriticalFilter

handlers:
  console:
    class: logging.StreamHandler
    level: DEBUG
    formatter: simple
    stream: ext://sys.stdout
    filters: [show_only_info]

  file_handler:
    class: logging.FileHandler
    filename: my_app.log
    formatter: extended
    level: DEBUG
    filters: [show_only_warningerrorcritical]

loggers:
  dev:
    handlers: [console, file_handler]
    propagate: false
  prod:
    handlers: [file_handler]

root:
  level: DEBUG
  handlers: [console]

Answered By - blueray