[FIXED] Setting log file and report HTML file locations

Issue

Below references to questions that have helped me get this far (there's been many more but I didn't save the links unfortunately):

• Change default location log file generated by logger in python
• Setting dynamic folder and report name in pytest

I want to specify the file name and location of both the .log file and the .html report file.

Below is my latest attempt at it.

Function to "create" the fileHandler logger:

def define_file_logger(log_level, formatter, file_location):
    # create directory
    os.makedirs(os.path.dirname(file_location), exist_ok=True)

    # create file handler for logging to file
    file_handler = logging.FileHandler(filename=Path(file_location) / 'log.log')
    file_handler.setLevel(log_level)

    # apply formatter to file handler logger
    file_handler.setFormatter(formatter)

    # request logger
    final_logger = logging.getLogger(__name__)

    # prevent double logs in console
    if final_logger.hasHandlers():
        final_logger.handlers.clear()

    final_logger.propagate = False

    # add handler
    final_logger.addHandler(file_handler)

    # return
    return final_logger

My config.py file (in the root directory):

@pytest.hookimpl(tryfirst=True)
def pytest_load_initial_conftests(args, early_config, parser):
    os.environ['reports_dir'] = str(Path('Reports', f'{datetime.now().strftime("%d.%m.%Y")}', f'{datetime.now().strftime("%H.%M")}'))

My conftest.py file (in the root directory):

@pytest.hookimpl(tryfirst=True)
def pytest_configure(config):
    # set custom options only if none are provided from command line
    if not config.option.htmlpath:
        # set path and file name
        config.option.htmlpath = Path(os.environ['reports_dir']) / 'report.html'
        config.option.self_contained_html = True

When I run this, I get the error:

No such file or directory

I can get it so that the log file is generated in the root directory and the html report in the reports directory if I make this change to the define_file_logger function:

Before:

# create file handler for logging to file
file_handler = logging.FileHandler(filename=Path(file_location) / 'log.log')

After:

# create file handler for logging to file
file_handler = logging.FileHandler(filename='log.log')

Solution

I came back to this and actually managed to figure it out..

This section:

# create directory
os.makedirs(os.path.dirname(file_location), exist_ok=True)

Wasn't creating my desired directory.

The directory should have been like this:

├── Reports
│   └── 08.03.2022
│       ├── 08.35
│       │   └── report.html
│       └── 08.44
│           ├── Execution_Log.log
│           └── report.html

Which is essentially:

Reports > Date > Time > file

BUT it was getting caught up on the folder creation for the TIME folder (for whatever reason).

So now, I have this:

config.py in src directory:

@pytest.hookimpl(tryfirst=True)
def pytest_load_initial_conftests(args, early_config, parser):
    # set directories
    os.environ['current_dir'] = str(Path(__file__))
    os.environ['project_dir'] = str(str([p for p in Path(os.environ['current_dir']).parents if p.parts[-1] == 'src'][0]))
    os.environ['reports_dir'] = str(Path('Reports', f'{datetime.now().strftime("%d.%m.%Y")}', f'{datetime.now().strftime("%H.%M")}'))

conftest.py in src directory:

@pytest.hookimpl(tryfirst=True)
def pytest_configure(config):
    # set path and file name
    config.option.htmlpath = Path(os.environ['reports_dir'], 'report.html')
    config.option.self_contained_html = True

custom_logger.py (doesn't have to be in src directory):

def define_file_logger(log_level: str, formatter: logging.Formatter, file_location: str = None) -> logging.FileHandler:
    # create directory
    os.makedirs(file_location, exist_ok=True)

    # create file handler for logging to file
    file_handler = logging.FileHandler(filename=Path(os.environ['reports_dir'], 'Execution_Log.log'))
    file_handler.setLevel(LOG_MAP[log_level])

    # apply formatter to file handler logger
    file_handler.setFormatter(formatter)

    # return logger
    return file_handler

I believe this is what the cause of my issue was:

# create directory
os.makedirs(os.path.dirname(file_location), exist_ok=True)

Should not have included os.path.dirname, like this:

# create directory
os.makedirs(file_location, exist_ok=True)

I also changed this (Path(file_location) / 'log.log'):

# create file handler for logging to file
file_handler = logging.FileHandler(filename=Path(file_location) / 'log.log')

To this:

# create file handler for logging to file
file_handler = logging.FileHandler(filename=Path(file_location, 'log.log'))

Answered By - Eitel Dagnin