[FIXED] Python - Hydra - Obtaining configuration in sub-module

Issue

Question

I am currently having trouble with getting hydra to work when not using @hydra.main in the main.py script (refer to error section). I have tried the compose API, but that prevents me from using most of the functionality that hydra provides. I am unsure what I am trying to achieve is possible.

Current Configuration Structure

Project Root
|
├── config
│   ├── default.yaml
│   ├── configuration1.yaml
│   └── configuration2.yaml
└── src
     ├── utils
     │   └── initialise.py
     └── main.py

Code

initialise.py

import hydra
import omegaconf

CONFIG = None

@hydra.main(version_base=None, config_path='../../config', config_name='default')
def init_configs(cfg: omegaconf.DictConfig) -> None:

    global CONFIG
    CONFIG = cfg

    # Other initialisation code here.   

init_configs()

main.py

from utils import initialise

print(initialise.CONFIG)

Error

Primary config module 'config' not found.
Check that it's correct and contains an __init__.py file

Solution

This is an attempt to implement many anti-patterns.

@hydra.main() is designed to be your entry point. Using it a utility function is wrong.

1. Hydra config composition is very expressive. Allow Hydra to fully initialize your config. Do not "initialize" it further. This is preventing overriding things from the command line.
2. Do not pass your config via a global. Pass individual config nodes to sub modules/functions. If you need to access the same config variable from many functional modules use interpolation (key: ${foo.bar}).
3. Attempting to reuse the entry point is a cause of many issues. You didn't write it explicitly, but your code smells a bit like an attempt to reuse the entry point. Don't be afraid to have as many @hydra.main() entry points as you have runnable modules

If you insist on having a config initialization utility, you need to let go of @hydra.main() and use the compose API. You can also combine the two but my recommendation to just let hydra.main() compose your config.

Answered By - Omry Yadan