[FIXED] How to make `import` and `from import` diverge in Python?

Issue

Say I have the following consecutive lines in a python code base:

from foo import BAR          # succeeds
log.info(f"{dir(BAR)=}")     # succeeds
import foo.BAR               # succeeds
log.info(f"{dir(foo.BAR)=}") # fails, AttributeError no field BAR in module foo

There's no other code in between. If I deliberately wanted to create this effect, how would I do it? I know it's possible because I'm observing it in a large code base running under Python 3.11, but I have no idea how. What feature of the python import system lets these two forms of import diverge? It seems like the ability to import foo.BAR must require foo.BAR to exist, and we even confirm it exists first with from foo import BAR and logging the fields of BAR.

foo is a directory. foo/__init__.py exists and is empty. foo/BAR.py exists and contains top level items like functions and classes.

Solution

This can happen due to dynamic imports. Typical code to do a dynamic import looks like this:

spec = importlib.util.spec_from_file_location(module_name, file_path)
new_module = importlib.util.module_from_spec(spec)
sys.modules[module_name] = new_module

However, if module_name has multiple . separated parts, e.g. foo.bar.buzz, then this diverges from typical Python import behavior in two ways.

1. Normally if you write import foo.bar.buzz python will first import foo then import foo.bar then import foo.bar.buzz. The code above doesn't reproduce this behavior.
2. Additionally, Python takes care of setting an attribute on the parent module for each child module in order to make references like foo.child work.

So mimicking Python's regular import behavior for foo.bar requires:

1. Importing foo and setting sys.modules["foo"].
2. Importing foo.bar and setting sys.modules["foo.bar"].
3. Setting the bar attribute on the foo module object to point at bar. You can do this with setattr(foo, "bar", bar) where foo and bar are the module objects returned from module_from_spec.

Answered By - Joseph Garvin