[FIXED] How to handle imports within imported files when they are needed only in some programs

Issue

I have one file common_file.py which contains functions required by other parts of my main program. At the same time, common_file.py imports from only_main_required.py some data that only uses when the main program is running. My problem is that I have another program helper_program.py which uses common_file.py but it's not located in the same directory as the main file (where the program starts, or the current directory in other words). So when I import common_file.py, it attempts to import only_main_required.py but since the current directory is different, the import fails with a ModuleNotFoundError.

A minimal reproducible example:

I have this project structure (super simplified but enough to reproduce what I mean):

project1

├── folder1
│   │
│   ├── only_main_required.py
│   
└── folder2
├    ├── common_file.py
├    └── helper_program.py
└main_file.py (origin of main program)

#only_main_required.py

random_var = False

#common_file.py

from folder1.only_main_required import random_var

#helper_program.py

import common_file

#main_file.py

from folder2 import common_file

So when you import only_main_required when running the main program, there's no problem because the main program originated in project1 folder, so folder1.only_main_required makes sense and Python can find it, but when you import only_main_required from helper_program, it fails because it doesn't know of the existence of project1 or folder1.

I've been looking for a while now but all the solutions I've found were for different situations (for example, a shared lib, but this is not the case here, I don't want a shared lib, I just want some code from a module that happens to import "non-importable" code when running from a specific source) and while I have possible solutions, I am still not sure on which one to choose (or if to choose another solution I am unaware of).

For example, one solution is to wrap the import from common_file.py in a try-except block, and in case there's a ModuleNotFoundError, I just pass the error since I don't need that module anyway.

Another solution is to add the path using sys.path in common_file.py, but this is the solution I like the least since it makes it harder for me to understand what is going on (with the try-except and a couple of comments, it will be clear enough that I am handling the specific case when the source is helper_program.py and not main_file.py)

And another solution is to move helper_program to project1 (the root folder), but I also don't like this solution because it doesn't make any sense at all considering the overall structure of the program (helper_program.py is a program that helps in dealing with stuff related to the folder2 folder).

Also, I need the imports to be in common_file.py and not in an if name guard clause because I need them when the main program is the file importing common_file.py, just not when helper_program.py is the source.

I am amazed that I wasn't able to find anything like my situation. Maybe there's another way in which these situations are usually managed?

Solution

So I want to expand on the comment I made in @Ted Possible's answer.

What ended up working best for my usecase is both changing the current directory AND adding the parent directory to sys.path, like this:

#helper_program.py
import os
os.chdir("../")
import sys
currentdir = os.path.dirname(os.path.abspath(__file__))
parentdir = os.path.dirname(currentdir)
sys.path.insert(0, parentdir)
import common_file

This way, only helper_program.py has to change (which makes sense for my usecase since it's for developing purposes only anyway, that is, it shouldn't be included in the release version). Also, it makes sense because I have too many imports (it's a huge project) and many filepaths, so I would have to make too many changes if I went for any of the other approaches.

Answered By - Luciano P.