[FIXED] Alternative to using __file__ find the binary folder in my python package

Issue

I published a library to PyPI ('nmrsim') that has the following structure (edited to only include the relevant files/folders):

nmrsim
├── nmrsim
│   ├── __init__.py
│   ├── bin
│   │   ├── T1.npz
│   │   ⋮
│   │   └── T11.npz
│   | 
│   └── qm.py
│   
└── tests
    └── test_qm.py

The bin directory contains binaries (sparse matrices of partial solutions) for faster calculations. For qm.py to discover the bin directory regardless of where it's installed and called from (e.g. an end user's main app, the module's test suite tests/test_qm.py, an IDE...), references to __file__ are used, e.g. in qm.py:

def _tm_cache(nspins):
    filename = f'T{nspins}.npz'
    bin_dir = os.path.join(os.path.dirname(__file__), 'bin')
    path = os.path.join(bin_dir, filename)
    try:
        T_sparse = sparse.load_npz(path)
        return T_sparse
    except FileNotFoundError:
        <work around missing file>

The library can be successfully pip-installed and used across platforms. However, using PyInstaller to create a 1-file frozen app fails because the bin directory isn't included. Looking through documentation for PyInstaller and PyOxidizer (a possible alernative), it seems that the use of file is problematic for "freezing" python apps.

How might I refactor code such as this to avoid using __file__?

Solution

I found a working solution:

1. Add an __init__.py to the bin folder.
2. Import the bin folder as a module, e.g. qm.py will have an 'import nmrsim.bin' statement.
3. Use importlib.resources.path to create a context manager object:

filename = f'T{nspins}.npz'
path_context = resources.path(nmrsim.bin, filename)

4. Consume the context manager with a "with...as" block to obtain the path:

with path_context as p:
    path = p

Answered By - Geoffrey Sametz