[FIXED] How do I use a module which employs multiprocessing in a Jupyter Notebook on macOS Sonoma?

Issue

I am trying to run a function from a module I have written myself. This function includes the multiprocessing and I am having trouble getting it to run on macOS. If i call the function in a separate .py file it runs with no errors (on macOS), so I think it is the jupyter notebook, which is the problem.

Update

I forgot to mention that when calling the function on Ubuntu, it runs with no errors - both in the notebook and as the standalone .py file. Also, the multiprocessing package I am using is concurrent futures

The code in question is

import HSTI as hsti
import numpy as np

if __name__ == '__main__':
    lams = np.linspace(8,16,256)*1e-6
    d = np.linspace(3,13,128)*1e-6
    FPI = hsti.FPI(d,lams,297)

When I run this as a standalone python file, I don't get any errors, but when using it in a jupyter notebook i get the following error:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/spawn.py", line 122, in spawn_main
    exitcode = _main(fd, parent_sentinel)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/spawn.py", line 131, in _main
    prepare(preparation_data)
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/spawn.py", line 246, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/spawn.py", line 297, in _fixup_main_from_path
    main_content = runpy.run_path(main_path,
                   ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen runpy>", line 290, in run_path
  File "<frozen runpy>", line 254, in _get_code_from_file
FileNotFoundError: [Errno 2] No such file or directory: '/Users/madslarsen/Dropbox/HSTI/spectral_calibration/HSTI/spectral_calibration/tmm_test.ipynb'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/spawn.py", line 122, in spawn_main
    exitcode = _main(fd, parent_sentinel)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/spawn.py", line 131, in _main
    prepare(preparation_data)
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/spawn.py", line 246, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/spawn.py", line 297, in _fixup_main_from_path
    main_content = runpy.run_path(main_path,
                   ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen runpy>", line 290, in run_path
  File "<frozen runpy>", line 254, in _get_code_from_file
FileNotFoundError: [Errno 2] No such file or directory: '/Users/madslarsen/Dropbox/HSTI/spectral_calibration/HSTI/spectral_calibration/tmm_test.ipynb'

Do you have any suggestions of what to do in this case?

Solution

I was calling the module containing concurrent futures from within a jupyter notebook version 7.0.5. I had success using a jupyter notebook version 6.4.8 instead.

Answered By - madsnibe