[FIXED] joblib.Parallel running through spyder hanging on Windows

Issue

I'm running Python 3.5.1 on Windows Server 2013 at work. I have some embarrassingly parallel tasks that seem to work on Python 2.7 with basically the same code, but I am unable to figure out how to get it to run on Python 3.5.1.

I'm using Anaconda 2.4.1

The code looks like this... I've stripped it down to basically the minimum.

\
->main.py
\apackage\
->__init__.py
->amodule.py

Code for main.py

from tpackage import AClass

def go():
    x = AClass().AFunction()
    return x
if __name__ == '__main__':
    x = go()
    print(x)

Code for __init__.py

from .amodule import AClass

__all__ = ['AClass']

Code for amodule.py

from joblib import Parallel, delayed

class AClass(object):
    def AFunction(self):
        x = Parallel(n_jobs=2,verbose=100)(
            delayed(add1)(i) for i in range(10)
            )
        return x

def add1(x):   
    return x + 1

Does this have anything to do with the need for a if __name__ == '__main__': statement? I didn't think I would need this because the Parallel is protected inside a def statement already and should only run when the __main__ module is called, which should only happen once.

I should add that if I change n_jobs=1 in amodule.py, everything works fine.

Update:

So after further review, it appears that this probably something to do with spyder. I'm using spyder 2.3.8. When I have spyder execute this is a dedicated window, it works. But when it runs in the interactive IPython console, it fails. I can also run the program directly from the command line without problems.

Update 2:

After further review, this really has to do with IPython being in a different working directory than the *.py file. Get those lined up and this works.

Solution

(Spyder dev here) If this problem is caused by runfile setting the working directory, you can prevent that to happen by going to the menu entry

Run > Configuration per file

(or pressing Ctrl+F6) and selecting the option called The current working directory.

Notes:

1. The run configuration is saved for each file and Spyder remembers it after restarts.
2. The answer has been updated for Spyder 3.2 and above.

Answered By - Carlos Cordoba