[FIXED] How to change the PYTHONPATH of an ipython kernel?

Issue

I am doing analysis and I intend to restructure my scripts into a package. I have a directory structure like so:

project
|   README.md
|   setup.py
|   requirements.txt
|
└───data
└───notebooks
|     notebook_A.ipynb
|
└───my_package
    |   __init__.py
    |
    └───module_A
    |    __init__.py
    |    source_A.py
    |
    └───module_B
        __init__.py
        source_B.py

First I will crete an environment with Conda:

conda create my_environment

Then, the goal is to make my_package importable in notebooks without loosing the ability to edit the source. So I will run:

$ (my_environment) pip install -e .

This works as expected and creates:

/Applications/anaconda3/envs/my_environment/lib/python3.6/site-packages/my_package.egg-link

Then I want to check everything works:

$ (my_environment) cd notebooks

Everything works in ipython:

$ (my_environment) ipython


In [1]: import src
In [2]: src.__path__
Out[2]: ['/Users/jalmarituominen/Desktop/my_environment_project/src']

But when I run jupyter notebook and run it with my_environment kernel, I get

import sys
sys.path
[1]:
['',
'/Applications/anaconda3/lib/python36.zip',
'/Applications/anaconda3/lib/python3.6',
'/Applications/anaconda3/lib/python3.6/lib-dynload',
'/Applications/anaconda3/lib/python3.6/site-packages',
'/Applications/anaconda3/lib/python3.6/site-packages/aeosa',
'/Applications/anaconda3/lib/python3.6/site-packages/IPython/extensions',
'/Users/jalmarituominen/.ipython']

Obiviously I can't import my_package since its not in the PATH.

However, when I change the kernel to Python 3, I get:

import sys
sys.path
[1]:
['/Applications/anaconda3/envs/my_environment/lib/python36.zip',
'/Applications/anaconda3/envs/my_environment/lib/python3.6',
'/Applications/anaconda3/envs/my_environment/lib/python3.6/lib-dynload',
'',
'/Applications/anaconda3/envs/my_environment/lib/python3.6/site-packages',
'/Users/jalmarituominen/Desktop/my_environment_project',
'/Applications/anaconda3/envs/my_environment/lib/python3.6/site-packages/IPython/extensions',
'/Users/jalmarituominen/.ipython']

And my_package is importable.

For some reason PATHS of these two environments are mixed up. Any idea how to resolve this? Is it possible to manually change the PATH of a Kernel?

Solution

Here is where I wish Jupyter do something to make changing Environments easy. This is what I did to get corrected results:

$ conda activate my_env
$ (my_env) conda install ipykernel -y
$ (my_env) python -m ipykernel install --user --name my_env --display-name "My Env"

I then made sure that I have correct Python path to my env

$ (my_env) jupyter kernelspec list | grep my_env

This gave me the location of my environmental kernel. In it there is a .json setting file that you can edit the path to correct Python, display name, and other things.

After this I could toggle between kernels.

Answered By - Prayson W. Daniel