[FIXED] Why am I unable to import the pcse (python crop simulation environment) into my jupyter notebook and how do I fix it?

Issue

I installed pcse using this user guide: https://pcse.readthedocs.io/en/stable/user_guide.html#getting-started . I am unable to import the package in jupyter notebook while I have the environment activated in the miniconda terminal.

(base) C:\Users\merej>activate py3_pcse

(py3_pcse) C:\Users\merej>

I don't know if I'm required to add the virtual environment to jupyter notebook in order to use it so I added it using

pip install --user ipykernel
python -m ipykernel install --user --name=py3_pcse

The Kernel shows up in Jupyter notebook;However, Jupyter notebook is unable to connect to the notebook server. This is not a problem when I create a python3 notebook.

I saw that this may be a problem with Tornado so I uninstalled my current tornado, installed tornado 5.1.1, got an error message saying the tornando 5.1.1 would not work with my current version of jupyter notebook, uninstalled tornado 5.1.1, and reinstalled the updated version of tornado.

(py3_pcse) C:\Users\merej>pip uninstall tornado
Found existing installation: tornado 6.2
Uninstalling tornado-6.2:
  Would remove:
    c:\users\merej\miniconda3\envs\py3_pcse\lib\site-packages\tornado-6.2.dist-info\*
    c:\users\merej\miniconda3\envs\py3_pcse\lib\site-packages\tornado\*
Proceed (Y/n)? y
  Successfully uninstalled tornado-6.2

(py3_pcse) C:\Users\merej>pip install tornado==5.1.1
Collecting tornado==5.1.1
  Downloading tornado-5.1.1.tar.gz (516 kB)
     ---------------------------------------- 516.8/516.8 kB 550.0 kB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: tornado
  Building wheel for tornado (setup.py) ... done
  Created wheel for tornado: filename=tornado-5.1.1-cp38-cp38-win_amd64.whl size=449834 sha256=160b8413fde8328df013e8729e86a92138e32c365dc6d9dc97a3e04b6cca17ea
  Stored in directory: c:\users\merej\appdata\local\pip\cache\wheels\25\a1\e3\b0d37c6c451fc21f290cf026f6352382e6cbced32dc3f6a37a
Successfully built tornado
Installing collected packages: tornado
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
notebook 6.4.12 requires tornado>=6.1, but you have tornado 5.1.1 which is incompatible.
jupyterlab 3.4.4 requires tornado>=6.1.0, but you have tornado 5.1.1 which is incompatible.
jupyter-server 1.18.1 requires tornado>=6.1.0, but you have tornado 5.1.1 which is incompatible.
jupyter-client 7.3.5 requires tornado>=6.2, but you have tornado 5.1.1 which is incompatible.
ipykernel 6.15.2 requires tornado>=6.1, but you have tornado 5.1.1 which is incompatible.
Successfully installed tornado-5.1.1

(py3_pcse) C:\Users\merej>pip uninstall tornado
Found existing installation: tornado 5.1.1
Uninstalling tornado-5.1.1:
  Would remove:
    c:\users\merej\miniconda3\envs\py3_pcse\lib\site-packages\tornado-5.1.1.dist-info\*
    c:\users\merej\miniconda3\envs\py3_pcse\lib\site-packages\tornado\*
Proceed (Y/n)? y
  Successfully uninstalled tornado-5.1.1

(py3_pcse) C:\Users\merej>pip install tornado
Collecting tornado
  Using cached tornado-6.2-cp37-abi3-win_amd64.whl (425 kB)
Installing collected packages: tornado
Successfully installed tornado-6.2

This Jupyter notebook: https://github.com/ajwdewit/pcse_notebooks/blob/master/01%20Getting%20Started%20with%20PCSE.ipynb doesn't say anything about a kernel so perhaps I installed pcse into the wrong place. My installation passes all of the tests that are listed in the user guide using the miniconda terminal. I saw that this may be a problem with the version of pip that I used to install it since pcse uses python version 3.8; However, I believe that I am using the correct version of pip.

(py3_pcse) C:\Users\merej>pip --version
pip 22.2.2 from C:\Users\merej\miniconda3\envs\py3_pcse\lib\site-packages\pip (python 3.8)

(py3_pcse) C:\Users\merej>python --version
Python 3.8.13

Then I thought it may be a path problem so I made a python path in jupyter notebook using the path given by using python in the miniconda terminal but that also did not work.

(py3_pcse) C:\Users\merej>python
Python 3.8.13 (default, Mar 28 2022, 06:59:08) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import pcse
>>> pcse.__path__
['C:\\Users\\merej\\miniconda3\\envs\\py3_pcse\\lib\\site-packages\\pcse']

My level of understanding of all this is relatively low so feel free to explain anything you might find helpful.

Thanks!

Solution

tl;dr: You want to import pcse, but your jupyter kernel cannot yet do that.

---

You have correctly diagnosed this as being a virtual-environment issue. And you're using conda, so the battle is half won already! You chose to name your conda environment "py3_pcse", a perfectly good name. That environment contains both PCSE and Jupyter dependencies.

Start by trying this in the base environment:

$ python -c 'import pcse'

Notice that, unsurprisingly, it fails.

Now $ conda activate py3_pcse, repeat the import, notice that it (silently) succeeds.

Here is the critically important piece. While the environment is still active, start the kernel with $ jupyter notebook

If the cPython interpreter could access pcse, then the jupyter kernel will be able to, also.

Verify, by importing within a cell.

Answered By - J_H