[FIXED] ModuleNotFoundError in poetry-installed ipython kernel

Issue

I'm trying to use Jupyter Notebooks in a poetry virtual environment but get ModuleNotFoundError when trying to import python packages that are already installed (and managed) by poetry.

I'm running python v3.8.5, poetry v1.1.13 and PyCharm 2022.1.1.

I already added jupyter and ipykernel as dependencies and can confirm that they were added to both pyproject.toml:

jupyter = "^1.0.0"
ipykernel = "^6.13.0"

and poetry.lock:

[[package]]
name = "ipykernel"
version = "6.13.0"
description = "IPython Kernel for Jupyter"
category = "dev"
optional = false
python-versions = ">=3.7"


[[package]]
name = "jupyter"
version = "1.0.0"
description = "Jupyter metapackage. Install all the Jupyter components in one go."
category = "dev"
optional = false
python-versions = "*"

I installed an ipython kernel using poetry:

poetry run ipython kernel install --user --name=poetry_kernel

I launched Jupyter Notebook:

poetry run jupyter notebook

I selected Kernel > Change kernel > poetry_kernel and ran import numpy (which is installed in the poetry virtual environment) but numpy wasn't found. I got the same error trying to import other packages managed by poetry.

I then discovered that I need to set "CONDA_DLL_SEARCH_MODIFICATION_ENABLE" to 1 in kernel.json for poetry_kernel.

I modified kernel.json so it now has the following:

{
 "argv": [
  "C:\\Users\\my-user-name\\AppData\\Local\\pypoetry\\Cache\\virtualenvs\\project-name--o0gsFoF-py3.8\\Scripts\\python.exe",
  "-m",
  "ipykernel_launcher",
  "-f",
  "{connection_file}"
 ],
 "display_name": "poetry_kernel",
 "language": "python",
 "metadata": {
  "debugger": true
 },
 "env": {
     "CONDA_DLL_SEARCH_MODIFICATION_ENABLE": 1
 }
}

I closed the jupyter notebook, and re-ran poetry run jupyter notebook, selected poetry_kernel and re-ran import numpy but got the same error.

I can confirm that I am able to import numpy in PyCharm's Python console, and poetry run some_module.py that imports numpy.

I've little experience using poetry (I was previously launching jupyter notebook from virtualenv environments prior to the migration to poetry) and haven't been able to find any clues.

I should add that don't want to simply pip install numpy (I would need to install many others), and besides I want to utilise poetry's dependency management.

Side note:

After running poetry run jupyter notebook I got the following error and warning:

[I 10:44:24.123 NotebookApp] Kernel started: 220004ff-7963-4f47-abad-59d74789ff98, name: poetry_kernel [IPKernelApp] ERROR | No such comm target registered: jupyter.widget.control [IPKernelApp] WARNING | No such comm: 6b457f5c-1066-48f7-b795-4fa80cf50947

which is evidently harmless, so I'm assuming that the ModuleNotFoundError is not associated with the traceback. I've not been able to find any solutions to the warning and error.

Solution

I think the answer might be similar to this question, where the core problem is that a poetry venv doesn't define packages if they are already in your system lib/python3.{version} install.

You can verify this by looking at the {my_poetry_venv}/lib/python3.{version}/site-packages folder. If numpy isn't there, it won't be in your jupyter kernel.

I've verified on my system that:

• only a subset of poetry.lock is in the venv sitepackages
• Pycharm successfully understands the poetry python interpreter to build system + venv package set
• Jupyter kernel will only find the subset of packages in sitepackages of the venv when you try import {some_package}.

Answered By - Mark_Anderson