[FIXED] How to install a package on Jupyter Notebook running from the Docker Image on Windows10

Issue

I've tried to setup pySpark on Windows 10. After some various challenges, I've decided to use Docker Image instead and it worked great.

The hello world script is working. However, I'm not able to install any packages on Jupyter powered by Docker. Please advise.

Normally, I can use the code below on Anaconda terminal:

Issue:

The following command must be run outside of the IPython shell:

    $ pip install fastavro

I cannot find how to install INSIDE docker. Please advise.

Resources:

• Docker image - jupyter/pyspark-notebook
• Operating System - Windows 10

Solution

In Jupyter cell/IPython shell, you can run:

!pip install PACKAGENAME 

To install package(s). Note the '!' Prefix.

Update

When having multiple environment, in use the system executor(Python) used in that environment.

import sys

!{sys.executable} -m pip install PACKAGENAME

Answered By - Prayson W. Daniel