[FIXED] Configure Docker to dynamically choose first available Jupyter port if 8888 is taken

Issue

tl;dr Is there a way we can configure our Dockerfile and/or docker run command to dynamically choose the next-available port for Jupyter if port 8888 is taken?

We have a Docker that users can run Jupyter notebooks in. However, we're running into issues when users already have an existing Jupyter server open. The reason is because, when they run the Docker, it defaults to port 8888. If another local Jupyter server has port 8888, the Docker Jupyter link generated from docker run will not work (it will display some token authentication screen, but the only token it will accept is the one that is associated with their local Jupyter server, and not the one associated with our Docker).

We tried changing our start_docker.sh script (users will run bash start_docker.sh to start the Docker) to:

# find lowest open port available
PORT=8888

until [[ $(lsof -i -P -n | grep 127.0.0.1:$PORT | wc -l) -eq 0 ]]
  do
    ((PORT=PORT+1))
done

echo $PORT

# run docker and start notebook server
docker run -it \
  -p $PORT:8888 \
  -v ...
  ...

However, Jupyter still opens on port 8888 in this case, and not on the desired $PORT calculated.

In our Dockerfile, our Jupyter configuration settings are:

# jupyter notebook
CMD ["jupyter", "notebook", "--ip=0.0.0.0", "--allow-root"]

We can obviously try to add a parameter --port=8889 (or some other hard-coded port), but that could be a problem if the user also has a local process using the specified port.

Is it possible to configure this, or do we have to insist that the user not have anything using port 8888 when starting our Docker?

Solution

If you run docker run -p with only one port number, Docker picks the host port. You can then use docker port to find the actual host port it picked.

# Start the container.
#
# Start it _in the background_, since we need to probe the dynamically
# assigned port.
#
# If we expect to have arbitrarily many of these containers, we also
# can't assign a fixed --name, so depend on `docker run -d` printing the
# container ID.
#
# Note only one port number `-p 8888` to let Docker pick the host port.
cid=$(docker run -d -i -t -p 8888 ...)

# Print out the port number
echo -n "Listening on "
docker port "$cid" 8888

# Wait for the container to exit on its own
docker wait "$cid"

# Clean up
docker rm "$cid"

Note that the docker port output includes the bind address, usually 0.0.0.0. This isn't an address you can actually use to reach the container. You need to know the host's IP address, or in some environments you might have configured a load balancer or other gateway in front of the system. Docker isn't aware of these details.

Answered By - David Maze