[FIXED] Jupyter Notebook hangs after trying to upgrade to new version of Python

Issue

I recently realized that while I have Python 3.12.0 installed, Jupyter notebook was using Python 3.7.3. I thought it would be a good idea to try and get Jupyter to use the same version of Python that python3 is referring to. I spent a few hours on this and seem to have completely broken Jupyter. I am using MacOS 14.0.

Jupyter appears to startup just fine. Although I do see this warning which I believe is new:

/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/json/encoder.py:257: UserWarning: date_default is deprecated since jupyter_client 7.0.0. Use jupyter_client.jsonutil.json_default.
  return _iterencode(o, 0)

But when I try to create a new notebook it hangs. I see a ton of output on the console. The output is probably too long to print in its entirety but it start with:

[I 15:08:35.774 NotebookApp] Creating new notebook in
/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/json/encoder.py:257: UserWarning: date_default is deprecated since jupyter_client 7.0.0. Use jupyter_client.jsonutil.json_default.
  return _iterencode(o, 0)
Exception in callback <TaskWakeupMethWrapper object at 0x10fa70fd8>(<Future finis...c7b"\r\n\r\n'>)
handle: <Handle <TaskWakeupMethWrapper object at 0x10fa70fd8>(<Future finis...c7b"\r\n\r\n'>)>
Traceback (most recent call last):
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/asyncio/events.py", line 88, in _run
    self._context.run(self._callback, *self._args)
RuntimeError: Cannot enter into task <Task pending coro=<HTTP1ServerConnection._server_request_loop() running at /Users/arilamstein/Library/Python/3.7/lib/python/site-packages/tornado/http1connection.py:825> wait_for=<Future finished result=b'GET /api/co...ac7b"\r\n\r\n'> cb=[IOLoop.add_future.<locals>.<lambda>() at /Users/arilamstein/Library/Python/3.7/lib/python/site-packages/tornado/ioloop.py:687]> while another task <Task pending coro=<RequestHandler._execute() running at /Users/arilamstein/Library/Python/3.7/lib/python/site-packages/tornado/web.py:1711> cb=[_HandlerDelegate.execute.<locals>.<lambda>() at /Users/arilamstein/Library/Python/3.7/lib/python/site-packages/tornado/web.py:2361]> is being executed.
Exception in callback <TaskWakeupMethWrapper object at 0x10f0dd6a8>(<Future finis...c7b"\r\n\r\n'>)
handle: <Handle <TaskWakeupMethWrapper object at 0x10f0dd6a8>(<Future finis...c7b"\r\n\r\n'>)>
Traceback (most recent call last):
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/asyncio/events.py", line 88, in _run
    self._context.run(self._callback, *self._args)
RuntimeError: Cannot enter into task <Task pending coro=<HTTP1ServerConnection._server_request_loop() running at /Users/arilamstein/Library/Python/3.7/lib/python/site-packages/tornado/http1connection.py:825> wait_for=<Future finished result=b'GET /nbexte...ac7b"\r\n\r\n'> cb=[IOLoop.add_future.<locals>.<lambda>() at /Users/arilamstein/Library/Python/3.7/lib/python/site-packages/tornado/ioloop.py:687]> while another task <Task pending coro=<RequestHandler._execute() running at /Users/arilamstein/Library/Python/3.7/lib/python/site-packages/tornado/web.py:1711> cb=[_HandlerDelegate.execute.<locals>.<lambda>() at /Users/arilamstein/Library/Python/3.7/lib/python/site-packages/tornado/web.py:2361]> is being executed.
Exception in callback <TaskWakeupMethWrapper object at 0x10f0dd528>(<Future finis... GMT\r\n\r\n'>)
handle: <Handle <TaskWakeupMethWrapper object at 0x10f0dd528>(<Future finis... GMT\r\n\r\n'>)>

While I would still like to have Jupyter use the same version of Python that python3 refers to, at this point I would settle for simply unbreaking Jupyter.

Any help would be appreciated!

Solution

A friend just helped me solve this problem. Rather than delete the question altogether, I'll leave it here in case it helps someone in the future.

Many years ago (my first go with learning Python) I believe that I installed jupyter with some flavor of conda. That, in turn, I believe installed jupyter + several helper files in /usr/local/bin.

I was now (trying to) use jupyter with normal pip. But even after deleting the packages and conda, these files were still getting called from the /System version of Python. This happened even when running python in a virtual environment.

Deleting those files from /usr/local/bin solved my problem. Note that the computer gave me a strong warning before doing that (I did not have to sudo to delete them, but it was some other form of warning).

Someone had once warned me about mixing jupyter versions (conda vs pip). I think that this is what they were referring to.

Answered By - Ari