[FIXED] Using profiles with ipython/jupyter

Issue

Here is help output from ipython:

Examples

ipython notebook                       # start the notebook
ipython notebook --profile=sympy       # use the sympy profile
ipython notebook --certfile=mycert.pem # use SSL/TLS certificate

Seems straightforward .. but then when invoking

$ipython notebook --profile=pyspark

The following warning occurs:

[W 20:54:38.623 NotebookApp] Unrecognized alias: '--profile=pyspark', 
                            it will probably have no effect.

So then the online help is inconsistent with the warning message.

What is the correct way to activate the profile?

Update I tried reversing the order as follows:

$ipython --profile=pyspark notebook

But then a different warning occurs:

[TerminalIPythonApp] WARNING | File not found: u'notebook

Solution

Online docs are inaccurate. The order needs to be reversed - with the -- option before notebook:

$ipython --profile=pyspark notebook 

But .. the issues go beyond that even ..

It seems that jupyter (newer version of ipython) may not respect ipython profiles at all.

There are multiple references to same. Here is one from the Spark mailing list

Does anyone have a pointer to Jupyter configuration with pyspark? The current material on python inotebook is out of date, and jupyter ignores ipython profiles.

Answered By - WestCoastProjects