[FIXED] What is the environment variable for passing options to the python interpreter when running pyspark?

Issue

There is an environment variable for passing extra options to the python interpreter when invoking pyspark. What is this variable?

$> PYSPARK_DRIVER_PYTHON=ipython <WHAT_NAME_HERE?>="--pylab" pyspark 

It's not mentioned in the official docs, but I know it exists.

Solution

It is PYSPARK_DRIVER_PYTHON_OPTS, sensibly enough.

Answered By - Dave