[FIXED] %debug in Jupyter Notebook - accessing missing traceback frames

Issue

When I try to debug my code in Jupyter Notebook with %debug I hit the "Oldest frame".

> $HOME/anaconda3/envs/py36/lib/python3.6/site-packages/matplotlib/axes/_base.py(244)_xy_from_xy()
    242         if x.shape[0] != y.shape[0]:
    243             raise ValueError("x and y must have same first dimension, but "
--> 244                              "have shapes {} and {}".format(x.shape, y.shape))
    245         if x.ndim > 2 or y.ndim > 2:
    246             raise ValueError("x and y can be no greater than 2-D, but have "

ipdb> up
> $HOME/anaconda3/envs/py36/lib/python3.6/site-packages/matplotlib/axes/_base.py(385)_plot_args()
    383             x, y = index_of(tup[-1])
    384 
--> 385         x, y = self._xy_from_xy(x, y)
    386 
    387         if self.command == 'plot':

ipdb> up
> $HOME/anaconda3/envs/py36/lib/python3.6/site-packages/matplotlib/axes/_base.py(407)_grab_next_args()
    405                 return
    406             if len(remaining) <= 3:
--> 407                 for seg in self._plot_args(remaining, kwargs):
    408                     yield seg
    409                 return

ipdb> up
*** Oldest frame

The problem is the traceback of the exception is much longer:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-52-c0be78c78358> in <module>()
----> 1 _ = plotSensitivity('inverse_corelation_uniform_log_boost_serial_smooth_lite', 1)

<ipython-input-51-c479a5eeae49> in plotSensitivity(dataset, threshold)
     75                      truePositives * 100,
     76                      label=method,
---> 77                      ls=LS[method[0]])
     78 
     79         plt.xscale('log')

$HOME/anaconda3/envs/py36/lib/python3.6/site-packages/matplotlib/pyplot.py in plot(*args, **kwargs)
   3315                       mplDeprecation)
   3316     try:
-> 3317         ret = ax.plot(*args, **kwargs)
   3318     finally:
   3319         ax._hold = washold

$HOME/anaconda3/envs/py36/lib/python3.6/site-packages/matplotlib/__init__.py in inner(ax, *args, **kwargs)
   1896                     warnings.warn(msg % (label_namer, func.__name__),
   1897                                   RuntimeWarning, stacklevel=2)
-> 1898             return func(ax, *args, **kwargs)
   1899         pre_doc = inner.__doc__
   1900         if pre_doc is None:

$HOME/anaconda3/envs/py36/lib/python3.6/site-packages/matplotlib/axes/_axes.py in plot(self, *args, **kwargs)
   1404         kwargs = cbook.normalize_kwargs(kwargs, _alias_map)
   1405 
-> 1406         for line in self._get_lines(*args, **kwargs):
   1407             self.add_line(line)
   1408             lines.append(line)

$HOME/anaconda3/envs/py36/lib/python3.6/site-packages/matplotlib/axes/_base.py in _grab_next_args(self, *args, **kwargs)
    405                 return
    406             if len(remaining) <= 3:
--> 407                 for seg in self._plot_args(remaining, kwargs):
    408                     yield seg
    409                 return

$HOME/anaconda3/envs/py36/lib/python3.6/site-packages/matplotlib/axes/_base.py in _plot_args(self, tup, kwargs)
    383             x, y = index_of(tup[-1])
    384 
--> 385         x, y = self._xy_from_xy(x, y)
    386 
    387         if self.command == 'plot':

$HOME/anaconda3/envs/py36/lib/python3.6/site-packages/matplotlib/axes/_base.py in _xy_from_xy(self, x, y)
    242         if x.shape[0] != y.shape[0]:
    243             raise ValueError("x and y must have same first dimension, but "
--> 244                              "have shapes {} and {}".format(x.shape, y.shape))
    245         if x.ndim > 2 or y.ndim > 2:
    246             raise ValueError("x and y can be no greater than 2-D, but have "

ValueError: x and y must have same first dimension, but have shapes (5,) and (500,)

Same happens with other packages (e.g. patsy used by statsmodels).

How may I access the frames with my code from the debugger?

Solution

Unfortunately I think you have hit a long-standing bug in ipython where it can't debug the call stack up through generators.

See https://github.com/ipython/ipython/issues/6251 for details. I suspect that this is a result of their logic to hide the last frame (which would be inside ipython), but have no hard evidence for that.

Until they fix it, you can work around the problem by using import pdb; pdb.pm() instead of the magic.

Answered By - Peter Brittain