[FIXED] Can't drop into Python debugger in a function called asynchronously?

Issue

In order to debug an Airbrake issue described in Airbrake throwing error "pybrake - ERROR - strconv.ParseInt: parsing "None": invalid syntax", I'm trying to inspect requests prior to sending them to Airbrake by dropping into the iPython debugger using import ipdb; ipdb.set_trace().

To inspect the request, I've set a trace in the send_notic_sync() method of the Notifier (see https://github.com/airbrake/pybrake/blob/master/pybrake/notifier.py):

 def send_notice_sync(self, notice):
    """Sends notice to Airbrake.

    It returns notice with 2 possible new keys:
    - {'id' => str} - notice id on success.
    - {'error' => str|Exception} - error on failure.
    """
    for fn in self._filters:
      r = fn(notice)
      if r is None:
        notice['error'] = 'notice is filtered out'
        return notice
      notice = r

    if time.time() < self._rate_limit_reset:
      notice['error'] = _ERR_IP_RATE_LIMITED
      return notice

    data = jsonify_notice(notice)
    req = urllib.request.Request(self._airbrake_url,
                                 data=data,
                                 headers=self._airbrake_headers)

    try:
      import ipdb; ipdb.set_trace()
      resp = urllib.request.urlopen(req, timeout=5)
    except urllib.error.HTTPError as err:
      resp = err
    except Exception as err: # pylint: disable=broad-except
      notice['error'] = err
      logger.error(notice['error'])
      return notice

This method gets submitted to a ThreadPoolExecutor in pybrake's source code. The problem is, when I try to import a script which calls this function, I am unable to drop into the debugger. Here is what I see when I try:

(venv) Kurts-MacBook-Pro-2:lucy-web kurtpeek$ python manage.py shell
Python 3.6.4 (v3.6.4:d48ecebad5, Dec 18 2017, 21:07:28) 
Type 'copyright', 'credits' or 'license' for more information
IPython 6.3.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import lucy_web.tests.test_airbrake

In [2]: ipdb>                                                                                                       
2018-05-31 11:52:14,155 - pybrake - ERROR - 
> /Users/kurtpeek/Documents/Dev/lucy2/lucy-web/venv/lib/python3.6/site-packages/pybrake/notifier.py(119)send_notice_sync()
    118       import ipdb; ipdb.set_trace()
--> 119       resp = urllib.request.urlopen(req, timeout=5)
    120     except urllib.error.HTTPError as err:
ipdb> ^[ipdb>                                                                                                       
In [2]: 2018-05-31 11:52:14,159 - pybrake - ERROR - 

^[[37;1R> /Users/kurtpeek/Documents/Dev/lucy2/lucy-web/venv/lib/python3.6/site-packages/pybrake/notifier.py(119)send_notice_sync()
    118       import ipdb; ipdb.set_trace()
--> 119       resp = urllib.request.urlopen(req, timeout=5)
    120     except urllib.error.HTTPError as err:
^[[37;1R
^[[37;1RIn [2]: dir()
In [2]: dir()
Out[2]: 
['In',
 'Out',
 '_',
 '__',
 '___',
 '__builtin__',
 '__builtins__',
 '__doc__',
 '__loader__',
 '__name__',
 '__package__',
 '__spec__',
 '_dh',
 '_i',
 '_i1',
 '_i2',
 '_ih',
 '_ii',
 '_iii',
 '_oh',
 'exit',
 'get_ipython',
 'lucy_web',
 'quit']

So, although there are 'flashes' of the output of the debugger, in the end I just end up in the scope of my original ipdb session. How can I make it so that I can set a trace within the send_notice_sync() method?

Solution

I ended up working around this issue by breaking into the debugger in the Notifier's __init__() method, which worked because it is not called asynchronously. Here I was able to determine that the project_id was being passed in as None because I hadn't set my Django AIRBRAKE setting correctly.

Answered By - Kurt Peek