[FIXED] Child thread can't update parent thread variable?

Issue

I found a very perplexing issue in aiosmtpd/244, sharing my puzzlement here to help me find inspiration on how to troubleshoot.

Situation

• Two servers running Red Hat 7.9
• Works on one, doesn't work on the other
• Problematic code, simplified:

>>> from aiosmtpd.controller import Controller
>>> from aiosmtpd.handlers import Sink
>>> cont = Controller(Sink())
>>> cont.start()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.6/site-packages/aiosmtpd/controller.py", line 180, in start
    raise RuntimeError("Unknown Error, failed to init SMTP server")
RuntimeError: Unknown Error, failed to init SMTP server

The above RuntimeError can only be generated if, at the end of the start() method, self.smtpd is still None.

Expected flow

start() -> _run() -> loop.create_server()

Then upon first connection:

loop.create_server() -> _factory_invoker() -> factory()

The attribute smtpd is set within _factory_invoker in these lines:

        try:
            self.smtpd = self.factory()
            if self.smtpd is None:
                raise RuntimeError("factory() returned None")
            return self.smtpd
        except Exception as err:
            self._thread_exception = err
            return _FakeServer(self.loop)

self._thread_exception is interpreted in these lines:

        if self._thread_exception is not None:
            raise self._thread_exception
        # Defensive
        if self.smtpd is None:
            raise RuntimeError("Unknown Error, failed to init SMTP server")

As you can see, if self.smtpd is None, then it would only happen if there's an error in _factory_invoker(). If so, the error should've been caught and recorded in self._thread_exception. If self._thread_exception is None, then _factory_invoker() had succeeded, and thus self.smtpd couldn't be None.

My main problem in troubleshooting this is that on all my test systems (Windows, Ubuntu, OpenSUSE, MacOS, and FreeBSD), I never encountered a similar error.

So I'm stumped. I'd appreciate any ideas on solving this problem.

Solution

Okay, so apparently I was sent on a wild goose chase.

The compounding factor is because I have suppressed ALL exceptions in these lines:

        try:
            self._testconn()
        except Exception:
            # We totally don't care of exceptions experienced by _testconn,
            # which _will_ happen if factory() experienced problems.
            pass

As a result, a very helpful exception raised within self._testconn() was swallowed.

Apparently self._testconn() failed in its attempt to connect to the host:port, and thus _factory_invoker() never got called.

Once the try..except block is modified to only swallow socket.socket_timeout, the actual problem reared its head, and we can quickly fix it.

It's all documented in aiosmtpd/244 and thus I won't repeat them here :-)

Thank you all who spent the time & effort to try solving this puzzle!

Answered By - pepoluan