[FIXED] Python Asyncio global variable update not detected

Issue

I have a program that uses asyncio. I don't really use it in my own code, but need it for a dependency. So just my main function is async. My main function is basically an infinite while loop that should run until a stop function is called. I tried to use a global variable, but it seems, that the while loop does not register any change of this global variable.

Here is the problematic code, broken down to the problem:

This is my main.py

import asyncio
import time
import foo # < file with my own class doing work

RUNNING = True

async def main():
  f = foo.Foo()
  i = 0

  # global RUNNING < should not be needed as we only read (even with the statement it does not work)
  while RUNNING:
    print("running")
    time.sleep(1) # normal sleep to simulate normal computation time of program
    
    f.do_work(i)

    i = i + 1

def stop():
  print("stop call received")
  global RUNNING
  RUNNING = False


if __name__ == "__main__":
  print("starting")
  asyncio.run(main())

and my foo.py:

import main

class Foo:
  def do_work(self, i):

    # stop if a specific condition is satisfied
    if i == 3:
      main.stop()
      print(main.RUNNING)

When I run this, my output is:

starting
running
running
running
running
stop call received
False
running
running

So clearly the stop function is called and the global value is updated. But still this update is not detected in the main-function. Why?

I am also open for other possibilities. I tried to use asyncio.Event, but I have the same problem that the event.set() has no effect inside the main-function.

I tried using global RUNNING at different locations or changing the import statement. What confuses me the most, is that reading the value of RUNNING or main.RUNNING in the other file results in the correct value of False.

Solution

Usually Python will import a single instance of each module file and keep it in memory.

That is for normal programs, and when all .py files are part of a properly configured package.

What you are operating with here is not a proper configured package: it is a bunch of .py files with a lucky configuration (likely after some trial and error), so that you don't get any import errors.

And in this case, you end up with your main.py file imported twice and and existing twice in memory: each copy of which will contain a distinct RUNNING variable, You will get there both a sys.modules["__main__"] module, corresponding to the main file when your process is started, and then when it is imported from foo you will get another instance at sys.modules["main"].

The proper fix would be to create a directory, move both your main.py and your foo.py file into there, and create a pyproject.toml so that you have a proper package, and all your files are loaded properly in memory.

So - to recap: you create a pyproject.toml file, and a mypackage (or whatewver name) where you move your ".py" files. Then you create a __main__.py file that will to the role of the block guarded by if __name__ == "__main__": in your original project.

then you install your project with pip install -e . :this will turn your local files into an installed package in your active virtual-environment (or otherwise active Python interpreter). And finally you can run your project with python -m mypackage: one single copy of each of your files is read, they can locate and refer to each other without fuzz, and you are good to go. (just pay attention because you are on the brink of a circular import problem there - it does not happen as you just change the RUNNING variable from inside a function called after everything is imported).

Minimal pyproject.toml

[build-system]
requires = ["setuptools", "wheel"]
build-backend = "setuptools.build_meta"

[project]
name = "mypackage" 
version = "1.0"

[tool.setuptools]

Inside "mypackage" folder:

main.py:

import time, asyncio

from .foo import Foo

RUNNING = True

async def main():

    foo = Foo()
    while RUNNING:
        await foo.do_work()
        print("running")
        await asyncio.sleep(0.1)
    print("stopping")

foo.py:

from . import main

class Foo:
    def __init__(self):
        self.i = 0
    async def do_work(self):
        self.i += 1
        if self.i == 3:
            main.RUNNING = False

and __main__.py (yes,a file named like that, underscores and all: it is used by Python as the default entry point to run a package)

from . import main

import asyncio
asyncio.run(main.main())

---

As I mentioned above, that is the proper fix. Otherwise, if you want to keep things the way you are doing, the main.py file is the only one actually doubled in memory. Just move your "RUNNING" global state into the foo module:

async def main():
  f = foo.Foo()
  i = 0
  
  # it can be set here, instead of in the module source code
  foo.RUNNING = True
  while foo.RUNNING:
    print("running")
    time.sleep(1) # normal sleep to simulate normal computation time of program
    
    f.do_work(i)

    i = i + 1

def stop():
  print("stop call received")

  # no need for the "global" directive - the variable
  # in `foo` just behaves like an attribute:
  foo.RUNNING = False

Answered By - jsbueno