[FIXED] Python, tkinter: Use multiprocessing.Pool() in a class instance, get new data from pool in a function that runs every second

Issue

I'm still struggling to get a thread pool working inside a class instance. In a file we'll call test.py I have the following:

import multiprocessing as mp

class Test:
    def __init__(self):
        pass

    def exec(self):
        pool = mp.Pool()
        result = pool.map(self.thread, range(0, 4))
        for r in result:
            print(r)

    def thread(self, i):
        return i * 2

In init.py I then have:

import test

t = test.Test()
if __name__ == "__main__":
    t.exec()

This works successfully: When I run init.py from a console the threads execute and return the doubled numbers. The problem is that I want to make my thread pool a class property, not a local variable. The correct content of test.py is thus supposed to be:

import multiprocessing as mp

class Test:
    def __init__(self):
        self.pool = mp.Pool()

    def exec(self):
        result = self.pool.map(self.thread, range(0, 4))
        for r in result:
            print(r)

    def thread(self, i):
        return i * 2

But this returns the following error:

NotImplementedError: pool objects cannot be passed between processes or pickled

I'm not trying to pass the pool between processes: The class instance of Test calling the pool is supposed to remain on the main thread only, hence why it's executed under the if __name__ == "__main__" check. I do however want the pool to be a property of self so it can be reused, otherwise I have to create a new pool each time exec is called.

exec is meant to be called repeatedly every 1 second and use the thread pool to get new data each time. In the full code I use tkinter so self.root.after(1000, self.exec) is used after init.py initially activates this function which is then meant to reschedule itself and run repeatedly: root.after is the only way I'm aware of without the main loop of TK blocking my other loop from running. Here's a full view of what test.py is supposed to look like if we also take tkinter into account, please let me know how that would need to be rewritten to work.

import multiprocessing as mp
import tkinter as tk

class Test:
    def __init__(self):
        self.pool = mp.Pool()
        self.root = tk.Tk()
        self.root.mainloop()

    def exec(self):
        result = self.pool.map(self.thread, range(0, 4))
        for r in result:
            print(r)
        self.root.after(1000, self.exec)

    def thread(self, i):
        return i * 2

Solution

Got it working. The trick was to use two classes, main class issues the thread pool to the secondary class which then uses it to pipe in results from a local function. Something among the following lines worked out for me, rewritten as a representation since my final result diverged a bit already... let me know if you think it can be further improved.

import multiprocessing as mp

class Test1:
    def __init__(self):
        pass

    def exec(self, pool):
        return pool.map(self.thread, range(0, 4))

    def thread(self, i):
        return i * 2

class Test2:
    def __init__(self):
        self.pool = mp.Pool()
        self.other = Test1()

    def obtain(self):
        return self.other.exec(self.pool)

t = Test2()
t.obtain()

Answered By - MirceaKitsune