[FIXED] QTimer.singleshot is causing the GUI to hangout

Issue

I am creating a table that shows the running processes, with a decorator to update these information. Using the decorator as in the code below, causes the GUI to hangs out every time the singleshot is called (every second).

Why the singleshot is causing the GUI to hangs, and how can I get better logic ?

# First create table
data = getProcesses()
tableWidget = QTableWidget()
Layout.addWidget(tableWidget)
fillTable(data, len(data['pid']), len(data), tableWidget)

# get the processes
    def getProcesses():
        allprocesses = {}
        for p in psutil.process_iter():
            try:
                if p.name().lower() in ["python.exe", "pythonw.exe"]: # console, window
                    with p.oneshot():
                        allprocesses.setdefault('pid', []).append(p.pid)
                        allprocesses.setdefault('memory(MB)', []).append(p.memory_full_info().uss/(1024**2))
                        allprocesses.setdefault('memory(%)', []).append(p.memory_percent(memtype="rss"))
                        allprocesses.setdefault('cpu_times(s)', []).append(sum(p.cpu_times()[:2]))
                        allprocesses.setdefault('create_time', []).append(datetime.datetime.fromtimestamp(p.create_time()).strftime("%Y-%m-%d %H:%M:%S"))
                        allprocesses.setdefault('cpu(%)', []).append(p.cpu_percent()/psutil.cpu_count())
            except:
                continue
        del p
        return allprocesses

def updateInfo(data, table):
    try:
        table.clear()
        for p in psutil.process_iter():
            if p.pid in data['pid']:
                try:
                    with p.oneshot():
                        data['memory(MB)'][data['pid'].index(p.pid)]    = p.memory_full_info().uss/(1024**2)
                        data['memory(%)'][data['pid'].index(p.pid)]     = p.memory_percent(memtype="rss")
                        data['cpu_times(s)'][data['pid'].index(p.pid)]  = sum(p.cpu_times()[:2])
                        data['cpu(%)'][data['pid'].index(p.pid)]        = p.cpu_percent()/psutil.cpu_count()
                        self.fillTable(data, len(data['pid']), len(data), table) 
                except:
                    continue
    except:
        pass                      

def tabledecorator(func):
    @functools.wraps(func)
    def wrapper(data, r, c, table):
        func(data, r, c, table)
        QTimer.singleShot(1000, lambda: self.updateInfo(data, table))            
    return wrapper

@tabledecorator
def fillTable(data, r, c, table):
    table.setRowCount(r) 
    table.setColumnCount(c)
    horHeaders = []
    for n, key in enumerate(reversed(sorted(data.keys()))):
        horHeaders.append(key)
        for m, item in enumerate(data[key]):
            newitem = QTableWidgetItem()
            newitem.setData(Qt.DisplayRole, item)
            table.setItem(m, n, newitem)
    table.setHorizontalHeaderLabels(horHeaders)
    table.resizeColumnsToContents()
    table.resizeRowsToContents()
    del horHeaders, n, key, m, item, newitem

Solution

There various performance issues in your implementation, but the most important is that fillTable is called for all items.

Since that function is decorated with the timer, the result is that you will be calling a delayed updateInfo for each row in the table, and since that function again calls the decorated fillTable, you're actually having a huge recursion problem: at every new cycle, the number of function calls grows exponentially.

If you have 2 matching processes, the first time updateInfo is called it will call fillTable two times while creating two QTimers. After one second you'll have two calls to updateInfo, resulting in 4 calls (2 processes multiplied by 2 calls to updateInfo); after another second they will be 8, then 16 and so on.

Another problem with your code is that, at each call to fillTable you're calling three functions that should only be executed once for each cycle:

• setHorizontalHeaderLabels;
• resizeColumnsToContents;
• resizeRowsToContents;

The first one is pointless in any case, since the column labels will certainly not change through the lifetime of your program, and when that function is called it cycles through all its header items to check if any has changed.
The other two are very demanding in terms of performance, since the view is forced to query all items and call underlying functions to compute size hints of each row and column and then adapt the sizes accordingly.

There's really no point in using a single shot timer for this purpose, as the problem is that you're relying on function arguments to update the data, while a better approach is to properly use objects and references.

Other performance issues in your implementation:

• since the keys of the dictionary are known and won't change, there's no point in using setdefault();
• you're constantly retrieving the same lists at each cycle;
• you're constructing the search list every time (which is time and memory consuming);
• some values are clearly constants, there's no need in computing/retrieving them at every cycle;

A possible reimplementation and simplification of your code could be the following:

1. create a normal QTimer in the __init__, which updates the process list (which might change for obvious reasons) and update it;
2. create an empty dictionary with all keys set as empty lists;
3. set the table with the predefined column count and horizontal labels;
4. create a function that cycles through the processes, if one already exists, update its data, otherwise create a new row and append the new data;
5. optimize the function to improve its execution time and memory usage;

CpuCount = psutil.cpu_count()
MemoryRatio = 1024 ** 2

class ProcView(QtWidgets.QWidget):
    def __init__(self):
        super().__init__()
        layout = QtWidgets.QVBoxLayout(self)
        self.table = QtWidgets.QTableWidget(0, 6)
        layout.addWidget(self.table)

        self.headers = 'pid', 'memory(MB)', 'memory(%)', 'cpu_times(s)', 'create_time', 'cpu(%)'
        self.updateColumns = 1, 2, 3, 5
        self.table.setHorizontalHeaderLabels(self.headers)

        self.procTimer = QtCore.QTimer(interval=1000, timeout=self.updateInfo)
        self.filter = 'python3', 'python'
        self.procs = {header:[] for header in self.headers}
        self.procTimer.start()
        self.updateInfo()

    def updateInfo(self):
        pids = self.procs['pid']
        memoryMb = self.procs['memory(MB)']
        memoryPerc = self.procs['memory(%)']
        cpu_times = self.procs['cpu_times(s)']
        create_times = self.procs['create_time']
        cpuPerc = self.procs['cpu(%)']
        for p in psutil.process_iter():
            if not p.name().lower() in self.filter:
                continue
            with p.oneshot():
                if p.pid not in pids:
                    row = len(pids)
                    self.table.insertRow(row)
                    pids.append(p.pid)
                    memoryMb.append(p.memory_full_info().uss / MemoryRatio)
                    memoryPerc.append(p.memory_percent(memtype="rss"))
                    cpu_times.append(sum(p.cpu_times()[:2]))
                    create_times.append(datetime.datetime.fromtimestamp(p.create_time()).strftime("%Y-%m-%d %H:%M:%S"))
                    cpuPerc.append(p.cpu_percent() / CpuCount)
                    for col, header in enumerate(self.headers):
                        item = QtWidgets.QTableWidgetItem()
                        item.setData(QtCore.Qt.DisplayRole, self.procs[header][row])
                        self.table.setItem(row, col, item)
                else:
                    row = pids.index(p.pid)
                    memoryMb[row] = p.memory_full_info().uss / MemoryRatio
                    memoryPerc[row] = p.memory_percent(memtype="rss")
                    cpu_times[row] = sum(p.cpu_times()[:2])
                    cpuPerc[row] = p.cpu_percent() / CpuCount
                    for col in self.updateColumns:
                        item = self.table.item(row, col)
                        item.setData(QtCore.Qt.DisplayRole, self.procs[self.headers[col]][row])

        self.table.resizeColumnsToContents()
        self.table.resizeRowsToContents()

Note that a proper implementation should possibly:

• avoid a dictionary with fields as keys and lists for each field, but eventually a dictionary with pid as keys and a dictionary for each item's field (a specialized class that works as an abstraction layer for the process would be even better);
• use a custom model (with a QTableView);
• verify whenever processes terminate;
• use fixed row sizes and avoid automatic column resizing at each cycle for all columns (it's better to use the Fixed resize mode for common fixed sized columns like the CPU usage and leave the resizing to the user);

Answered By - musicamante