How to compile pyqt5 app including parallel function with pyinstaller?

[hcji]

Small example:

#!/usr/bin/env python3

import sys
from PyQt5 import QtCore, QtWidgets
from joblib import Parallel, delayed
import threading

def run_iot_uart1():
    print("Job 1")

def run_iot_uart2():
    print("Job 2")

def run_iot_uart3():
    print("Job 3")

def run_iot_uart4():
    print("Job 4")

def run_iot_uart5():
    print("Job 5")

def run_iot_uart6():
    print("Job 6")

def run_iot_uart7():
    print("Job 7")

def run_iot_uart8():
    print("Job 8")

def run_iot_uart9():
    print("Job 9")

def run_iot_uart10():
    print("Job 10")


class Window(QtWidgets.QMainWindow):

    #uart_list = [1,2,3,4,5]

    def __init__(self):
        super(Window, self).__init__()

        self.x_size = 800
        self.y_size = 650
        self.setFixedSize(self.x_size, self.y_size)
        self.setWindowTitle("Test")

        buttons = list()

        self.button1 = QtWidgets.QPushButton("ONE", self)
        self.button1.move(100,100)
        buttons.append(self.button1)

        self.button2 = QtWidgets.QPushButton("TWO", self)
        self.button2.move(100,150)
        buttons.append(self.button2)

        for button in buttons:
            button.clicked.connect(self.button_click_parallel)

        self.show()

    def button_click_parallel(self): # Click any button, process all UART in parallel
        command_list=[run_iot_uart1, run_iot_uart2, run_iot_uart3,
                      run_iot_uart4, run_iot_uart5, run_iot_uart6,
                      run_iot_uart7, run_iot_uart8, run_iot_uart9,
                      run_iot_uart10]
        Parallel(n_jobs=10)(delayed(command_list[i])() for i in range(10))
        sys.stdout.flush()

def run():
    QtWidgets.QApplication.setAttribute(QtCore.Qt.AA_EnableHighDpiScaling, True)
    app = QtWidgets.QApplication(sys.argv)
    GUI = Window()
    sys.exit(app.exec_())

run()

I compile this code and run the exe. when clicking the button, the system popuped a series of windows and breakdown. But it runs aright on pyCharm and spyder.

[rokm]

This won't work with PyInstaller, because the parallel framework you are using (joblib) or at least its default backend (loky) is not supported. (It might work with multiprocess backend, but you will probably explicitly need to add multiprocess.freeze_support()...)

Either way, what's happening is that when you press a button, the multiprocess framework attempts to spawn worker processes using your frozen executable, and with following arguments:
['program.exe', '--multiprocessing-fork', '1420']
['program.exe', '--multiprocessing-fork', '1516']
['program.exe', '--multiprocessing-fork', '1324']
['program.exe', '--multiprocessing-fork', '1548']
['program.exe', '-B', '-S', '-E', '-s', '-c', 'from joblib.externals.loky.backend.resource_tracker import main; main(1212, False)']
['program.exe', '--multiprocessing-fork', '1288']
['program.exe', '--multiprocessing-fork', '1484']
['program.exe', '--multiprocessing-fork', '1356']
['program.exe', '--multiprocessing-fork', '1388']
['program.exe', '--multiprocessing-fork', '1452']
['program.exe', '--multiprocessing-fork', '1580']

i.e., a resource tracker process, and 10 worker processes corresponding to your jobs. But because the entry-point script in your frozen program does not explicitly handle these arguments, your run function is called in each of these processes, which is why you get multiple windows.

A similar pattern occurs with regular multiprocess if multiprocess.freeze_support is not called (because PyInstaller overrides that function to inject its support for spawning multiprocess workers and resource tracker).

So as long as similar support for joblib/loki is missing in PyInstaller (if it is even possible to implement it in such fashion), you cannot use that in PyInstaller-frozen applications.

[rokm]

And for the reference: joblib/loky#236