-
-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Integrating OpenCV with FastAPI and Redis RQ Causes Task/Job Failures #2071
Comments
I guess it was because subprocess call fork() , i am working on this problem by modifing rq core to support multiprocessing |
Looking forward, as I am stuck for last 2 days. And alternative libraries are not as good as OpenCV. Thanks |
Any updates on this? |
@HuseynG I wonder if this has to do with https://www.wefearchange.org/2018/11/forkmacos.rst.html |
I'm working on integrating a set of microservices into our application, focusing on a FastAPI server that interacts with Redis through RQ for task queueing. The setup involves three main files:
The flow is as follows: The FastAPI endpoint receives some inputs and queues jobs in Redis. These jobs are defined in 'worker_tasks.py' and are supposed to be processed by background workers.
However, I'm encountering a peculiar issue: whenever I import 'cv2' (OpenCV) into 'worker_tasks.py', and I receive the following error message: "Work-horse terminated unexpectedly; waitpid returned 6 (signal 6).", but not on terminal though. This problem seems to be specifically related to the inclusion of the OpenCV library in 'worker_tasks.py'.
Here's a simplified version of the code for each component:
'server.py' snippet:
'redis_processes.py' snippet:
'worker_tasks.py' snippet:
The tasks fail only after adding 'import cv2' in 'worker_tasks.py'. I've verified that the tasks work as expected when this import is commented out. My initial guess was it might be a problem with worker environments or a conflict between libraries, but I haven't found conclusive evidence to pinpoint the issue.
Has anyone faced a similar issue or does anyone have insights on why importing OpenCV in a worker task could lead to job failures in this setup? Any suggestions on how to debug or resolve this issue would be greatly appreciated.
I am using python 3.9.18, and here is the link for the requirement.txt link
If you would like to replicate the issue, please following the following link.
The text was updated successfully, but these errors were encountered: