shoomow
shoomow12mo ago

Exception when importing Python's CV2 lib

Cv2 was added to the list of libraries that are installed with one name, but used with another. However, it still doesn't work.
{
"error": {
"name": "ExecutionErr",
"message": "ExitCode: 1, last log lines:\nFile \"/tmp/windmill/wk-default-5adb1337158b-gtrgE/018c0760-ff83-780a-28a4-cf6d47fe78c2/wrapper.py\", line 8, in <module>\n from u.shoomow import new_patient as inner_script\n File \"/tmp/windmill/wk-default-5adb1337158b-gtrgE/018c0760-ff83-780a-28a4-cf6d47fe78c2/u/shoomow/new_patient.py\", line 6, in <module>\n import cv2\n File \"/tmp/windmill/cache/pip/opencv-python==4.8.1.78/cv2/__init__.py\", line 181, in <module>\n bootstrap()\n File \"/tmp/windmill/cache/pip/opencv-python==4.8.1.78/cv2/__init__.py\", line 153, in bootstrap\n native_module = importlib.import_module(\"cv2\")\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/usr/local/lib/python3.11/importlib/__init__.py\", line 126, in import_module\n return _bootstrap._gcd_import(name[level:], package, level)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nImportError: libGL.so.1: cannot open shared object file: No such file or directory"
}
}
{
"error": {
"name": "ExecutionErr",
"message": "ExitCode: 1, last log lines:\nFile \"/tmp/windmill/wk-default-5adb1337158b-gtrgE/018c0760-ff83-780a-28a4-cf6d47fe78c2/wrapper.py\", line 8, in <module>\n from u.shoomow import new_patient as inner_script\n File \"/tmp/windmill/wk-default-5adb1337158b-gtrgE/018c0760-ff83-780a-28a4-cf6d47fe78c2/u/shoomow/new_patient.py\", line 6, in <module>\n import cv2\n File \"/tmp/windmill/cache/pip/opencv-python==4.8.1.78/cv2/__init__.py\", line 181, in <module>\n bootstrap()\n File \"/tmp/windmill/cache/pip/opencv-python==4.8.1.78/cv2/__init__.py\", line 153, in bootstrap\n native_module = importlib.import_module(\"cv2\")\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/usr/local/lib/python3.11/importlib/__init__.py\", line 126, in import_module\n return _bootstrap._gcd_import(name[level:], package, level)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nImportError: libGL.so.1: cannot open shared object file: No such file or directory"
}
}
14 Replies
Damian Szetela
Damian Szetela12mo ago
It's not a problem with import. It is missing libgl dependency. Install it and it should work
rubenf
rubenf12mo ago
You need to add a shared lib to the image
shoomow
shoomow12mo ago
Pardon my stupidity, but can you tell me how to do that? AI wasn't of any help
Tiago Serafim
Tiago Serafim12mo ago
You have to self host and you have to either use the INIT_SCRIPT env variable to run apt install or bake a new image that extends the original image from Windmill and installs the lib.
rubenf
rubenf12mo ago
^
Damian Szetela
Damian Szetela12mo ago
In my case I use the docker compose file with build directive - it references docker build folder. That in turn takes windmill image and adds needed changes (certificates in my case). You can do the same, but with libgl. See this link for a bit more description how this works: https://stackoverflow.com/a/45173812
Stack Overflow
Docker-compose.yml file that builds a base image, then children bas...
For clarification, when I say base image, I mean the parent image that has all the common configurations, so that the children based on it don't need to download the dependencies individually. Fro...
shoomow
shoomow12mo ago
thank you! would it be too presumptuous of me to ask you to share a sanitized version of the Dockerfile you use in this process? I'm fairly new to Docker, and sometimes it's hard to wrap my mind around these things
rubenf
rubenf12mo ago
@shoomow you probably could benefit from INIT_SCRIPT
rubenf
rubenf12mo ago
Preinstall Binaries | Windmill
Workers in Windmill can preinstall binaries. This allows them to execute these binaries in subprocesses or directly within bash. While some common binaries like npm, aws-cli, kubectl, and helm are already present in the standard images, you can add more by extending the base image of Windmill.
rubenf
rubenf12mo ago
INIT_SCRIPT="apt-get ..." pass as env variable
shoomow
shoomow12mo ago
I guess I missed this article somehow. Thanks!
Damian Szetela
Damian Szetela12mo ago
It seems the Init script is much better for your use case. I will supply dockerfiles if init script won't work for you
shoomow
shoomow12mo ago
INIT_SCRIPT seems to be simple for ee. "pass it as env variable" - pass where? I don't know. So I tried the dockerfile way. I created a Dockerfile with following contents:
FROM ghcr.io/windmill-labs/windmill:main

RUN apt-get update
RUN apt install ffmpeg libsm6 libxext6 libgl1

CMD ["windmill"]
FROM ghcr.io/windmill-labs/windmill:main

RUN apt-get update
RUN apt install ffmpeg libsm6 libxext6 libgl1

CMD ["windmill"]
Then I changed docker-compose file like so:
windmill_worker:
pull_policy: always
build:
context: ./build
deploy:
replicas: 1
resources:
limits:
cpus: "1"
memory: 2048M
restart: unless-stopped
environment:
- DATABASE_URL=${DATABASE_URL}
- MODE=worker
- WORKER_GROUP=default
depends_on:
db:
condition: service_healthy
# to mount the worker folder to debug, KEEP_JOB_DIR=true and mount /tmp/windmill
volumes:
# mount the docker socket to allow to run docker containers from within the workers
- /var/run/docker.sock:/var/run/docker.sock
- worker_dependency_cache:/tmp/windmill/cache
windmill_worker:
pull_policy: always
build:
context: ./build
deploy:
replicas: 1
resources:
limits:
cpus: "1"
memory: 2048M
restart: unless-stopped
environment:
- DATABASE_URL=${DATABASE_URL}
- MODE=worker
- WORKER_GROUP=default
depends_on:
db:
condition: service_healthy
# to mount the worker folder to debug, KEEP_JOB_DIR=true and mount /tmp/windmill
volumes:
# mount the docker socket to allow to run docker containers from within the workers
- /var/run/docker.sock:/var/run/docker.sock
- worker_dependency_cache:/tmp/windmill/cache
Then I recieved an Exception when trying to run docker compose up -d
rubenf
rubenf12mo ago
the env variables are passed in environment: block of your docker-compose