shoomow•12mo ago

Exception when importing Python's CV2 lib

Cv2 was added to the list of libraries that are installed with one name, but used with another. However, it still doesn't work.

{
    "error": {
        "name": "ExecutionErr",
        "message": "ExitCode: 1, last log lines:\nFile \"/tmp/windmill/wk-default-5adb1337158b-gtrgE/018c0760-ff83-780a-28a4-cf6d47fe78c2/wrapper.py\", line 8, in <module>\n    from u.shoomow import new_patient as inner_script\n  File \"/tmp/windmill/wk-default-5adb1337158b-gtrgE/018c0760-ff83-780a-28a4-cf6d47fe78c2/u/shoomow/new_patient.py\", line 6, in <module>\n    import cv2\n  File \"/tmp/windmill/cache/pip/opencv-python==4.8.1.78/cv2/__init__.py\", line 181, in <module>\n    bootstrap()\n  File \"/tmp/windmill/cache/pip/opencv-python==4.8.1.78/cv2/__init__.py\", line 153, in bootstrap\n    native_module = importlib.import_module(\"cv2\")\n                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/usr/local/lib/python3.11/importlib/__init__.py\", line 126, in import_module\n    return _bootstrap._gcd_import(name[level:], package, level)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nImportError: libGL.so.1: cannot open shared object file: No such file or directory"
    }
}

{
    "error": {
        "name": "ExecutionErr",
        "message": "ExitCode: 1, last log lines:\nFile \"/tmp/windmill/wk-default-5adb1337158b-gtrgE/018c0760-ff83-780a-28a4-cf6d47fe78c2/wrapper.py\", line 8, in <module>\n    from u.shoomow import new_patient as inner_script\n  File \"/tmp/windmill/wk-default-5adb1337158b-gtrgE/018c0760-ff83-780a-28a4-cf6d47fe78c2/u/shoomow/new_patient.py\", line 6, in <module>\n    import cv2\n  File \"/tmp/windmill/cache/pip/opencv-python==4.8.1.78/cv2/__init__.py\", line 181, in <module>\n    bootstrap()\n  File \"/tmp/windmill/cache/pip/opencv-python==4.8.1.78/cv2/__init__.py\", line 153, in bootstrap\n    native_module = importlib.import_module(\"cv2\")\n                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/usr/local/lib/python3.11/importlib/__init__.py\", line 126, in import_module\n    return _bootstrap._gcd_import(name[level:], package, level)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nImportError: libGL.so.1: cannot open shared object file: No such file or directory"
    }
}

14 Replies

Damian Szetela•12mo ago

It's not a problem with import. It is missing libgl dependency. Install it and it should work

rubenf•12mo ago

You need to add a shared lib to the image

shoomow•12mo ago

Pardon my stupidity, but can you tell me how to do that? AI wasn't of any help

Tiago Serafim•12mo ago

You have to self host and you have to either use the INIT_SCRIPT env variable to run apt install or bake a new image that extends the original image from Windmill and installs the lib.

rubenf•12mo ago

Damian Szetela•12mo ago

In my case I use the docker compose file with build directive - it references docker build folder. That in turn takes windmill image and adds needed changes (certificates in my case). You can do the same, but with libgl. See this link for a bit more description how this works: https://stackoverflow.com/a/45173812

Stack Overflow

Docker-compose.yml file that builds a base image, then children bas...

For clarification, when I say base image, I mean the parent image that has all the common configurations, so that the children based on it don't need to download the dependencies individually. Fro...

shoomow•12mo ago

thank you! would it be too presumptuous of me to ask you to share a sanitized version of the Dockerfile you use in this process? I'm fairly new to Docker, and sometimes it's hard to wrap my mind around these things

rubenf•12mo ago

@shoomow you probably could benefit from INIT_SCRIPT

rubenf•12mo ago

https://www.windmill.dev/docs/advanced/preinstall_binaries#init-scripts

Preinstall Binaries | Windmill

Workers in Windmill can preinstall binaries. This allows them to execute these binaries in subprocesses or directly within bash. While some common binaries like npm, aws-cli, kubectl, and helm are already present in the standard images, you can add more by extending the base image of Windmill.

rubenf•12mo ago

INIT_SCRIPT="apt-get ..." pass as env variable

shoomow•12mo ago

I guess I missed this article somehow. Thanks!

Damian Szetela•12mo ago

It seems the Init script is much better for your use case. I will supply dockerfiles if init script won't work for you

shoomow•12mo ago

INIT_SCRIPT seems to be simple for ee. "pass it as env variable" - pass where? I don't know. So I tried the dockerfile way. I created a Dockerfile with following contents:

FROM ghcr.io/windmill-labs/windmill:main

RUN apt-get update
RUN apt install ffmpeg libsm6 libxext6 libgl1

CMD ["windmill"]

FROM ghcr.io/windmill-labs/windmill:main

RUN apt-get update
RUN apt install ffmpeg libsm6 libxext6 libgl1

CMD ["windmill"]

Then I changed docker-compose file like so:

 windmill_worker:
    pull_policy: always
    build:
      context: ./build
    deploy:
      replicas: 1
      resources:
        limits:
          cpus: "1"
          memory: 2048M
    restart: unless-stopped
    environment:
      - DATABASE_URL=${DATABASE_URL}
      - MODE=worker
      - WORKER_GROUP=default
    depends_on:
      db:
        condition: service_healthy
    # to mount the worker folder to debug, KEEP_JOB_DIR=true and mount /tmp/windmill
    volumes:
      # mount the docker socket to allow to run docker containers from within the workers
      - /var/run/docker.sock:/var/run/docker.sock
      - worker_dependency_cache:/tmp/windmill/cache

 windmill_worker:
    pull_policy: always
    build:
      context: ./build
    deploy:
      replicas: 1
      resources:
        limits:
          cpus: "1"
          memory: 2048M
    restart: unless-stopped
    environment:
      - DATABASE_URL=${DATABASE_URL}
      - MODE=worker
      - WORKER_GROUP=default
    depends_on:
      db:
        condition: service_healthy
    # to mount the worker folder to debug, KEEP_JOB_DIR=true and mount /tmp/windmill
    volumes:
      # mount the docker socket to allow to run docker containers from within the workers
      - /var/run/docker.sock:/var/run/docker.sock
      - worker_dependency_cache:/tmp/windmill/cache

Then I recieved an Exception when trying to run docker compose up -d

rubenf•12mo ago

the env variables are passed in environment: block of your docker-compose