Josh•5w ago

Almost all executions fail with 'error: Unexpected reading "/tmp/windmill/wk-default-a8f...'

I recently deployed a new instance of Windmill using the provided docker-compose reference configurations. I'm able to log in and navigate the application, but when I attempt to execute Flows/Scripts, they all fail with error:

--- BUN INSTALL --- error: Unexpected reading "/tmp/windmill/wk-default-cdbe7a8fd670-vi95I/0195394b-643d-3729-5eba-2d400a8cd5cc/build.js" Bun v1.2.3 (Linux x64 baseline)

I have not been able to find any additional logs that would indicate what's happening, and I'm not sure how to proceed. I should note that I first encountered this when trying to sync items from the hub. I have never been able to successfully do so. Most manually created test flows/scripts fail as well. Interestilngly, some do: e.g. Go seems to work while Bash does not.

11 Replies

rubenf•5w ago

you must be on a special setup, vanilla docker-compose works fine maybe remove volume mounts just in case?

JoshOP•5w ago

It is a bit special - I'm deploying on a Synology NAS, so I have volume mounts pointing to a specific folder safe for storage. I'll try removing the mounts to see if that clears things up at least and go from there

tim•3w ago

I just have the same issue, I was upgrading from old docker compose, aftering running docker compose pull and restart the server, both bun and deno scripts are unable to deploy and execute.

tim•3w ago

I found the path of the folder is empty. Maybe I should reset the worker?

tim•3w ago

Deno is doing fine, but the bun environment seems to be broken, is there manual way to reconfigure the bun dependencies? Or is there any logs I could look into further? I am currently switching to deno for all scripts, but I think it is worth look into this issue.

rubenf•3w ago

Did you remove the mounts? And are you on a standard vps ?

tim•3w ago

Yes, I am running in a standard virtual machine, the deploy and run of Deno script have no problem creating new folder and file under

/tmp/windmill

/tmp/windmill

directory. Just as Josh mentioned, all new deploy and old script executions failed for bun.

rubenf•3w ago

Did you remove the mounts?

tim•3w ago

What do you mean by removing the mounts? Do I have to remove the mount for the docker local volume and then reattached? I saw the worker is configured with these "cache" and "logs" 2 volumes, doesn't seems to be relevant to the error message like "/tmp/windmill/wk-default-cdbe7a8fd670-vi95I/0195394b-643d-3729-5eba-2d400a8cd5cc/build.js".

  windmill_worker:
    image: ${WM_IMAGE}
    pull_policy: always
    deploy:
      replicas: 3
      resources:
        limits:
          cpus: "1"
          memory: 2048M
          # for GB, use syntax '2Gi'
    restart: unless-stopped
    environment:
      - DATABASE_URL=${DATABASE_URL}
      - MODE=worker
      - WORKER_GROUP=default
    depends_on:
      db:
        condition: service_healthy
    # to mount the worker folder to debug, KEEP_JOB_DIR=true and mount /tmp/windmill
    volumes:
      # mount the docker socket to allow to run docker containers from within the workers
      - /var/run/docker.sock:/var/run/docker.sock
      - worker_dependency_cache:/tmp/windmill/cache
      - worker_logs:/tmp/windmill/logs
    logging: *default-logging

  windmill_worker:
    image: ${WM_IMAGE}
    pull_policy: always
    deploy:
      replicas: 3
      resources:
        limits:
          cpus: "1"
          memory: 2048M
          # for GB, use syntax '2Gi'
    restart: unless-stopped
    environment:
      - DATABASE_URL=${DATABASE_URL}
      - MODE=worker
      - WORKER_GROUP=default
    depends_on:
      db:
        condition: service_healthy
    # to mount the worker folder to debug, KEEP_JOB_DIR=true and mount /tmp/windmill
    volumes:
      # mount the docker socket to allow to run docker containers from within the workers
      - /var/run/docker.sock:/var/run/docker.sock
      - worker_dependency_cache:/tmp/windmill/cache
      - worker_logs:/tmp/windmill/logs
    logging: *default-logging

After removing the

worker_dependency_cache

worker_dependency_cache

and

worker_logs

worker_logs

, still having the same error when deploying new bun script

job=01957638-f2bb-b7cb-6238-488cb16750fc tag=bun worker=wk-default-42e890aec2d2-XWzal hostname=42e890aec2d2

error: Unexpected reading "/tmp/windmill/wk-default-42e890aec2d2-XWzal/01957638-f2bb-b7cb-6238-488cb16750fc/build.js"

Bun v1.2.3 (Linux x64 baseline)
Non-zero exit status for bun build: 1

job=01957638-f2bb-b7cb-6238-488cb16750fc tag=bun worker=wk-default-42e890aec2d2-XWzal hostname=42e890aec2d2

error: Unexpected reading "/tmp/windmill/wk-default-42e890aec2d2-XWzal/01957638-f2bb-b7cb-6238-488cb16750fc/build.js"

Bun v1.2.3 (Linux x64 baseline)
Non-zero exit status for bun build: 1

rubenf•3w ago

No idea, try reproducing on a fresh vps on a new instance. If it was a common issue we would have heard about it a lot.

tim•3w ago

I solved this by upgrading the system from Centos 7 to Rocky Linux 8. Really appreciate your help!

Almost all executions fail with 'error: Unexpected reading "/tmp/windmill/wk-default-a8f...'

Did you find this page helpful?