Josh
Josh2mo ago

Almost all executions fail with 'error: Unexpected reading "/tmp/windmill/wk-default-a8f...'

I recently deployed a new instance of Windmill using the provided docker-compose reference configurations. I'm able to log in and navigate the application, but when I attempt to execute Flows/Scripts, they all fail with error:
--- BUN INSTALL --- error: Unexpected reading "/tmp/windmill/wk-default-cdbe7a8fd670-vi95I/0195394b-643d-3729-5eba-2d400a8cd5cc/build.js" Bun v1.2.3 (Linux x64 baseline)
I have not been able to find any additional logs that would indicate what's happening, and I'm not sure how to proceed. I should note that I first encountered this when trying to sync items from the hub. I have never been able to successfully do so. Most manually created test flows/scripts fail as well. Interestilngly, some do: e.g. Go seems to work while Bash does not.
11 Replies
rubenf
rubenf2mo ago
you must be on a special setup, vanilla docker-compose works fine maybe remove volume mounts just in case?
Josh
JoshOP2mo ago
It is a bit special - I'm deploying on a Synology NAS, so I have volume mounts pointing to a specific folder safe for storage. I'll try removing the mounts to see if that clears things up at least and go from there
tim
tim4w ago
I just have the same issue, I was upgrading from old docker compose, aftering running docker compose pull and restart the server, both bun and deno scripts are unable to deploy and execute.
tim
tim4w ago
I found the path of the folder is empty. Maybe I should reset the worker?
No description
tim
tim4w ago
Deno is doing fine, but the bun environment seems to be broken, is there manual way to reconfigure the bun dependencies? Or is there any logs I could look into further? I am currently switching to deno for all scripts, but I think it is worth look into this issue.
rubenf
rubenf4w ago
Did you remove the mounts? And are you on a standard vps ?
tim
tim4w ago
Yes, I am running in a standard virtual machine, the deploy and run of Deno script have no problem creating new folder and file under
/tmp/windmill
/tmp/windmill
directory. Just as Josh mentioned, all new deploy and old script executions failed for bun.
rubenf
rubenf4w ago
Did you remove the mounts?
tim
tim4w ago
What do you mean by removing the mounts? Do I have to remove the mount for the docker local volume and then reattached? I saw the worker is configured with these "cache" and "logs" 2 volumes, doesn't seems to be relevant to the error message like "/tmp/windmill/wk-default-cdbe7a8fd670-vi95I/0195394b-643d-3729-5eba-2d400a8cd5cc/build.js".
windmill_worker:
image: ${WM_IMAGE}
pull_policy: always
deploy:
replicas: 3
resources:
limits:
cpus: "1"
memory: 2048M
# for GB, use syntax '2Gi'
restart: unless-stopped
environment:
- DATABASE_URL=${DATABASE_URL}
- MODE=worker
- WORKER_GROUP=default
depends_on:
db:
condition: service_healthy
# to mount the worker folder to debug, KEEP_JOB_DIR=true and mount /tmp/windmill
volumes:
# mount the docker socket to allow to run docker containers from within the workers
- /var/run/docker.sock:/var/run/docker.sock
- worker_dependency_cache:/tmp/windmill/cache
- worker_logs:/tmp/windmill/logs
logging: *default-logging
windmill_worker:
image: ${WM_IMAGE}
pull_policy: always
deploy:
replicas: 3
resources:
limits:
cpus: "1"
memory: 2048M
# for GB, use syntax '2Gi'
restart: unless-stopped
environment:
- DATABASE_URL=${DATABASE_URL}
- MODE=worker
- WORKER_GROUP=default
depends_on:
db:
condition: service_healthy
# to mount the worker folder to debug, KEEP_JOB_DIR=true and mount /tmp/windmill
volumes:
# mount the docker socket to allow to run docker containers from within the workers
- /var/run/docker.sock:/var/run/docker.sock
- worker_dependency_cache:/tmp/windmill/cache
- worker_logs:/tmp/windmill/logs
logging: *default-logging
After removing the
worker_dependency_cache
worker_dependency_cache
and
worker_logs
worker_logs
, still having the same error when deploying new bun script
job=01957638-f2bb-b7cb-6238-488cb16750fc tag=bun worker=wk-default-42e890aec2d2-XWzal hostname=42e890aec2d2

error: Unexpected reading "/tmp/windmill/wk-default-42e890aec2d2-XWzal/01957638-f2bb-b7cb-6238-488cb16750fc/build.js"

Bun v1.2.3 (Linux x64 baseline)
Non-zero exit status for bun build: 1
job=01957638-f2bb-b7cb-6238-488cb16750fc tag=bun worker=wk-default-42e890aec2d2-XWzal hostname=42e890aec2d2

error: Unexpected reading "/tmp/windmill/wk-default-42e890aec2d2-XWzal/01957638-f2bb-b7cb-6238-488cb16750fc/build.js"

Bun v1.2.3 (Linux x64 baseline)
Non-zero exit status for bun build: 1
rubenf
rubenf4w ago
No idea, try reproducing on a fresh vps on a new instance. If it was a common issue we would have heard about it a lot.
tim
tim4w ago
I solved this by upgrading the system from Centos 7 to Rocky Linux 8. Really appreciate your help!

Did you find this page helpful?