W
Windmill
W
Windmill
Hello our Windmill instance went down it
Original message was deleted
Windmill
Join
3,296
Members
View on Discord
R
rubenf
•
5/23/23, 9:04 AM
Available to debug whenever
R
rubenf
•
5/23/23, 9:09 AM
What
's likely to have happened is that a job took all the memory and your whole instance was OOM
. I would have thought the OOM killer would trigger in those case but it depends on your setup
, It
's possible to set memory limit in docker compose
:
e
.g
R
rubenf
•
5/23/23, 9:12 AM
but what
's odd is that kernel oom should have triggered
, which would have resulted in one container being killed
:
https://docs.docker.com/config/containers/resource_constraints/#understand-the-risks-of-running-out-of-memory
Docker Documentation
Runtime options with Memory, CPUs, and GPUs
Specify the runtime options for a container
R
rubenf
•
5/23/23, 10:17 AM
if it
's truly oom then even ssh cannot allocate any memory
services:
app:
image: nginx
cpus: "0.5"
mem_reservation: "10M"
mem_limit: "250M"
services:
app:
image: nginx
cpus: "0.5"
mem_reservation: "10M"
mem_limit: "250M"