Proxmox deployment and OOM(?) errors
Hello everyone,
I've been using Windmill for months on an Ubuntu server, with Windmill deployed in a Docker container. It's been working flawlessly.
Lately I've been looking into using Proxmox on this server for my home setup.
So, I backed up my workspace, installed Promox, created an Ubuntu VM with 16GB of RAM and I deployed Windmill on it in a Docker container.
I then pushed my backed up workspace, which means I am now roughly back to how it was in my previous setup.
Sadly none of my flows work. Lightweight scripts getting and setting Windmill variables work, but heavier flows involving pulling data from an API and pushing to a PostgreSQL db fail immediately with the following error:
Someone else posted a question with a similar error log, and the answer was that he was getting an Out Of Memory error.
I don't see how this could be the case though:
- The VM is currently using 800M of mem out of 15G
-
docker info
returns
I'm a bit lost. Not sure what to do.
Does anyone have an idea of what could be going wrong here? I really just have a barebone setup with default configs.19 Replies
Signal 4 is illegal instruction
which I have no idea how could that happen
So before investigating if it's windmill, I would try to run on same hardware/machinme without proxmox
The previous setup is actually still there on the same machine. I have Ubuntu Server installed on a different SSD. I tried to boot on this SSD yesterday to test my flows, and they work.
So, what I did is just add a new SSD, installed Proxmox on it and booted with that SSD.
I'll try again today just in case and report back!
- I booted on the same machine running Proxmox, except on the SSD which just runs Ubuntu Server 22.04 and the flows worked as expected.
- I happen to have a different machine with Proxmox installed. I deployed Windmill on it in exactly the same way. Tested my flows and I get the same Signal 4 error. So it's not hardware related. It's got to be Proxmox somehow.
So to make it clear:
- Machine 1 -> Proxmox -> Ubuntu Server 22.04 -> Windmill using Docker -> ❌
- Machine 1 -> Ubuntu Server 22.04 -> Windmill using Docker -> ✅
- Machine 2 -> Proxmox -> Ubuntu Server 22.04 -> Windmill using Docker -> ❌
Note: everything is deployed with default configs.
Next thing I'd like to try is:
- Machine 1 -> Proxmox -> Fedora 38 -> Windmill using Docker
I don't have much hope but at least it's something!
All I can tell you is that native jobs and non native jobs use very different runtimes
native jobs are pure rust embedded in the backend
non native fork a runtime
polars use specific hardware acceleration like SIMD that are probably not available on proxmox
Alright then I'll see if there's anything I can do about this in the proxmox config
Importing Polars doesn't even work at all within the VM.
But, it's possible to make it work! So for the future peeps who run into this:
SOLUTION
1) Go to the Proxmox UI
2) Go to the Hardware settings of the VM
3) Edit the
Processors
config
4) Change the processor Type
to host
5) Restart your VM
ReferenceGitHub
"Illegal instruction (core dumped)" with pip installation on non-AV...
I installed polars 0.13.12 with pip in Python 3.10. When I try to import it (e.g. with python -m polars), it segfaults with Illegal instruction (core dumped): $ pip install polars Collecting polars...
qq, what's the benefit of using proxmox ?
There are some new changes in Proxmox 8, might want to check this option https://youtu.be/sZcOlW-DwrU?si=kdbD591GDSm3zAIp&t=1009
Craft Computing
YouTube
Let's Install Proxmox 8.0!
Visit https://Linode.com/CraftComputing to start your $100, 60-Day Trial, and move your services to your own personal cloud today!
It's been a while since I've updated my tutorial videos, so today, let's install Proxmox 8.0! This is a beginner's guide to the installation, with step-by-step instructions for installing Proxmox, setting up your fi...
Not completely sure so far, I'm still testing it. It's quite popular within the self-hosting community, so I thought I'd give it a go.
Initially I wanted to upgrade from running the Home Asisstant OS on my pi. But I already had Windmill and a bunch of containers running on my minipc. And running Home Assistant in a container prevents you from using a lot of nice functionalities you otherwise have access to with the Home Assistant OS.
So Proxmox sounded like a perfect solution for this: Proxmox is running on my minipc, and I use it to manage a VM running the Home Assistant OS AND another VM running a regular Ubuntu Server with all my containers.
I quite like the ability to spin up VMs in a really simple way in basically 1 minute. For the people like me who don't know much about VMs or Linux, it makes it really easy to cross the bridge 🙂
Though I haven't tested it yet, it looks like the functionality to backup VMs via snapshots is very powerful and ridiculously easy to use.
Would you be interested in a small PR to the self-host page of windmill to mention proxmox ?
Absolutely! I've never contributed to a public repository on GitHub though, but shouldn't be a problem
🙏
What do you envision for the mention of Proxmox?
Because I gave it some thoughts yesterday, and ultimately I'm just spinning up Ubuntu VMs and deploying Windmill on it via Docker containers. There is nothing special here and Windmill works just as expected following the Docker deployment instructions!
The issue I encountered was not linked to Windmill, but to Proxmox.
Do you think it still makes sense to mention it?
I think a note mentionning how to fix your proxmox specific issue might be sufficient
I have been running windmill in both proxmox vm and lxc without a hitch. i have a fair bit of experience in using proxmox for almost a decade but windmill behaves predictably and configuring it in different environments is actually very ergonomic in my experience. most recently i am trying to patch up the last pieces of a railway template for a one click deploy of all 6 pieces of infra etc. @rubenf such a clean experience so far kudos...
Amazing for the railway template. We always wanted to provide good templates for all cloud providers but always lack time to do so. Happy to reference it in our self hosted docs
@trogdor did you have any luck with your railway template? please link if so!
@trogdor good job mate, don't keep it to yourself now 🙂