I have reached the limit of my 1000 executions on the free plan: how to self-host on free services?

I started experimenting with Windmill on Saturday just to explore possibilities.

And I was surprised by the power of the tool and very excited.

So, I continued experimenting for a few hours per day from Saturday until yesterday, and my 1000 executions have run out.

This made me think of self-hosting: but here I have several challenges.

a) I don't have the technical knowledge for it.
Is there any possibility of having "click and install" options available, without the need for additional technical configurations?

b) Why don't I try running it on localhost: I don't have a powerful machine to run it locally (I use a Chromebook with penguin Linux to develop my products).

c) I would love to know about any hosting service that is easy to host Windmill, with affordable prices (preferably with a free plan, not just a trial) and with a defined cap when I need a paid plan.

There is a huge list of services that could possibly host Windmill (https://github.com/cloudcommunity/Cloud-Free-Tier-Comparison), but most of them do not have a completely free plan. Since I am from Brazil and the currency Real is devalued against the dollar, it would be great to have a free plan for my first experiments and even for certain side projects.

It seems that the following services would have that.

1) Could someone please confirm if these plans would run Windmill? I do not have technical knowledge about Docker and Kubernetes to evaluate.
2) And assist me by providing a step-by-step guide on how to set up Windmill on any of them? Or any other free service.

Services:

render (Individual plan)
scaleway (Containers - Kubernetes Kapsule plan)
IBM Cloud (always free plan)

aurelienbrabant•8/16/23, 9:37 AM

While it could technically run anywhere (that's software after all) I believe the primary focus for self hosted environments is indeed docker and kubernetes.

If you want to be able to control the cost and manage it using docker you could rent a VPS for like 5 or 10 bucks a month and run the windmill stack on it. Of course you'll be limited in terms of computing power RAM and all but that's what I'd go for.

Sindre•8/16/23, 10:23 AM

Understandable that you want to find something free, a lot of people is strugggeling to get the ends to meet. I will try to help. But remember that most free plan comes with some kind of disadvantage. Like IBM cloud delete it after 30 days with no use. (https://www.ibm.com/cloud/free#FAQs)

1) you need to find a service that gives you a machine you can log into, like aws ec2, in general it's called IaaS. If you have that it can run docker or kubernetes. If you do not have a lot of experience I would use docker. it's simpler. I do believe that most servers you get your hands on will be strong enough to run windmill, but do remember that the server you are getting is crap. so if you do have problems it could be that. I would also try to find a free DB and not host the db on the same server so that the server is doing as little as possible.

2) There is no step-by-step instruction created yet. I suggest that you start with finding a provider that you want to use. And start a server on that cloud provider, then try to follow the docker instructions.
There are also discord message about setting up windmill on cloud providers, so search for that also.
Depending on your use case you may want to change docker-compose and caddyfile (like setting up https and remove the db).

Just report back and I will help you. but you need to do most of the work yourself. but will help you when you get stuck.

IBM Cloud Free Tier

Create your free IBM Cloud account to access over 40 always-free products without time limits. Use your USD 200 credit on any product in the IBM Cloud catalog.

Tiago Serafim•8/16/23, 11:17 AM

Hi Cali, I'm also from BR, nice to meet you. I'm running it on the always free plan from Oracle. They give a 4 CPU arm64 machine with 24GB of RAM. You'll have to google how to set up the basic machine (Oracle's interface is a mess) and then setup Caprover with the Windmill's one-click install.

aurelienbrabant•8/16/23, 11:32 AM

How are they even giving 24GB RAM for free ? It must be shared stuff isn't it ?

Aaurelienbrabant How are they even giving 24GB RAM for free ? It must be shared stuff isn't it ?

Tiago Serafim•8/16/23, 11:35 AM

It's not shared, they are ARM CPUs that they're trying to promote pretty aggressively, I guess.

aurelienbrabant•8/16/23, 11:42 AM

definitely something to consider then!

cali (brazil)OP•8/16/23, 11:46 AM

@aurelienbrabant
Thank you for the suggestion. I did see some VPS options, but I was interested in something free, at least initially.
However, thinking about VPS, what is the minimum amount of RAM would you suggest running windmill without any issues?

cali (brazil)OP•8/16/23, 11:46 AM

@sindresvendby, I appreciate the context and tips. Due to my lack of understanding about Docker and Kubernetes, any technical step in that is a big challenge for me. If I manage to progress on my own and have any doubts, I will ask for your help.

cali (brazil)OP•8/16/23, 11:47 AM

@tserafim, nice to meet you!

Thanks for sharing this alternative. In this Oracle plan, is there an automatic limit to stay in the free tier, or does it start charging automatically if it exceeds?

My challenge would be to understand how to configure these technical aspects. I am looking forward to a future recipe that simplifies the process, like "click-and-install".

Sindre•8/16/23, 11:48 AM

I think the oracle + caprover from @tserafim is the best tip so far. Try that

aurelienbrabant•8/16/23, 11:49 AM

Yeah you should try this one. Just FYI I believe 2GB is enough for 2 workers or so.

Ccali (brazil)@tserafim, nice to meet you! 🇧🇷 Thanks for sharing this alternative. In this ...

Tiago Serafim•8/16/23, 11:49 AM

If you stick to the free tier limits (up to 4 Ampere CPUs and 24GB of RAM and 100GB of disk), they do not charge and you don't need to convert to a paid account. You must, however, use the machines, they cannot sit idle or you can convert to a paid plan and they you'll let you keep it idle if you want but won't charge if you keep within the limits.

Tiago Serafim•8/16/23, 11:52 AM

Before installing Windmill, I run (IIRC) Yet-Another-Bench-Script, and the performance was better than the Intel CPUs from Digital Ocean. So it's definitely something to try out if you don't want to spend money. Beware that I do not run production stuff on that machine yet, so try for yourself if it fits your needs.

cali (brazil)OP•8/16/23, 11:54 AM

great to know.

any tips on automated ways to prevent the machine from idling?
if it goes into idle mode, does it automatically switch to a paid plan or is there any confirmation? I confess that I have a great fear of services that start charging and I only notice it later. I have had challenges with Amazon and Google, for example. And that's why I avoid services like that nowadays haha. They used to be difficult to manage the costs.

Ccali (brazil)great to know. - any tips on automated ways to prevent the machine from idling?...

Tiago Serafim•8/16/23, 11:56 AM

There are some tips online, such as running a bench script in the night. With Windmill you can do that easily, just run a fibonacci script at 3am.

TTiago Serafim There are some tips online, such as running a bench script in the night. With Wi...

cali (brazil)OP•8/16/23, 12:12 PM

Thanks!

cali (brazil)OP•8/19/23, 9:12 PM

After trying several times to get Oracle account and having an error with my card, I finally managed to register with the same card

and configured a large part of what I found about self-hosting Windmill. I also installed CapRover.

I ran the 4 curls I found here (https://www.windmill.dev/docs/advanced/self_host), and then successfully ran "docker compose up -d".

However, I have some questions:

What is the purpose of CapRover and how do I use it? Does it have any relation to Caddy mentioned in the material I followed (*https://www.windmill.dev/docs/advanced/self_host) ?
How can I access the public IP outside the Oracle machine, from my computer? Is this dangerous in terms of vulnerability?
How could I access Windmill from inside the Oracle machine? Could I install something like Ubuntu and run it from there? But how would I run the services in parallel?

cali (brazil)OP•8/19/23, 9:14 PM

I haven't figured out a way to run Windmill yet.

Sindre•8/19/23, 9:23 PM

so caprover is a PaaS that focus on easily install other services. If you use it, you do not need to install windmill with trought the curl commands, you just select windmill and click install. see https://caprover.com/docs/one-click-apps.html

One-Click Apps · CapRover

Sindre•8/19/23, 9:26 PM

if you have run

docker compose up -d

docker compose up -d

then try doing

curl http://localhost

curl http://localhost

to see if you got something up and running.
But it may be easierer to see if you can get up caprover and install it from there.

Tiago Serafim•8/19/23, 9:53 PM

Go to the Caprover admin, and choose the Windmill from the one click apps. You don't have to deal with Caddy or manually set up Windmill. It's ideal for your use case since you only care about running Windmill, not learning how to maintain and glue together everything.
You'll have to set up a domain to connect to your Oracle machine, and Caprover will update its nginx config to map a domain/subdomain to a specific app.

Tiago Serafim•8/19/23, 9:54 PM

You can hide your public IP if you use Cloudflare, but I'd go with this easy install first.

Tiago Serafim•8/19/23, 9:56 PM

You either use Caprover to manage your apps OR use the provided docker-compose from Windmill

cali (brazil)OP•8/20/23, 10:05 AM

Thanks! So, I tried these steps without success:

docker compose stop

sudo docker run -p 80:80 -p 443:443 -p 3000:3000 -v /var/run/docker.sock:/var/run/docker.sock -v /captain:/captain caprover/caprover

sudo docker run -p 80:80 -p 443:443 -p 3000:3000 -v /var/run/docker.sock:/var/run/docker.sock -v /captain:/captain caprover/caprover

(I'm not sure about these commands, but it's what I found to try out.)

But I got errors with the ports.
E.g.:

Bind for 0.0.0.0:3000 failed: port is already allocated.

Bind for 0.0.0.0:3000 failed: port is already allocated.

And I also had problems with ports 80 and 443.

I thought it was a problem with Docker using these ports, and I tried some commands.
For example:

docker container ls

docker container ls

docker rm -f <container-name>

docker rm -f <container-name>

But without success. Because the ports become occupied again shortly after.

Then I checked the Ingress Rules of the Oracle instance.
Previously, I had specified the ports because I believed that I had to open them due to other errors I had.
But now, to test, I also removed those rules.

---

Since I don't understand much of what I'm doing and I'm following scattered tips from these services, it's probably a failure due to my lack of knowledge about the steps and what I should do.

Sindre•8/20/23, 11:04 AM

@tserafim i do not have any experience with caprover. Are you able to help? If not i can help you on the docker approach @cali.renato

Tiago Serafim•8/20/23, 11:43 AM

Hi! You need to open a number of ports using the ingress rules, those ports are required by Caprover https://caprover.com/docs/firewall.html

But the error you're getting about ports 80, 443 and 3000 being used are weird. When creating the machine, did you choose Ubuntu? Because the default is some weird Oracle distro.

@sindresvendby I'm really not sure how to help from here, as a new machine should have those ports unused. In my installs, I just copy and paste the command from Caprover's docs on a new machine and it works.

cali (brazil)OP•8/21/23, 9:19 AM

My instance was actually with Oracle system. I didn't know it could be changed.

So I deleted the instance and created one with Ubuntu.

I ran the ufw commands and the rules were added correctly:

sudo ufw allow 80,443,3000,996,7946,4789,2377/tcp; sudo ufw allow 7946,4789,2377/udp;

sudo ufw allow 80,443,3000,996,7946,4789,2377/tcp; sudo ufw allow 7946,4789,2377/udp;

However, when I run the command

sudo docker run -p 80:80 -p 443:443 -p 3000:3000 -v /var/run/docker.sock:/var/run/docker.sock -v /captain:/captain caprover/caprover

sudo docker run -p 80:80 -p 443:443 -p 3000:3000 -v /var/run/docker.sock:/var/run/docker.sock -v /captain:/captain caprover/caprover

, it gives the error:

Installation failed. Error: Port timed out: 80    at Timeout._onTimeout (/usr/src/app/built/utils/CaptainInstaller.js:137:20)

Installation failed. Error: Port timed out: 80    at Timeout._onTimeout (/usr/src/app/built/utils/CaptainInstaller.js:137:20)

I have already tried some things about it, but without success.

cali (brazil)OP•8/21/23, 9:20 AM

I know this may be outside the scope of your help. So, anyway, I greatly appreciate your help so far.

Unfortunately, I don't know how to proceed yet, but I will explore a little more.
If I discover something, I will let you know.

Otherwise, I will give up and think of another alternative for my needs.

cali (brazil)OP•8/21/23, 9:21 AM

cali (brazil)OP•8/21/23, 11:12 AM

I put

-e BY_PASS_PROXY_CHECK='TRUE'

-e BY_PASS_PROXY_CHECK='TRUE'

in the command and it seems to have worked:

sudo docker run -e BY_PASS_PROXY_CHECK='TRUE' -p 80:80 -p 443:443 -p 3000:3000 -v /var/run/docker.sock:/var/run/docker.sock dockersaturn/captainduckduck

sudo docker run -e BY_PASS_PROXY_CHECK='TRUE' -p 80:80 -p 443:443 -p 3000:3000 -v /var/run/docker.sock:/var/run/docker.sock dockersaturn/captainduckduck

So I moved on to the next step and ran the command sudo caprover serversetup, but when I entered the IP address of the server, it returned

'You may have specified a wrong IP address or not already started CapRover container on your server! Error: RequestError: Error: connect ETIMEDOUT XXX.XXX.XX.XX:3000'

'You may have specified a wrong IP address or not already started CapRover container on your server! Error: RequestError: Error: connect ETIMEDOUT XXX.XXX.XX.XX:3000'

However, when I run the command

sudo docker container ls

sudo docker container ls

, it seems that the CapRover is running.

cali (brazil)OP•8/21/23, 11:12 AM

Ccali (brazil)I put `-e BY_PASS_PROXY_CHECK='TRUE'` in the command and it seems to have worked...

Tiago Serafim•8/21/23, 11:15 AM

Hi Cali, I noticed that in this command you're using

dockersaturn/captainduckduck

dockersaturn/captainduckduck

instead of

caprover/caprover

caprover/caprover

, any reason?

Tiago Serafim•8/21/23, 11:16 AM

I'm not really sure why you're experiencing these errors, sorry I couldn't help you with a precise reason. But, maybe you installed docker from Snap? Caprover mention on its docs to not use that because it's buggy.

cali (brazil)OP•8/21/23, 11:17 AM

Oops! I probably copied and pasted this part from a tutorial mistakenly.
I will try to redo the command.

I need to find how I can stop all Docker containers

Tiago Serafim•8/21/23, 11:17 AM

Try this section https://caprover.com/docs/troubleshooting.html#how-to-stop-and-remove-captain

Troubleshooting · CapRover

TTiago Serafim I'm not really sure why you're experiencing these errors, sorry I couldn't help ...

cali (brazil)OP•8/21/23, 11:18 AM

I used apt-get install

TTiago Serafim Try this section https://caprover.com/docs/troubleshooting.html#how-to-stop-and-...

cali (brazil)OP•8/21/23, 11:18 AM

great. I'll try that

Tiago Serafim•8/21/23, 11:32 PM

Hi @cali.renato, did it work?

cali (brazil)OP•8/22/23, 9:25 AM

After many attempts, I finally managed to install caprover. Thank you for your support.
The issue with ports was quite a challenge.
Additionally, I'm using Cloudflare, which could also cause problems at some point, from what I've seen.

I just asked to install Windmill and encountered an error. Do you know what the issue was?

Deploying windmill-lsp (might take up to a minute)

Deploying windmill-lsp (might take up to a minute)

Failed: Error: Request failed with status code 504

Failed: Error: Request failed with status code 504

cali (brazil)OP•8/22/23, 9:27 AM

And how can I configure WM Base URL (Server fully qualified url of the user-facing URL)?
Do I need to set up any DNS configurations on my server beforehand for this?

cali (brazil)OP•8/22/23, 9:31 AM

Out of curiosity, another issue was the configuration of the machine in Oracle. To stay in the always free tier, the configuration had to be set to:

Shape: VM.Standard.E2.1.Micro
OCPU count: 1
Network bandwidth (Gbps): 0.48
Memory (GB): 1
Local disk: Block storage only

Ccali (brazil)After many attempts, I finally managed to install caprover. Thank you for your s...

Tiago Serafim•8/22/23, 11:29 AM

That might be because you asked for a specific version of

windmill

windmill

that doesn't have the

windmill-lsp

windmill-lsp

image published or because the GH image hosted errored.

TTiago Serafim That might be because you asked for a specific version of `windmill` that doesn'...

cali (brazil)OP•8/22/23, 11:30 AM

That is the error

Ccali (brazil)And how can I configure WM Base URL (Server fully qualified url of the user-faci...

Tiago Serafim•8/22/23, 11:30 AM

When seting up using the one click, you must choose a domain e.g, wm.example.com, and then on Cloudflare you set that

wm

wm

to point to your Oracle machine IP. You can always change this in the ENV variables

Ccali (brazil)Out of curiosity, another issue was the configuration of the machine in Oracle. ...

Tiago Serafim•8/22/23, 11:31 AM

It seems like you chose the x86 machine, but the good thing about Oracle's offer is the arm64 version which you can use up to 4vCPUs and 24GB of ram

TTiago Serafim It seems like you chose the x86 machine, but the good thing about Oracle's offer...

cali (brazil)OP•8/22/23, 11:54 AM

Now I see! Great to know.

TTiago Serafim That might be because you asked for a specific version of `windmill` that doesn'...

cali (brazil)OP•8/22/23, 11:56 AM

I left the pre-filled version

Ccali (brazil)I left the pre-filled version

Tiago Serafim•8/22/23, 12:13 PM

That's weird, this version exists and I tested before submitting the one-click yaml

cali (brazil)OP•8/22/23, 12:37 PM

I deleted the app and reinstalled. Now it worked.

nowN the only issue is that it does not have https in the subdomain.
and I think that has to do with cloudflare and the levels of subdomains it offers ssl.
problem for another time. haha.

I've already made a lot of progress with your help.

thanks!

cali (brazil)OP•8/22/23, 12:42 PM

How can I upgrade the Windmill version over time using CapRover?