Tiago Serafim•9mo ago

Hi, yes, I was figuring out how I did it

Hi, yes, I was figuring out how I did it. As far as I remember, I installed the tailscale on the host machine, and it worked. As a proof of concept, I run the sshd one click install from Caprover, and on my local machine, I run a lscr.io/linuxserver/openssh-server along with the original docker-compose from Windmill, that way I could run GPU workers on my local machine that connected to the remote WM server.

15 Replies

Unknown User•9mo ago

Message Not Public

Tiago Serafim•9mo ago

I believe you're trying to set up both WM and Tailscale on the same "app" inside Caprover, correct? It's not that way. Windmill should stay as it is, and you should install Tailscale on the host machine At least that how I remember doing

Unknown User•9mo ago

Message Not Public

Tiago Serafim•9mo ago

I remember running tailscale on the host machine, along with Caprover. Inside Caprover, I run sshd app which is a tunnel. Then on my local machine, I run a docker-compose with a WM worker modified to connect to the Postgres instance on the remote machine.

  ssh-tunnel:
    image: lscr.io/linuxserver/openssh-server
    command: >
      /bin/sh -c "apk add --no-cache sshpass postgresql-client &&
                 sshpass -p '${SSH_PASSWORD}' ssh -f -N -L ${TUNNEL_PORT} ${SSH_USER}@${SSH_HOST} -o ExitOnForwardFailure=yes -o StrictHostKeyChecking=no -o ServerAliveInterval=60 -p 4646 && tail -f /dev/null"

    environment:
      - DOCKER_MODS=ghcr.io/tailscale-dev/docker-mod:main
      - TAILSCALE_AUTHKEY=${TAILSCALE_AUTHKEY}
      - TAILSCALE_HOSTNAME=${TAILSCALE_HOSTNAME}
      - TAILSCALE_STATE_DIR=/var/lib/tailscale
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -h localhost"]
      interval: 10s
      timeout: 5s
      retries: 5
    volumes:
      - tailscale:/var/lib/tailscale

  ssh-tunnel:
    image: lscr.io/linuxserver/openssh-server
    command: >
      /bin/sh -c "apk add --no-cache sshpass postgresql-client &&
                 sshpass -p '${SSH_PASSWORD}' ssh -f -N -L ${TUNNEL_PORT} ${SSH_USER}@${SSH_HOST} -o ExitOnForwardFailure=yes -o StrictHostKeyChecking=no -o ServerAliveInterval=60 -p 4646 && tail -f /dev/null"

    environment:
      - DOCKER_MODS=ghcr.io/tailscale-dev/docker-mod:main
      - TAILSCALE_AUTHKEY=${TAILSCALE_AUTHKEY}
      - TAILSCALE_HOSTNAME=${TAILSCALE_HOSTNAME}
      - TAILSCALE_STATE_DIR=/var/lib/tailscale
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -h localhost"]
      interval: 10s
      timeout: 5s
      retries: 5
    volumes:
      - tailscale:/var/lib/tailscale

This was on my local machine, along with a defaul worker (from the original docker-compose WM file), with DATABASE_URL changed:

DATABASE_URL=postgres://postgres:pass@ssh-tunnel/windmill?sslmode=disable

DATABASE_URL=postgres://postgres:pass@ssh-tunnel/windmill?sslmode=disable

Tiago Serafim•9mo ago

The trick was to run this on Caprover https://github.com/caprover/one-click-apps/blob/master/public/v4/apps/sshd.yml in the latest version.

GitHub

one-click-apps/public/v4/apps/sshd.yml at master · caprover/one-cli...

Community Maintained One Click Apps (https://github.com/caprover/caprover) - caprover/one-click-apps

Unknown User•9mo ago

Message Not Public

Tiago Serafim•9mo ago

I'm not sure I follow what set up you want. You have a complete WM set up running somewhere in the cloud and you want to have another worker hosted on you local machine? Windmill workers only need access to the Postgres database in order to work together, so you just have to figure out how to tunnel that connection over tailscale

Unknown User•9mo ago

Message Not Public

Tiago Serafim•9mo ago

Yes, thanks! I'm not sure how to "bundle" both WM and tailscale together. I believe that you'd have to use their official image and run it along with your WM workers, and somehow change the network config of your WM worker/server to be on the same network created by tailscale. But that's way beyond what I have done/researched 6 months ago.

Unknown User•9mo ago

Message Not Public

Tiago Serafim•9mo ago

but that DOCKER_MODS only works on linuxserver.io images, right?

Unknown User•9mo ago

Message Not Public

Tiago Serafim•9mo ago

Really not sure how to do that

Unknown User•9mo ago

Message Not Public

rubenf•9mo ago

I only know how to do this on k8s given it's been requested by our EE customers, sorry