djm Comments - Windmill

djm

•Created by Ross Creighton on 2/19/2024 in #help

Python depencies - install from github repo

we do have one but with PIP_EXTRA_INDEX_URL we were getting a very strange issue where it would try and install our repo URL as a package, that is the reproduction i am talking about above

24 replies

WWindmill

•Created by Ross Creighton on 2/19/2024 in #help

Python depencies - install from github repo

Thanks, will be in touch re: licensing 🙂

24 replies

WWindmill

•Created by Ross Creighton on 2/19/2024 in #help

Python depencies - install from github repo

hmm well it works on one, you mean try with ~/.ssh?

24 replies

WWindmill

•Created by Ross Creighton on 2/19/2024 in #help

Python depencies - install from github repo

Thanks for getting back to me, I was looking at the EE pricing today with my teammate, I asked you previously about Non Profits but the pricing page doesn't seem clear. Does Pro(NfP) = EE? I have the ssh-keyscan command in step 2 above, already in init script. For some reason it works on the first worker, but as soon as the script is allocated to another worker it fails. Re-running incessantly until the original worker is chosen again will allow it to work successfully

24 replies

WWindmill

•Created by Ross Creighton on 2/19/2024 in #help

Python depencies - install from github repo

This seems to have issues when re-using the cached resolution on another worker; I would imagine INIT_SCRIPT runs on both workers so I'm not sure why the host key would fail to verify

Collecting canteen-common@ git+ssh://git@github.com/Canteen-Australia/canteen-common-py@v0.5.3
  Cloning ssh://****@github.com/Canteen-Australia/canteen-common-py (to revision v0.5.3) to /tmp/pip-install-dzscmz_k/canteen-common_eccee4f9a9ad4f719bf9ecf23204270d
  Running command git clone --filter=blob:none --quiet 'ssh://****@github.com/Canteen-Australia/canteen-common-py' /tmp/pip-install-dzscmz_k/canteen-common_eccee4f9a9ad4f719bf9ecf23204270d

  Host key verification failed.

Collecting canteen-common@ git+ssh://git@github.com/Canteen-Australia/canteen-common-py@v0.5.3
  Cloning ssh://****@github.com/Canteen-Australia/canteen-common-py (to revision v0.5.3) to /tmp/pip-install-dzscmz_k/canteen-common_eccee4f9a9ad4f719bf9ecf23204270d
  Running command git clone --filter=blob:none --quiet 'ssh://****@github.com/Canteen-Australia/canteen-common-py' /tmp/pip-install-dzscmz_k/canteen-common_eccee4f9a9ad4f719bf9ecf23204270d

  Host key verification failed.

Scaling to just one worker fixes this; also had to run delete from pip_resolution_cache ; on the postgres container

24 replies

WWindmill

•Created by Ross Creighton on 2/19/2024 in #help

Python depencies - install from github repo

I spent the last few days looking at this for our private pypi package: I was not able to get it working with PIP_EXTRA_INDEX_URL, might need your help with that one @rubenf . I will look at a minimum reproducible example soon. I was able to get it working in our Self-Hosted environment like so: * use #requirements: to specify the exact Git SSH URL with the version tagged using @:

#requirements:
#git+ssh://git@github.com/Canteen-Australia/canteen-common-py@v0.5.3

from canteen.dbricks import DBricks

def main(
    notebook_path: str,
):
    DBricks.start_notebook(notebook_path)
    print(f"Starting {notebook_path}")

#requirements:
#git+ssh://git@github.com/Canteen-Australia/canteen-common-py@v0.5.3

from canteen.dbricks import DBricks

def main(
    notebook_path: str,
):
    DBricks.start_notebook(notebook_path)
    print(f"Starting {notebook_path}")

* Add Init Script to worker which adds github.com to known_hosts for SSH:

touch /root/.ssh/known_hosts
ssh-keyscan -t rsa github.com >> /root/.ssh/known_hosts

touch /root/.ssh/known_hosts
ssh-keyscan -t rsa github.com >> /root/.ssh/known_hosts

* Mount on the workers the private SSH key authorised to access the Git repo:

windmill_worker:
...
  volumes:
...
      # mount the GitHub SSH Identity
      - ./gh_identity:/root/.ssh/id_rsa

windmill_worker:
...
  volumes:
...
      # mount the GitHub SSH Identity
      - ./gh_identity:/root/.ssh/id_rsa

24 replies

WWindmill

•Created by rubenf on 9/8/2023 in #general

Package.json for Bun action

Works fine via Script Editor test button, but on Deploy runs into that error

8 replies

WWindmill

•Created by rubenf on 9/8/2023 in #general

Package.json for Bun action

I ran into this today after updating (was working before update), I saw my Bun code had a mix of import and require, but changing all to import like so gets the same error:

import * as wmill from "windmill-client";

// const { DataFactoryManagementClient } = require("@azure/arm-datafactory");
// const { DefaultAzureCredential } = require("@azure/identity");
import {DataFactoryManagementClient} from "@azure/arm-datafactory";
import {DefaultAzureCredential} from "@azure/identity";

import { subDays } from 'date-fns';

import * as wmill from "windmill-client";

// const { DataFactoryManagementClient } = require("@azure/arm-datafactory");
// const { DefaultAzureCredential } = require("@azure/identity");
import {DataFactoryManagementClient} from "@azure/arm-datafactory";
import {DefaultAzureCredential} from "@azure/identity";

import { subDays } from 'date-fns';

This script has not been deployed successfully because of the following errors:

job 018a872b-c134-0491-9185-d003ec97463b on worker wk-c3b14eb0bd72-U3QWv (tag: bun)

100 |         ranges[_key2] = arguments[_key2];
101 |     }
102 | 
103 |     return ranges.reduce(function (result, range) {
104 |         var validRange = semver.validRange(range);
105 |         var validRanges = validRange.split(regex.whitespace);
                              ^
TypeError: null is not an object (evaluating 'validRange.split')
      at /tmp/windmill/cache/bun/semver-intersect@1.4.0/semver-intersect.js:105:26
      at reduce (:1:20)
      at expandRanges (/tmp/windmill/cache/bun/semver-intersect@1.4.0/semver-intersect.js:103:11)
      at intersect (/tmp/windmill/cache/bun/semver-intersect@1.4.0/semver-intersect.js:128:13)
      at reduce (:1:20)
      at /tmp/windmill/wk-c3b14eb0bd72-U3QWv/018a872b-c134-0491-9185-d003ec97463b/build.ts:84:10
Non-zero exit status: 1

This script has not been deployed successfully because of the following errors:

job 018a872b-c134-0491-9185-d003ec97463b on worker wk-c3b14eb0bd72-U3QWv (tag: bun)

100 |         ranges[_key2] = arguments[_key2];
101 |     }
102 | 
103 |     return ranges.reduce(function (result, range) {
104 |         var validRange = semver.validRange(range);
105 |         var validRanges = validRange.split(regex.whitespace);
                              ^
TypeError: null is not an object (evaluating 'validRange.split')
      at /tmp/windmill/cache/bun/semver-intersect@1.4.0/semver-intersect.js:105:26
      at reduce (:1:20)
      at expandRanges (/tmp/windmill/cache/bun/semver-intersect@1.4.0/semver-intersect.js:103:11)
      at intersect (/tmp/windmill/cache/bun/semver-intersect@1.4.0/semver-intersect.js:128:13)
      at reduce (:1:20)
      at /tmp/windmill/wk-c3b14eb0bd72-U3QWv/018a872b-c134-0491-9185-d003ec97463b/build.ts:84:10
Non-zero exit status: 1

8 replies