bluecoconut•2y ago

Installing Python Package from Github URL

I am trying to install a python package from a Github URL, and it almost works. This path works with pip install and requirements.txt with local interpeters, but with the windmill workers it seems to not install the final package? Eg.

# requirements:
# git+https://github.com/approximatelabs/sketch.git@main
import sketch
def main():
    print(dir(sketch))

# requirements:
# git+https://github.com/approximatelabs/sketch.git@main
import sketch
def main():
    print(dir(sketch))

This errors:

....
  Created wheel for sketch: filename=sketch-0.4.2-py3-none-any.whl size=16979 sha256=32c98391433c4655602119c9a851dfbcee94b3c65322c553721063da387d6029
  Stored in directory: /tmp/pip-ephem-wheel-cache-t86gklto/wheels/69/5e/88/aa45af04b5d8c8f01f38ba0a292462960c873640f7415da33e
Successfully built sketch
Installing collected packages: sketch
Successfully installed sketch-0.4.2
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
--- PIP INSTALL ---
...{skipped}...
--- PYTHON CODE EXECUTION ---

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/tmp/windmill/wk-e779c0f6b32d-q67zX/01892efe-eb8b-4898-6006-b6c65cbd3e54/wrapper.py", line 8, in <module>
    from u.justin import github_python_test as inner_script
  File "/tmp/windmill/wk-e779c0f6b32d-q67zX/01892efe-eb8b-4898-6006-b6c65cbd3e54/u/justin/github_python_test.py", line 4, in <module>
    import sketch
ModuleNotFoundError: No module named 'sketch'

....
  Created wheel for sketch: filename=sketch-0.4.2-py3-none-any.whl size=16979 sha256=32c98391433c4655602119c9a851dfbcee94b3c65322c553721063da387d6029
  Stored in directory: /tmp/pip-ephem-wheel-cache-t86gklto/wheels/69/5e/88/aa45af04b5d8c8f01f38ba0a292462960c873640f7415da33e
Successfully built sketch
Installing collected packages: sketch
Successfully installed sketch-0.4.2
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
--- PIP INSTALL ---
...{skipped}...
--- PYTHON CODE EXECUTION ---

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/tmp/windmill/wk-e779c0f6b32d-q67zX/01892efe-eb8b-4898-6006-b6c65cbd3e54/wrapper.py", line 8, in <module>
    from u.justin import github_python_test as inner_script
  File "/tmp/windmill/wk-e779c0f6b32d-q67zX/01892efe-eb8b-4898-6006-b6c65cbd3e54/u/justin/github_python_test.py", line 4, in <module>
    import sketch
ModuleNotFoundError: No module named 'sketch'

My goal is to actually update to something like

git+https://{GITHUB_PAT}@github.com/approximatelabs/sketch.git@{COMMIT_SHA}

git+https://{GITHUB_PAT}@github.com/approximatelabs/sketch.git@{COMMIT_SHA}

For introducing a bunch of private dependencies and scripts at once by installing them all as a package + getting the ability to use the scripts. I can raise this as an issue on Github as well, just figured i'd post in forum for now?

1 Reply

bluecoconutOP•2y ago

Made an issue -> https://github.com/windmill-labs/windmill/issues/1804

GitHub

bug: Python - Installing requirement based package from Github URL ...

Describe the bug I am trying to install a python package from a Github URL, and it almost works. This path works with pip install and requirements.txt with local interpeters, but with the windmill ...