WindmillWWindmill
Powered by
fr3fouF
Windmill•17mo ago•
1 reply
fr3fou

Relative imported module doesn't respect optional/non-imported Python dependencies

I have the following file in
f/main/requirements.txt
f/main/requirements.txt
:
duckdb==1.1.2
polars[pyarrow]==1.12.0
duckdb==1.1.2
polars[pyarrow]==1.12.0

I also have a script called
f/main/get_db
f/main/get_db
, which only imports
duckdb
duckdb
:
from typing import TypedDict
import duckdb
import textwrap
...
from typing import TypedDict
import duckdb
import textwrap
...

Running
wmill script generate-metadata
wmill script generate-metadata
produces a script.lockfile for the
get_db
get_db
script correctly:
duckdb==1.1.2
polars==1.12.0
pyarrow==18.0.0
duckdb==1.1.2
polars==1.12.0
pyarrow==18.0.0

(the pyarrow & polars dependencies are required if you want to convert duckdb results to a polars dataframe)
In my flow,
f/main/my_flow
f/main/my_flow
, inside an inline script, I import the
get_db
get_db
module:
from typing import TypedDict
from f.main.get_db import get_db
from typing import TypedDict
from f.main.get_db import get_db

The generated lockfile, however, doesn't include the optional polars & pyarrow dependencies:
duckdb==1.1.2
duckdb==1.1.2

I've tried running
wmill script generate-metadata
wmill script generate-metadata
and
wmill flow generate-locks
wmill flow generate-locks
, but none of them seem to produce the desired output. A workaround is for me to include the pyarrow / polars dependencies in a comment in my inline script (under a
extra_requirements
extra_requirements
section), but this means that I have to pin 2 versions separately which is undesired.
WindmillJoin
3,388Members
Resources
Recent Announcements

Similar Threads

Was this page helpful?
Recent Announcements
henri-c

We're live here https://discord.com/channels/930051556043276338/1278977038430240813

henri-c · 2d ago

rubenf

Infrastructure as code

rubenf · 3w ago

rubenf

As we're preparing for a new Windmill sandbox SDK, we took the opportunity **to open-source NSJAIL sandboxing** and make it available in CE and easier to set. In 1.634.0, just set nsjail to benefit from the tightest security and isolation per job. It's constraining so no need to use it unless you need but now you won't be restricted in security for CE if you need to.

rubenf · 4w ago

Similar Threads

python dependencies version
FanFFan / help
2y ago
Install Python module from GitHub
MarkyMMarky / help
2y ago
Sub module not working with Python
MarkyMMarky / help
2y ago
[python] subdependencies breaking wmill module installation
developerDdeveloper / help
2y ago