WindmillWWindmill
Powered by
fr3fouF
Windmill•16mo ago•
1 reply
fr3fou

Relative imported module doesn't respect optional/non-imported Python dependencies

I have the following file in
f/main/requirements.txt
f/main/requirements.txt
:
duckdb==1.1.2
polars[pyarrow]==1.12.0
duckdb==1.1.2
polars[pyarrow]==1.12.0

I also have a script called
f/main/get_db
f/main/get_db
, which only imports
duckdb
duckdb
:
from typing import TypedDict
import duckdb
import textwrap
...
from typing import TypedDict
import duckdb
import textwrap
...

Running
wmill script generate-metadata
wmill script generate-metadata
produces a script.lockfile for the
get_db
get_db
script correctly:
duckdb==1.1.2
polars==1.12.0
pyarrow==18.0.0
duckdb==1.1.2
polars==1.12.0
pyarrow==18.0.0

(the pyarrow & polars dependencies are required if you want to convert duckdb results to a polars dataframe)
In my flow,
f/main/my_flow
f/main/my_flow
, inside an inline script, I import the
get_db
get_db
module:
from typing import TypedDict
from f.main.get_db import get_db
from typing import TypedDict
from f.main.get_db import get_db

The generated lockfile, however, doesn't include the optional polars & pyarrow dependencies:
duckdb==1.1.2
duckdb==1.1.2

I've tried running
wmill script generate-metadata
wmill script generate-metadata
and
wmill flow generate-locks
wmill flow generate-locks
, but none of them seem to produce the desired output. A workaround is for me to include the pyarrow / polars dependencies in a comment in my inline script (under a
extra_requirements
extra_requirements
section), but this means that I have to pin 2 versions separately which is undesired.
WindmillJoin
3,362Members
Resources
Recent Announcements

Similar Threads

Was this page helpful?
Recent Announcements
henri-c

Weekly kenote to tell you about our latest updates https://discord.com/channels/930051556043276338/1278977038430240813 https://youtube.com/live/2dGd9TdT8xs?feature=share

henri-c · 4d ago

Pyra

### HTTP tracing (EE) Capture HTTP requests made by job scripts as observability spans Features: - View HTTP request traces (method, URL, status, timing) in the job details UI - Auto-instrumentation for Native TypeScript, MITM proxy for other languages - Integrates with external OpenTelemetry collectors changelog: https://www.windmill.dev/changelog/http-tracing docs: https://www.windmill.dev/docs/advanced/instance_settings#http-tracing Additionally jobs memory metrics are now fully OSS!

Pyra · 2w ago

henri-c

First keynote of the year here https://discord.com/channels/930051556043276338/1278977038430240813 🙂

henri-c · 4w ago

Similar Threads

python dependencies version
FanFFan / help
2y ago
Install Python module from GitHub
MarkyMMarky / help
2y ago
Sub module not working with Python
MarkyMMarky / help
2y ago
[python] subdependencies breaking wmill module installation
developerDdeveloper / help
2y ago