fr3fou
fr3fou•13mo ago

Relative imported module doesn't respect optional/non-imported Python dependencies

I have the following file in f/main/requirements.txt : 
duckdb==1.1.2
polars[pyarrow]==1.12.0
duckdb==1.1.2
polars[pyarrow]==1.12.0
I also have a script called f/main/get_db, which only imports duckdb: 
from typing import TypedDict
import duckdb
import textwrap
...
from typing import TypedDict
import duckdb
import textwrap
...
Running wmill script generate-metadata produces a script.lockfile for the get_db script correctly: 
duckdb==1.1.2
polars==1.12.0
pyarrow==18.0.0
duckdb==1.1.2
polars==1.12.0
pyarrow==18.0.0
(the pyarrow & polars dependencies are required if you want to convert duckdb results to a polars dataframe) In my flow, f/main/my_flow, inside an inline script, I import the get_db module: 
from typing import TypedDict
from f.main.get_db import get_db
from typing import TypedDict
from f.main.get_db import get_db
The generated lockfile, however, doesn't include the optional polars & pyarrow dependencies: 
duckdb==1.1.2
duckdb==1.1.2
I've tried running wmill script generate-metadata and wmill flow generate-locks, but none of them seem to produce the desired output. A workaround is for me to include the pyarrow / polars dependencies in a comment in my inline script (under a extra_requirements section), but this means that I have to pin 2 versions separately which is undesired.
1 Reply
Linear
Linear•13mo ago

Did you find this page helpful?