andnessA
Windmill•3y ago•
1 reply
andness

Hi, I'm trying out Windmill for

Hi, I'm trying out Windmill for orchestrating our analytics workflows, and as part of that I tried porting some existing code. The code interfaces with Clickhouse, so I have a small shared component for running Clickhouse queries. For simplicity say my 
script
 module depends the 
shared
 module. I got this working fine, but then I was going to write 
script2
 which also uses the 
shared
 module, and as part of that I added a dependency to 
shared
.

After doing that I noticed that the schedule that is invoking 
script
 is failing with the error "ModuleNotFoundError: No module named 'clickhouse_driver'". "clichouse_driver" is the module I added to 
shared
.

To me it appears that the automatic module installation doesn't notice that the upstream 
shared
 dependency has had a dependency change so when it attempt to run 
script
 it doesn't run pip install and as a result this fails.

To check this theory I made a spurious change to 
script
 and saved it again, and that fixed it.

I'm not sure I like this automagic dependency management, I think I would prefer to just manage a requirements.txt file like I'm used to and I don't consider that a hassle at all. But the issue I ran into here is a showstopper really as it would cause all scripts to break every time a dependency is added to one of the shared modules.
Was this page helpful?