Bjarne
Bjarne2w ago

Python requirement issues

Hey, I am trying to write a script that makes use of the azure opentelemetry sdk. When I run the script below locally, it just works. But when running in windmill. I get this error: ModuleNotFoundError: No module named 'azure.monitor.opentelemetry' The script:
#requirements:
#azure-monitor-events-extension
#azure-monitor-opentelemetry

import logging
import os

from azure.monitor.events.extension import track_event
from azure.monitor.opentelemetry import configure_azure_monitor

# Configure the connection to the application insights instance
configure_azure_monitor(connection_string=wmill.get_variable("f/playground/appi_connection_string"))

def main(application_name: str) -> None:
# Name indicates PROD / DEV split
event_name = f"Running flow xyz"

# Send the event to app insights
track_event(event_name)
#requirements:
#azure-monitor-events-extension
#azure-monitor-opentelemetry

import logging
import os

from azure.monitor.events.extension import track_event
from azure.monitor.opentelemetry import configure_azure_monitor

# Configure the connection to the application insights instance
configure_azure_monitor(connection_string=wmill.get_variable("f/playground/appi_connection_string"))

def main(application_name: str) -> None:
# Name indicates PROD / DEV split
event_name = f"Running flow xyz"

# Send the event to app insights
track_event(event_name)
I don't understand what I am doing wrong. Am I missing an important limitation of the windmill python requirements system?
14 Replies
rubenf
rubenf2w ago
I will investigate and let you know If you don't mind us asking @Bjarne , are you EE ? We need to assign priority on this investigation
Bjarne
BjarneOP2w ago
Yes, we are using EE
rubenf
rubenf2w ago
@Pyra can you prioritize this today pl
Pyra
Pyra2w ago
👍
rubenf
rubenf2w ago
@Bjarne We are still working on it but will have a fix for tomorrow: https://github.com/windmill-labs/windmill/pull/4793/files
GitHub
fix(python): Merge to site-packages packages with same name by pyra...
Iterate overall all python paths and if same folder has same name multiple times, then merge the content and put to /site-packages
Bjarne
BjarneOP2w ago
@rubenf @Pyra Any updates on this one?
Pyra
Pyra2w ago
Yes, we are finishing testing soon It should be done today
fooosieee
fooosieee2w ago
use extra_requirements
Bjarne
BjarneOP2w ago
Hey @rubenf @Pyra, The fix seems to work now, but when I import this file in other scripts, I get errors, as pinning the requirements seems to require me to also pin the requirements in all other scripts. As @fooosieee suggested, it would be way nicer if I could use the extra_requirements methods, but when I do this, Windmill automatically detects azure as something which needs to be pip installed, and then crashes on this. Is there any way for the windmill requirements inference to ignore this? Or handle this correctly?
rubenf
rubenf2w ago
it's a tough one tbh. We probably need a per import annotation #wm_ignore @Pyra wdyt?
Bjarne
BjarneOP2w ago
Ow sorry, nevermind, don't know what changed, but my statement about all files that use the script also needing pinned requirements seems to be false. Just imported it somewhere, and ran perfectly Although being able to just use extra_requirements would be nice, it is not super high priority if it works by just pinning the requirements in my one small script and then importing this everywhere where it is needed
Pyra
Pyra2w ago
I think we can make top-level scripts automatically inherit all pinned dependencies from referenced scripts. This way if dependency pinned in one script, all scripts referencing this one will also respect the pin. I will allocate some time investigating this and edge cases. Way with wm_ignore can also work
rubenf
rubenf2w ago
@Pyra
I think we can make top-level scripts automatically inherit all pinned dependencies from referenced scripts.
That's already the case
Pyra
Pyra2w ago
Ok, then i misunderstood the problem. wm_ignore should be the right solution.