Bjarne•2w ago

Python requirement issues

Hey, I am trying to write a script that makes use of the azure opentelemetry sdk. When I run the script below locally, it just works. But when running in windmill. I get this error: ModuleNotFoundError: No module named 'azure.monitor.opentelemetry' The script:

#requirements:
#azure-monitor-events-extension
#azure-monitor-opentelemetry

import logging
import os

from azure.monitor.events.extension import track_event
from azure.monitor.opentelemetry import configure_azure_monitor

# Configure the connection to the application insights instance
configure_azure_monitor(connection_string=wmill.get_variable("f/playground/appi_connection_string"))

def main(application_name: str) -> None:
    # Name indicates PROD / DEV split
    event_name = f"Running flow xyz"

    # Send the event to app insights
    track_event(event_name)

#requirements:
#azure-monitor-events-extension
#azure-monitor-opentelemetry

import logging
import os

from azure.monitor.events.extension import track_event
from azure.monitor.opentelemetry import configure_azure_monitor

# Configure the connection to the application insights instance
configure_azure_monitor(connection_string=wmill.get_variable("f/playground/appi_connection_string"))

def main(application_name: str) -> None:
    # Name indicates PROD / DEV split
    event_name = f"Running flow xyz"

    # Send the event to app insights
    track_event(event_name)

I don't understand what I am doing wrong. Am I missing an important limitation of the windmill python requirements system?

14 Replies

rubenf•2w ago

I will investigate and let you know If you don't mind us asking @Bjarne , are you EE ? We need to assign priority on this investigation

BjarneOP•2w ago

Yes, we are using EE

rubenf•2w ago

@Pyra can you prioritize this today pl

Pyra•2w ago

👍

rubenf•2w ago

@Bjarne We are still working on it but will have a fix for tomorrow: https://github.com/windmill-labs/windmill/pull/4793/files

GitHub

fix(python): Merge to site-packages packages with same name by pyra...

Iterate overall all python paths and if same folder has same name multiple times, then merge the content and put to /site-packages

BjarneOP•2w ago

@rubenf @Pyra Any updates on this one?

Pyra•2w ago

Yes, we are finishing testing soon It should be done today

fooosieee•2w ago

use extra_requirements

BjarneOP•2w ago

Hey @rubenf @Pyra, The fix seems to work now, but when I import this file in other scripts, I get errors, as pinning the requirements seems to require me to also pin the requirements in all other scripts. As @fooosieee suggested, it would be way nicer if I could use the extra_requirements methods, but when I do this, Windmill automatically detects azure as something which needs to be pip installed, and then crashes on this. Is there any way for the windmill requirements inference to ignore this? Or handle this correctly?

rubenf•2w ago

it's a tough one tbh. We probably need a per import annotation #wm_ignore @Pyra wdyt?

BjarneOP•2w ago

Ow sorry, nevermind, don't know what changed, but my statement about all files that use the script also needing pinned requirements seems to be false. Just imported it somewhere, and ran perfectly Although being able to just use extra_requirements would be nice, it is not super high priority if it works by just pinning the requirements in my one small script and then importing this everywhere where it is needed

Pyra•2w ago

I think we can make top-level scripts automatically inherit all pinned dependencies from referenced scripts. This way if dependency pinned in one script, all scripts referencing this one will also respect the pin. I will allocate some time investigating this and edge cases. Way with wm_ignore can also work

rubenf•2w ago

@Pyra

I think we can make top-level scripts automatically inherit all pinned dependencies from referenced scripts.

That's already the case

Pyra•2w ago

Ok, then i misunderstood the problem. wm_ignore should be the right solution.