sigcore•11mo ago

Internal server error when using ai code generator

I tried using the ai generator in the runnable editor and get this error: Failed to generate code: Internal Server Error In the logs, it says: ERROR request: windmill_common::error: error="Internal: Embeddings db not initialized" Is there some sort of embeddings db I need to set up? I might've missed it in the docs, but I didn't see it mentioned specifically on https://www.windmill.dev/docs/core_concepts/ai_generation either

Windmill AI | Windmill

Windmill provides ways to have AI help you in your coding experience.

12 Replies

rubenf•11mo ago

Hi @sigcore , the server fetches embedding from s3 or the hub directly. Are you server hosted with a slow connection or air-gapped ?

sigcore•11mo ago

Hmm it shouldn't be slow, it's on a colo server with a 1gbit link Does it download something during startup or on-demand? I can try restarting and look for errors then

rubenf•11mo ago

yes it downloads them at start of the server

rubenf•11mo ago

rubenf•11mo ago

sigcore•11mo ago

Ah it does fail:

2023-12-15T00:26:58.290540Z  INFO windmill_api::embeddings: Loading embedding model...
2023-12-15T00:26:58.383957Z  INFO windmill_api: server started on port=8000 and addr=0.0.0.0 instance=Wp0Ov
2023-12-15T00:26:58.384002Z ERROR windmill: Metrics are only available in the EE, ignoring...
2023-12-15T00:26:58.399528Z  INFO windmill: Successfully connected to pg listen
2023-12-15T00:26:58.492478Z ERROR windmill_api::embeddings: Failed to initialize model instance: could not get config.json

2023-12-15T00:26:58.290540Z  INFO windmill_api::embeddings: Loading embedding model...
2023-12-15T00:26:58.383957Z  INFO windmill_api: server started on port=8000 and addr=0.0.0.0 instance=Wp0Ov
2023-12-15T00:26:58.384002Z ERROR windmill: Metrics are only available in the EE, ignoring...
2023-12-15T00:26:58.399528Z  INFO windmill: Successfully connected to pg listen
2023-12-15T00:26:58.492478Z ERROR windmill_api::embeddings: Failed to initialize model instance: could not get config.json

rubenf•11mo ago

we need to improve the error message but it seems you're unable to fetch the model from api for some reasons it would be downloaded from hugging face

sigcore•11mo ago

Do you know what the url would be or where I can find it? I don't think huggingface.co is blocked it fails very quickly though so I don't think it's dying in the middle of a download

rubenf•11mo ago

I will do a patch to show a better error on very latest available in 15mins, the error message will have more info

sigcore•11mo ago

Thanks, I'll try it in a few minutes That was fast, thanks! Looks like a permission error, would it be downloading to the current directory or something like /tmp? It is running as non-root

2023-12-15T00:56:25.378662Z ERROR windmill_api::embeddings: Failed to get config.json from hugging face: I/O error Permission denied (os error 13)
2023-12-15T00:56:25.378818Z ERROR windmill_api::embeddings: Failed to initialize model instance: could not get config.json

2023-12-15T00:56:25.378662Z ERROR windmill_api::embeddings: Failed to get config.json from hugging face: I/O error Permission denied (os error 13)
2023-12-15T00:56:25.378818Z ERROR windmill_api::embeddings: Failed to initialize model instance: could not get config.json

rubenf•11mo ago

impl Default for Cache {
    fn default() -> Self {
        let mut path = match std::env::var("HF_HOME") {
            Ok(home) => home.into(),
            Err(_) => {
                let mut cache = dirs::home_dir().expect("Cache directory cannot be found");
                cache.push(".cache");
                cache.push("huggingface");
                cache
            }
        };
        path.push("hub");
        Self::new(path)
    }
}

impl Default for Cache {
    fn default() -> Self {
        let mut path = match std::env::var("HF_HOME") {
            Ok(home) => home.into(),
            Err(_) => {
                let mut cache = dirs::home_dir().expect("Cache directory cannot be found");
                cache.push(".cache");
                cache.push("huggingface");
                cache
            }
        };
        path.push("hub");
        Self::new(path)
    }
}

you can tweak it by modifying HF_HOME

sigcore•11mo ago

awesome, that fixed it, thanks @rubenf . It downloaded the model and the ai thing in the editor works too