can I somehow do something like that? It doesn't seem to work. @rubenf the use case is that users

can I somehow do something like that? It doesn't seem to work.
@rubenf

the use case is that users each have their own secret.

Wwayne can I somehow do something like that? It doesn't seem to work. @rubenf the use...

rubenf•6/6/24, 6:52 AM

Input transform don't support env variable, do that fetch in a separate step

Cctx for some reason, the scheduledFor when calling a workflow or script won't work. ...

rubenf•6/6/24, 7:29 AM

it should work so you might be doing something wrong.

Alper•6/6/24, 8:05 AM

any recommendations for python scripts throwing "RuntimeError: can't start new thread"? i thought about reducing the number of workers on each server (some have 8). but maybe there is a better way like increasing limits?

D3SL•6/6/24, 11:31 AM

I'm trying to understand the pricing for self-hosting. For example would 2 workers with 1 cpu each running for 1460 combined hours and 1 worker with 4 cpus that ran for 365 hours both equal $120 a month, since 1460/2 and 365*4 both equal the ~730 hours in a month?

rubenf•6/6/24, 11:38 AM

@D3SL we bill per provisioned vCPU (limit on helm/docker-compose) and we're not granular enough to adapt to auto-scaling (coming soon) but for now if you tell us that the telemetry data is incorrect because you auto-scaled up and down and we will trust that your vCPU count is correct and do a pro-rata of it being privisionned or not (if node is up only 1/4 of the month, it counts as 1/4 vCPU)

D3SL•6/6/24, 11:43 AM

So basically for now the billing system severely penalizes parallelizable workloads since it would see that I have workers with many cores, not that they're only actually running a fraction of the month?

D3SL•6/6/24, 11:45 AM

And by adapting to autoscaling I'm assuming that's what you're referring to, being able to track actual vcpu-hours as opposed to just what's in the compose.

rubenf•6/6/24, 11:47 AM

@D3SL I'm not saying that, I'm saying we will actually count vCPU-mins but our automated system is not fully ready so we will trust the vCPU numbers you give us until we deploy the new algorithm (ETA 1/2 weeks)

D3SL•6/6/24, 11:48 AM

Okay I misunderstood what you meant then. In either case it sounds like the whole issue is moot quite soon.

rubenf•6/6/24, 11:49 AM

exactly

D3SL•6/6/24, 11:52 AM

Spiffy, thanks for trying to explain it at least. I've only in the last month or two found out basically the entire field of "data engineering" exists. This is a major step up from bash aliases and cron jobs running R scripts.

rubenf•6/6/24, 11:55 AM

And we're only getting started, we're happy about where windmill is as a workflow engine, we have quite a bit of improvement to do for it to be a data pipeline engine (like giving you a pre-defined structure for incremental pipelines, maintaining column-wise lineage, etc)

D3SL•6/6/24, 12:40 PM

Don't sell yourself short. I've tried this, mage, reactive apps, DAG based R packages, and cronjobs. The only real hiccup I ran into was with the automatic serialization.

Kelora•6/6/24, 5:01 PM

does windmill have hard coded fixes for various automatic python imports? I've found a few that don't work out of the box and was wondering if I should report them to be fixed

rubenf•6/6/24, 5:02 PM

better, you can contribute them directly: https://github.com/windmill-labs/windmill/blob/main/backend/parsers/windmill-parser-py-imports/src/lib.rs#L57

GitHub

windmill/backend/parsers/windmill-parser-py-imports/src/lib.rs at m...

Open-source developer platform to turn scripts into workflows and UIs. Fastest workflow engine (5x vs Airflow). Open-source alternative to Airplane and Retool. - windmill-labs/windmill

Kelora•6/6/24, 5:04 PM

oh right on, what are your thoughts on post-install content that is required? Here is my workaround

import asyncio
import os
from playwright.async_api import async_playwright
from PIL import Image, ImageFont, ImageDraw
import base64
from io import BytesIO
import datetime
import logging
import subprocess
import sys


import cv2
import numpy as np
from PIL import Image


def install_playwright():
    try:
        subprocess.check_call([sys.executable, "-m", "pip", "install", "playwright"])
        subprocess.check_call([sys.executable, "-m", "pip", "install", "azure-storage-blob"])
        subprocess.check_call([sys.executable, "-m", "pip", "install", "azure-core"])
        subprocess.check_call([sys.executable, "-m", "playwright", "install"])
        subprocess.check_call([sys.executable, "-m", "playwright", "install-deps"])
        print("Playwright installed successfully.")
    except subprocess.CalledProcessError as e:
        print(f"An error occurred while installing Playwright: {e}")


install_playwright()

import asyncio
import os
from playwright.async_api import async_playwright
from PIL import Image, ImageFont, ImageDraw
import base64
from io import BytesIO
import datetime
import logging
import subprocess
import sys


import cv2
import numpy as np
from PIL import Image


def install_playwright():
    try:
        subprocess.check_call([sys.executable, "-m", "pip", "install", "playwright"])
        subprocess.check_call([sys.executable, "-m", "pip", "install", "azure-storage-blob"])
        subprocess.check_call([sys.executable, "-m", "pip", "install", "azure-core"])
        subprocess.check_call([sys.executable, "-m", "playwright", "install"])
        subprocess.check_call([sys.executable, "-m", "playwright", "install-deps"])
        print("Playwright installed successfully.")
    except subprocess.CalledProcessError as e:
        print(f"An error occurred while installing Playwright: {e}")


install_playwright()

Kelora•6/6/24, 5:05 PM

playwrite needs browsers installed after the pip install or else it can't really do anything

rubenf•6/6/24, 5:05 PM

you should use init scripts for that

Kelora•6/6/24, 5:06 PM

ok gotcha, ill just look into fixing the azure ones then, those are direct swaps but I think it is a many to one relationship which might not be as easy as the one to one relationships in the code you linked

Alper•6/6/24, 5:47 PM

@Kelora i am also using playwright in my scripts. I solved it by creating a custom worker image with preinstalled browser and dependencies

Kelora•6/6/24, 5:49 PM

i thought about doing that, but i have a working setup now, just wanted to see what best practice was

Kelora•6/6/24, 5:50 PM

and i installed with defaults which disables docker and didnt want to jump into that if i didnt have to

Ccassel someone could help me

antonioalves•6/6/24, 9:58 PM

Change the workspace to "Admins" and run the "Sync resources" script

Ccassel I run but didn't do anything 😦

antonioalves•6/6/24, 10:22 PM

do you want to use oauth to log in windmill?

antonioalves•6/6/24, 10:23 PM

antonioalves•6/6/24, 10:24 PM

it's not showing that for you?

rubenf•6/6/24, 10:25 PM

Instance Settings -> OAuth/SSO -> OAuth -> Gcal

South Summit•6/7/24, 5:43 PM

Where can I see what features are available with the Pro plan? The pricing page only says it includes a "subset" of the EE features but then the pro plan is missing entirely from the feature matrix? Also, I would have prefered to ask this over email. Do you really not have an email address (or web form) for sales question anywhere on your website? I can understand support being over discord (even though I hate it) but sales questions? And no, I don't have my email client set up to launch from the "Get in touch" button since I use webmail.

rubenf•6/7/24, 5:58 PM

ruben@windmill.dev / contact@windmill.dev. Pro is reserved to SMB, doing less than 250k in revenue

Kelora•6/7/24, 6:27 PM

https://www.windmill.dev/docs/core_concepts/rich_display_rendering

Does the type "file" only work if the result is not detected to be large?

Rich Display Rendering | Windmill

The result renderer in Windmill supports rich display rendering, allowing you to customize the display format of your results. By leveraging specific formats, you can display images, files, tables, HTML, JSON, and more.

Kelora•6/7/24, 6:28 PM

seems like no matter what i try i get a json download

CCraig How do I collect the results of a `For loop` and save them to a file that I can ...

Kelora•6/7/24, 6:29 PM

can you wrap the code in a try except where the except is to return the recordno and more?

South Summit•6/7/24, 6:31 PM

Hi ruben, I understand that, thank you. The question is about what features are included. What is the "subset" of EE features available on the "Pro" plan?

rubenf•6/7/24, 6:52 PM

@South Summit it's all of EE except the S3 integration (not sure how partial it is) and audit logs pretty much

rubenf•6/7/24, 6:53 PM

For S3 integration, it's mostly about limiting the distributed cache, everything else should be there and if it isn't, we will change it in the future

South Summit•6/7/24, 6:53 PM

Thanks @rubenf I'm looking forward to evaluating the product. It might be perfect for my use case.

KKelora https://www.windmill.dev/docs/core_concepts/rich_display_rendering Does the typ...

rubenf•6/7/24, 6:55 PM

It's possible that we should have edge-cased that but didn't. There are 2 kind of big. Too big for frontend, and too big to be included in jobs response (in favor of get result which can does things like streaming directly from db which json serialization of the job doesn't allow). The too big for frontend shouldn't apply to files since there is nothing to display so if that's not the case, it's a fix for us to do