20 Replies
If you load the default rust script, and change one letter of a println, and test the script, it compiles from the start
yes that's normal
Typically it does not rebuild all of the crates from the beginning as that takes minutes of compilation time per minor change for any sophisticated project
Locally the same changes im making are taking a few seconds to prepare
yes but locally you have a cache in target that is unique to your project
once rust has an efficient global cache we can use that
but it doesn't
and sscache is a bit complicated for now
we only cache the deployed bin
windmill is not yet the place one should iterate on for rust, only deploy
Imo it severely limits the usefulness of rust scripts, since doesn't make sense to develop in-app, and at that point might as well load and run the binary instead
I agree, it's more of a way to benefit from the other ability of having windmill managing your sources
easy review and diff, hash versioning, integrates with the CLI, etc
loading a bin with init script is way more inconvenient than using our rust integration
rust support is early and it's not a big enough business yet that we can spend too many cycles on this. Once we have enterprises users on it, we may be able to refine it more
Hmm, all I know is I had a lot of use-cases for it which all disappeared due to that small detail. Others might end up with the same problems & not utilize rust as a result. So it might not end up being popular enough to seem worth refining
In my case, I was deploying a finished, already working script, but naturally there was a error caused by the worker's container. Spent ~4 hours making changes, waiting for it to recompile everything and run, making changes, recompile, etc.
Would be more feasible to do all of it locally and make a script for automatically providing binary to workers
I understand the frustration but we do not have the resources to solve that right now, we're happy to take contribution to improve the incremental compilation ability of the workers
usually enterprise customers will let us know first the contract value, then ask us to solve it so it's easier to budget for it
I'm just providing my experience in case it helps
https://github.com/mozilla/sccache this is likely what would need to be used on every worker to solve your pain
GitHub
GitHub - mozilla/sccache: Sccache is a ccache-like tool. It is used...
Sccache is a ccache-like tool. It is used as a compiler wrapper and avoids compilation when possible. Sccache has the capability to utilize caching in remote storage environments, including various...
do workers have a folder that persists throughout jobs?
everything but the job dir persists
you can vote for it with a 👍
@rubenf spent a few hours experimenting and trying to get rust to use sccache, but it seems to disregard env variables, Cargo.toml doesn't seem to be able to override it either
It feels really close
I really think you would have an easier time modifying our sources directly
GitHub
windmill/backend/windmill-worker/src/rust_executor.rs at main · win...
Open-source developer platform to turn scripts into workflows and UIs. Fastest workflow engine (5x vs Airflow). Open-source alternative to Airplane and Retool. - windmill-labs/windmill
Any pointers for where to look? I may give it a h-
Epic
windmill is pretty easy to run from sources