wayne
wayne7mo ago

What's the cold start of python scripts in EE

Because in non EE it can be very slow. Asking because litellm is a fantastic python package and I think I will need to go enterprise for that
3 Replies
rubenf
rubenf7mo ago
Cold start has a few parts but are you seeing it to be slow on a worker with a hot cache as well ?
wayne
wayne7mo ago
My bad: I was testing after modifying the script (in editor mode) so of course it couldn't be cached. I will update you if it doesn't cache when running on prod. and what's the benefit of the distributed depency caching then?
No description
rubenf
rubenf7mo ago
if you're using docker compose, you are using a locally distributed cache using volumes. For larger scale setups that can't be used and that distributed cache is backed by S3 in EE So like a lot of things in EE, it only matters for prod setup at larger scale