What's the cold start of python scripts in EE
Because in non EE it can be very slow.
Asking because litellm is a fantastic python package and I think I will need to go enterprise for that
3 Replies
Cold start has a few parts but are you seeing it to be slow on a worker with a hot cache as well ?
My bad: I was testing after modifying the script (in editor mode) so of course it couldn't be cached.
I will update you if it doesn't cache when running on prod.
and what's the benefit of the distributed depency caching then?
if you're using docker compose, you are using a locally distributed cache using volumes. For larger scale setups that can't be used and that distributed cache is backed by S3 in EE
So like a lot of things in EE, it only matters for prod setup at larger scale