Dev/Staging/Prod

Original message was deleted

rubenf•6/13/23, 7:32 PM

I'm currently working on our story for staging/dev/prod:
There are mainly 3 ways to do this in windmill

draft & deploy
have them be different workspaces (and use cli + .wmillignore to select what to pick and what to ignore)
use different folders

The 2 formers already work well and we just need to document the second much better (for instance, how to keep secrets specific to a given workspace)

I don't find the use a different folder practical today. So my idea which i'm gonna start implementing today is to have a default folder called "prod" in which the script, flow, and apps will have a special and intuitive way of deploying to. Any thoughts about it ?

rubenf•6/13/23, 8:17 PM

There would be a submenu to the deploy button, that would offer to co-deploy to a similarly named path in prod, with some more options for flows (for instance, inlining all the inline scripts, or locking their hashes)

rubenf•6/13/23, 8:35 PM

@Jim would it be an annoyance that everything would now be under one folder, prod ?

rubenf•6/13/23, 8:35 PM

at least the things deployed through this scheme

rubenf•6/13/23, 8:42 PM

you can still use "subfolder"

rubenf•6/13/23, 8:43 PM

u/foo/bar -> f/prod/bar | u/foo/foobar/bar -> f/prod/foobar/bar
f/amazon/bar -> f/prod/amazon/bar

rubenf•6/13/23, 8:43 PM

Also note that this is a co-deployment

rubenf•6/13/23, 8:43 PM

and opt-in, you would have to go through a submenu to see that new deployment page

rubenf•6/13/23, 8:43 PM

if you just click deploy, it will be as before

rubenf•6/13/23, 8:50 PM

EXACTLY

rubenf•6/13/23, 8:50 PM

that's like exactly the thought of having this copy to deploy to

rubenf•6/13/23, 8:50 PM

because drafts don't have webhooks and stuff

rubenf•6/13/23, 8:50 PM

and also you can write tests against staging

rubenf•6/13/23, 8:50 PM

in theory

rubenf•6/13/23, 8:50 PM

it's really how to avoid having a second workspace

rubenf•6/13/23, 8:50 PM

but purely opt-in

rubenf•6/13/23, 8:55 PM

I think we could create also folder staging and have you toggle if one want to deploy to staging

rubenf•6/13/23, 8:55 PM

is that what you mean ?

rubenf•6/13/23, 8:55 PM

I think dev is truly the one we can skip since it's everything else

rubenf•6/13/23, 8:56 PM

but if we do dev/staging/prod then there should be a way to deploy to staging too

rubenf•6/13/23, 8:57 PM

I think what's gonna be tough is what to do about the secrets that would be inlined in the code

rubenf•6/13/23, 8:57 PM

because we cannot really port them

rubenf•6/13/23, 9:06 PM

I think it should probably be
path under prod prod -> deploy
path under staging -> deploy + deploy to prod
any path -> deploy + deploy to staging + deploy to prod

rubenf•6/13/23, 9:13 PM

yes

rubenf•6/13/23, 9:21 PM

you wouldn't edit them from there

rubenf•6/13/23, 9:21 PM

let say you edit a script/flow in u/foo/bar

rubenf•6/13/23, 9:22 PM

you have a submenu to deploy: "Deploy to prod" which show a drawer that has a few options and the destination path which is to override f/prod/bar (assuming you have prod rights)

Sindre•6/14/23, 10:11 AM

I'm not currently using any spes. schema to deploy to prod. I just deploy often to prod and fix bug, but I like the deploy to prod, button. I think I will use that.
It's not clear to me how it will behave with regards to resources. I think It would be hard to update env var. I think maybe that would be in the hands of the developer to make sure that is working as it should. But for resources you should have a way of updating the resource to select the correct one. I'm not sure if that would be to select a different resources, or that you have two versions of a resources, a prod resouces and a dev/stage resource that share the same name.

rubenf•6/14/23, 2:49 PM

my current iteration

rubenf•6/14/23, 2:51 PM

@Sindre I thought really long about how to differentiate prod variable/resources and after coming up with various complex scheme that wouldn't work well with local dev, here is the one that i retained:
Introducing the WM_PROD context variable that is true if the file is in prod. The expected usage is:

Sindre•6/14/23, 3:11 PM

Could work for some things, but is not best practice to have Resources as a parameter in the scripts?
So how do you align a flow that has some scripts with static inputs that are resources that should change?

rubenf•6/14/23, 3:13 PM

you are ... right

rubenf•6/14/23, 3:17 PM

One possibility is to add a remapping to flows to top level

rubenf•6/14/23, 3:18 PM

Something in the flow settings
In prod, replace the following resources:
A -> F
B -> G

rubenf•6/14/23, 3:21 PM

Thoughts @Jim and @Sindre on a remapping ?

rubenf•6/14/23, 3:23 PM

The issue is that you do not want to use the same variable for permission reasons