Get root workflow id
I have a data pipeline that normally runs in incremental mode, but sometimes we want to do a full reload. The full reload will be a workflow the reuses the normal incremental workflows with some extra config. One critical config is the target database. During the full reload we'll target a temporary database, and at the end of the flow we'll exchange tables between the old and new. This gives us a very safe mechanism for doing reloads non-destructively.
One way to achieve this would be to parameterize all the incremental loading scripts (and workflows). But this would complicate the code for something that happens rarely.
So, to achieve this I though instead I could set some ontextual config that should apply to all the jobs that run as a result of the top-level reload workflow to override the database they connect to. My idea was that I could set a Windmill resource which contains the necessary config as well as the workflow id of the main reload workflow.
In practice, I have some shared code for connecting to the database that would detect the presence of this override and target the temporary reload database instead. For this to work I must be able to find the "root workflow id", i.e. if I kick off the reload and it is assigned id
But it appears that there is no variable that contains this, just
So, is there a way I'm not seeing for accessing the root flow id from anywhere "inside" the flow?
One way to achieve this would be to parameterize all the incremental loading scripts (and workflows). But this would complicate the code for something that happens rarely.
So, to achieve this I though instead I could set some ontextual config that should apply to all the jobs that run as a result of the top-level reload workflow to override the database they connect to. My idea was that I could set a Windmill resource which contains the necessary config as well as the workflow id of the main reload workflow.
In practice, I have some shared code for connecting to the database that would detect the presence of this override and target the temporary reload database instead. For this to work I must be able to find the "root workflow id", i.e. if I kick off the reload and it is assigned id
123123But it appears that there is no variable that contains this, just
WM_FLOW_PATHWM_FLOW_JOB_IDSo, is there a way I'm not seeing for accessing the root flow id from anywhere "inside" the flow?
