Tony•5d ago

New image causing errors

We run a self-hosted version of Windmill on AWS ECS. The task config was set to use the latest Windmill image. We had not restarted the tasks in months. Yesterday I restarted them & they seem to have picked up a new Windmill image. Now the logs have some new errors in them. Here is an example: Sql error: error occurred while decoding column "job_kind": invalid value "flownode" for enum JobKind error=SqlErr(ColumnDecode { index: ""job_kind"", source: "invalid value "flownode" for enum JobKind" }) target=windmill_common::error span={"email":"admin@windmill.dev","method":"GET","trace_id":"c9b78efa-344b-4ccc-b8bd-8c2b53544252","uri":"/api/w/design-generation/jobs_u/get/0195d35e-dbfa-ae09-801f-ad7dc63afda2?no_logs=true","username":"admin","workspace_id":"design-generation","name":"request"} Is there a correct way to upgrade (or even to rollback) that we should follow to extricate ourselves from this situation?

10 Replies

rubenf•5d ago

what is the version of that instance (you can see at beginning of the logs)

TonyOP•5d ago

Server: Windmill Enterprise Edition v1.478.1-14-gb822c6626 Workers: Windmill Community Edition v1.478.1-14-gb822c6626

rubenf•5d ago

can you run \dT+ job_kind on your database with a psql client and can you triple check that the server you took the logs is on that version (curl <instance>/api/version)

TonyOP•5d ago

windmill=> \dT+ job_kind 
                                             List of data types
 Schema |   Name   | Internal name | Size |      Elements      |  Owner   | Access privileges | Description 
--------+----------+---------------+------+--------------------+----------+-------------------+-------------
 public | job_kind | job_kind      | 4    | script            +| postgres |                   | 
        |          |               |      | preview           +|          |                   |

windmill=> \dT+ job_kind 
                                             List of data types
 Schema |   Name   | Internal name | Size |      Elements      |  Owner   | Access privileges | Description 
--------+----------+---------------+------+--------------------+----------+-------------------+-------------
 public | job_kind | job_kind      | 4    | script            +| postgres |                   | 
        |          |               |      | preview           +|          |                   |

There are several other Elements too. flow +
dependencies +
flowpreview +
script_hub +
identity +
flowdependencies +
http +
graphql +
postgresql +
noop +
appdependencies +
deploymentcallback+
singlescriptflow +
flowscript +
flownode +
appscript

rubenf•5d ago

I think your server is not at the version you think it is if you take the head of its logs it logs its version at start

TonyOP•5d ago

Maybe our DB & server are mismatched?

rubenf•5d ago

find out the version of that server first

TonyOP•5d ago

I got "Windmill Enterprise Edition v1.478.1-14-gb822c6626" from looking at the server logs. But let me try using curl to see if I get the same thing. https://windmill.bondstudio.ai/api/version returns EE v1.478.1-14-gb822c6626 too

rubenf•5d ago

and you keep having the same error?

TonyOP•5d ago

Earlier today I was seeing a message in the top right corner of https://windmill.bondstudio.ai saying we neeeded to upgrade from a much older version. I no longer see that error. I'm not quite sure why though. Let me see if it's working again now. Looks like my attempt to redeploy the workers failed but the server had succeeded, which probably explains the errors. Now the workers match the server. Thanks for your help 🙂

New image causing errors

Did you find this page helpful?