@rubenf I'm seeing the following migration error when upgrading from v1.554.0 to v1.561.0 ``` 2025-1

@rubenf I'm seeing the following migration error when upgrading from v1.554.0 to v1.561.0

2025-10-16T18:12:51.094722Z  INFO windmill-api/src/db.rs:187: Finished applying migration 20251006143822
Error: Error migrating db: Migrating database: while executing migration 20251006143822: error returned from database: null value in column "value" of relation "global_settings" violates not-null constraint

2025-10-16T18:12:51.094722Z  INFO windmill-api/src/db.rs:187: Finished applying migration 20251006143822
Error: Error migrating db: Migrating database: while executing migration 20251006143822: error returned from database: null value in column "value" of relation "global_settings" violates not-null constraint

2025-10-16T18:12:51.094722Z  INFO windmill-api/src/db.rs:187: Finished applying migration 20251006143822
Error: Error migrating db: Migrating database: while executing migration 20251006143822: error returned from database: null value in column "value" of relation "global_settings" violates not-null constraint

2025-10-16T18:12:51.094722Z  INFO windmill-api/src/db.rs:187: Finished applying migration 20251006143822
Error: Error migrating db: Migrating database: while executing migration 20251006143822: error returned from database: null value in column "value" of relation "global_settings" violates not-null constraint

Checking my db I don't see any null settings values

windmill=# SELECT * from global_settings where value is NULL;
 name | value | updated_at
------+-------+------------

windmill=# SELECT * from global_settings where value is NULL;
 name | value | updated_at
------+-------+------------

windmill=# SELECT * from global_settings where value is NULL;
 name | value | updated_at
------+-------+------------

windmill=# SELECT * from global_settings where value is NULL;
 name | value | updated_at
------+-------+------------

Which points me to the migration might be the thing that is trying to insert a null value: https://github.com/windmill-labs/windmill/blob/6dfa4ff5ba26bc4912b1be1a9709728cf6c2d119/backend/migrations/20251006143822_ducklake_instance_settings_better.down.sql#L3. I don't have ducklake settings

windmill=# select name from global_settings;
         name
-----------------------
 jwt_secret
 dev_instance
 retention_period_secs
 disable_stats
 job_default_timeout
 oauths
 base_url
 license_key

windmill=# select name from global_settings;
         name
-----------------------
 jwt_secret
 dev_instance
 retention_period_secs
 disable_stats
 job_default_timeout
 oauths
 base_url
 license_key

windmill=# select name from global_settings;
         name
-----------------------
 jwt_secret
 dev_instance
 retention_period_secs
 disable_stats
 job_default_timeout
 oauths
 base_url
 license_key

windmill=# select name from global_settings;
         name
-----------------------
 jwt_secret
 dev_instance
 retention_period_secs
 disable_stats
 job_default_timeout
 oauths
 base_url
 license_key

rubenf•10/16/25, 6:25 PM

@Diego looks to be your migration:

INSERT INTO global_settings (name, value) VALUES (
  'ducklake_settings',
  (SELECT json_build_object(
    'ducklake_user_pg_pwd', g2.value,
    'instance_catalog_db_status', '{}'::json
  ) FROM global_settings g2 WHERE g2.name = 'ducklake_user_pg_pwd')
);

INSERT INTO global_settings (name, value) VALUES (
  'ducklake_settings',
  (SELECT json_build_object(
    'ducklake_user_pg_pwd', g2.value,
    'instance_catalog_db_status', '{}'::json
  ) FROM global_settings g2 WHERE g2.name = 'ducklake_user_pg_pwd')
);

Diego•10/16/25, 6:27 PM

Ttl_jacob @rubenf I'm seeing the following migration error when upgrading from v1.554.0 to...

Diego•10/16/25, 6:35 PM

Can you try to run this migration by hand

https://github.com/windmill-labs/windmill/blob/6dfa4ff5ba26bc4912b1be1a9709728cf6c2d119/backend/migrations/20250731132157_ducklake_instance_settings.up.sql

GitHub

windmill/backend/migrations/20250731132157_ducklake_instance_settin...

Open-source developer platform to power your entire infra and turn scripts into webhooks, workflows and UIs. Fastest workflow engine (13x vs Airflow). Open-source alternative to Retool and Temporal...

Diego•10/16/25, 6:36 PM

Somehow ducklake_user_pg_pwd does not exist

DDiego Can you try to run this migration by hand https://github.com/windmill-labs/wind...

tl_jacobOP•10/16/25, 6:44 PM

okay that fixed the issue, I'm wondering how my windmill db got in that state though since I never run any manual migrations. Did I miss an upgrade message about needing to run a manual migration?

Diego•10/16/25, 6:45 PM

no this should have been run automatically liked other migrations

Diego•10/16/25, 6:45 PM

i don't understand either

Diego•10/16/25, 6:45 PM

@rubenf if you have an idea how it's possible that a migration got skipped

rubenf•10/16/25, 6:45 PM

Not possible

Diego•10/16/25, 6:46 PM

well it's magic

Diego•10/16/25, 6:46 PM

never happened before

Rrubenf look for the associated dependency job

rubberducky•10/16/25, 7:18 PM

sorry im not seeing any issues with depedency jobs eitehr

Rrubberducky sorry im not seeing any issues with depedency jobs eitehr

Pyra•10/16/25, 7:36 PM

Is it just having infinite spinner e.g. deployment in progress? Are lockfiles generated?

If there is no locks there is likely a djob somewhere still running.

Did you try to redeploy?

rubberducky•10/16/25, 7:36 PM

I actually wiped my entire windmill instance and did a fresh push

rubberducky•10/16/25, 7:37 PM

" Deployment in progress" on the flow page

Pyra•10/16/25, 7:38 PM

What version are you on? The minimal reproduction is to just create a flow and deploy it?

rubberducky•10/16/25, 7:38 PM

1.555.1

PPyra What version are you on? The minimal reproduction is to just create a flow and d...

Pyra•10/16/25, 7:39 PM

Any specific actions? Imports?

Pyra•10/16/25, 7:41 PM

And what is your infra? k8s? Docker? Native?

Ttl_jacob okay that fixed the issue, I'm wondering how my windmill db got in that state th...

Diego•10/16/25, 7:42 PM

Did you ever do something to your global settings that might have deleted it ?

Diego•10/16/25, 7:42 PM

I'm going through the code and assuming the migration ran, I don't see how this was possible

PPyra And what is your infra? k8s? Docker? Native?

rubberducky•10/16/25, 7:42 PM

right now a local docker running

rubberducky•10/16/25, 7:43 PM

its a rather complicated flow, all python is checked and valid with tests and i can confirm they work.

rubberducky•10/16/25, 7:43 PM

I can view all scripts etc on the frontend

DDiego Did you ever do something to your global settings that might have deleted it ?

tl_jacobOP•10/16/25, 7:44 PM

I've been setting instance-level settings using a

instance_settings.yaml

instance_settings.yaml

that doesn't have any ducklake settings

Diego•10/16/25, 7:44 PM

Diego•10/16/25, 7:45 PM

there we go

thanks

tl_jacobOP•10/16/25, 7:47 PM

@Diego just to be clear I do feel like the way this has broken is not an optimal customer/developer experience. I don't think upgrading versions should break in this way.

Diego•10/16/25, 7:47 PM

yes it's a bug

Rrubberducky its a rather complicated flow, all python is checked and valid with tests and i ...

Pyra•10/16/25, 7:47 PM

Could it just be locking things? Could happen if you have poor connectivity.

But if it is complicated flow, I think it is the best to gradually reduce until it works

tl_jacobOP•10/16/25, 7:48 PM

@Diego gotcha! happy to be a point of contact on the fix if that's helpful

really appreciate how you + ruben have hopped in here so quickly

PPyra Could it just be locking things? Could happen if you have poor connectivity. Bu...

rubberducky•10/16/25, 7:51 PM

ill refactor it and start with each step until it breaks somewhere

Pyra•10/16/25, 7:52 PM

Yeah. But you should really check dependency jobs. You can filter by that job kind in run page

Pyra•10/16/25, 7:52 PM

I just think you have something heavy or it is somehow stuck on a a dependency

rubberducky•10/16/25, 7:53 PM

its a local instance, no jobs running except the deployment

Pyra•10/16/25, 7:54 PM

by heavy I mean you have hundreds of dependencies (including transitive) and your instance didn't finish fetching all necessarily metadata

rubberducky•10/16/25, 7:58 PM

they all finished, is there a chance one may have not?

Pyra•10/16/25, 8:11 PM

If it is a single flow and there is no relative imports, it would be a single dependency job. If that dependency job has finished then the flow should be viewable as deployed

metapontum•10/17/25, 11:02 AM

This is how simple it could be to interact with wmill storage / ducklake

metapontum•10/17/25, 11:04 AM

https://bun.com/blog/bun-v1.3

Bun

Bun 1.3

Bun 1.3 introduces zero-config frontend development, unified SQL API, built-in Redis client, security enhancements, package catalogs, async stack traces, VS Code test integration, and Node.js compatibility improvements.

metapontum•10/17/25, 11:16 AM

Very impressive from bun, when I was reading it I kept thinking that these can be made to interoperate with windmill by default

metapontum•10/17/25, 11:21 AM

Resources ≈ redis KV (types can be specified inline, defaulting to ‘state’ _resource_type: ‘s3’)
Variables ≈ secrets
Storage ≈ s3
* Ducklake ≈ sql

rubenf•10/17/25, 11:44 AM

Agreed @metapontum , we will probably want a simplified syntax, maybe a one function call in our sdk to setup the s3 client

Zach•10/17/25, 3:01 PM

Have you thought about Bun to build/serve the raw apps, @rubenf? (I’ve actually been using raw apps to iframe to a subdomain with a cookie.) Was reading through some of the raw apps branch and was just curious?

rubenf•10/17/25, 3:07 PM

@Zach raw apps are purely static frontend rendering + windmill jobs for background runnables

rubenf•10/17/25, 3:47 PM

https://youtube.com/live/mrnmNgQ22jE?feature=share weekly!

YouTubeWindmill

Windmill Weekly 17th October, 2025

Enjoy the videos and music you love, upload original content, and share it all with friends, family, and the world on YouTube.

Rrubenf https://youtube.com/live/mrnmNgQ22jE?feature=share weekly!

Kelora•10/17/25, 4:36 PM

excited for the ui polish, wanted to suggest some attention on this tiny papercut I run into a lot. Resizing these panes can be hard especially on high resolution/remote desktop.

rubenf•10/17/25, 4:56 PM

at very least we could make it bigger on hover I suppose

Kelora•10/17/25, 4:57 PM

i dont think it needs to be visibly bigger, but a few more pixels of "grip" would help a ton

Kelora•10/17/25, 4:57 PM

the area where my pointer turns into a resizer