One particular workspace's schedules page takes too long to load

Earlier it was 3-4 seconds, now upto 10 seconds.

Other workspace's schedule page load as usual 1-2 seconds
but this particular workspace takes too long, I check in chrome network inspector, the api takes too long to finish.

Not sure what is the issue

rubenf•10/30/23, 10:16 AM

It's the sql queries taking too long. You can do a sqldump of your jobs relative to your schedules and sent them to us but otherwise we won't be able to investigate

ym1198OP•10/30/23, 10:20 AM

yes it is related to db query only

but only for one workspace, is what surprised me, all other workspace schedule page loads fast

ym1198OP•10/30/23, 10:37 AM

I checked in src, this query is taking time for problematic case and not taking time for non-problematic case.

https://github.com/windmill-labs/windmill/blob/main/backend/windmill-api/src/schedule.rs#L331

GitHub

windmill/backend/windmill-api/src/schedule.rs at main · windmill-la...

Open-source developer platform to turn scripts into workflows and UIs. Open-source alternative to Airplane and Retool. - windmill-labs/windmill

ym1198OP•10/30/23, 10:38 AM

SELECT schedule., t.jobs
FROM schedule, LATERAL ( SELECT ARRAY (SELECT json_build_object('id', id, 'success', success, 'duration_ms', duration_ms) FROM completed_job WHERE
completed_job.schedule_path = schedule.path AND completed_job.workspace_id = 'problem' AND parent_job IS NULL ORDER BY started_at DESC LIMIT 20) AS jobs ) t
WHERE schedule.workspace_id = 'problem'
ORDER BY schedule.edited_at desc LIMIT 50 OFFSET 0;

SELECT schedule., t.jobs
FROM schedule, LATERAL ( SELECT ARRAY (SELECT json_build_object('id', id, 'success', success, 'duration_ms', duration_ms) FROM completed_job WHERE
completed_job.schedule_path = schedule.path AND completed_job.workspace_id = 'no-problem' AND parent_job IS NULL ORDER BY started_at DESC LIMIT 20) AS jobs ) t
WHERE schedule.workspace_id = 'no-problem'
ORDER BY schedule.edited_at desc LIMIT 50 OFFSET 0;

first query takes longer time, second takes less

rubenf•10/30/23, 10:38 AM

It's not the query that is the issue, it's the indexing

ym1198OP•10/30/23, 10:39 AM

i ran both directly on postgres container db

rubenf•10/30/23, 10:39 AM

which is why we need the sqldump to figure out why the indexing is not used properly

rubenf•10/30/23, 10:39 AM

for us to have more info, ideally you can do; EXPLAIN ANALYZE

rubenf•10/30/23, 10:39 AM

it will show you the query plan

rubenf•10/30/23, 10:39 AM

normally the query should use mostly the index

ym1198OP•10/30/23, 10:40 AM

i might not be able to give the sqldump, acc to company policy

let me share explain analyze plan for both

ym1198OP•10/30/23, 10:44 AM

non-problem.txt2.39KB

ym1198OP•10/30/23, 10:45 AM

problem.txt2.78KB

rubenf•10/30/23, 10:47 AM

Thanks, that's all we need right now

ym1198OP•12/18/23, 5:24 AM

Hi just checking if any updates on this as now all workspace are facing this issue
it loads only 2 out of 10 times and takes 10-20 seconds
sometimes gets 504

rubenf•12/18/23, 5:54 AM

Yes we solved that long ago normally

rubenf•12/18/23, 5:54 AM

What version are you using ?

ym1198OP•12/18/23, 5:56 AM

is it visible on ui anywhere ?

ym1198OP•12/18/23, 5:56 AM

or i need to check on server images ?

rubenf•12/18/23, 5:57 AM

Account settings

ym1198OP•12/18/23, 5:57 AM

Windmill CE v1.114.2-13

ym1198OP•12/18/23, 5:58 AM

i should upgrade then ?
any specific steps to follow ?

ym1198OP•12/18/23, 5:58 AM

i am using docker compose method

rubenf•12/18/23, 6:00 AM

Yes you're on a very old version

rubenf•12/18/23, 6:00 AM

Look at latest docker compose

rubenf•12/18/23, 6:00 AM

And adapts yours to it

rubenf•12/18/23, 6:02 AM

For more ample customer support, feel free to discuss getting an ee license at sales@windmill.dev

ym1198OP•12/18/23, 6:06 AM

ok will get new version,
I guess i can just do , add

for windmill container images

and then would update the versions