sledge
sledge2w ago

Worker caching weirdness

We are using a web KV store. For a specific key we have written three blobs of JSON - each write with a different number of array elements in this order (5, 8, and 16). We have a simple windmill typescript script that fetches the data and logs the number of array elements. When we run that script on the windmill deploy (self-hosted EE) the workers get a variety of responses from the fetch (sometimes 5, someimtes 8, sometimes 16). Its like windmill has cached the internal fetch results and is returning that instead of doing the actual fetch. Which seems weird to me - since its just running our script on a worker. However, if I run the exact same fetch on a cloud-hosted windmill, the script always gets 16 elements (which is all that instance has ever seen). Is WM doing something like this or have we just hit some weird distributed caching problem with the KV store we're using? The fetch is simple: let result = await fetch(url) let data = await result.json() console.log(data.length)
2 Replies
rubenf
rubenf2w ago
Are you using native scripts or normal deno scripts? if normal deno scripts, we're literally just running deno run
sledge
sledgeOP2w ago
Maybe a bug in the response headers from the KV store? HTTP/2 200 server: Fly/d42d3a7f1 (2024-11-25) date: Wed, 27 Nov 2024 15:09:05 GMT content-type: text/plain kvdb-edge-region: iad vary: Accept-Encoding vary: Origin access-control-allow-origin: * expires: Fri Dec 27 14:01:15 2024 GMT strict-transport-security: max-age=31536000; includeSubDomains; preload x-content-type-options: nosniff x-frame-options: SAMEORIGIN x-response-time: 0.467ms x-xss-protection: 1; mode=block kvdb-cache-status: HIT via: 2 fly.io fly-request-id: 01JDQ2PCK6MXMMG9KAP8ZJAQJ2-iad This is what I see from curl. I tried both as a normal deno and normal bun. That is what I thought too ... how could this be a WM issue if they're just launching the script on a worker using the specified runtime. Those headers are from running curl here locally on my machine. I'm gonna see what the response headers look like on the weird responses. Well the response headers look pretty much the same from the deno requests as from curl: Response Headers: access-control-allow-origin: * content-type: text/plain date: Wed, 27 Nov 2024 15:21:36 GMT expires: Fri Dec 27 13:33:55 2024 GMT fly-request-id: 01JDQ3DAFZC30W8XJXY9HJGJ6D-sea kvdb-cache-status: HIT kvdb-edge-region: sjc server: Fly/d42d3a7f1 (2024-11-25) strict-transport-security: max-age=31536000; includeSubDomains; preload vary: Origin via: 2 fly.io x-content-type-options: nosniff x-frame-options: SAMEORIGIN x-response-time: 0.490ms x-xss-protection: 1; mode=block Got 5 messages And I agree - since this is just fetch getting executed by deno - this has to be a caching issue with the KV store. One interesting tidbit - the bun requests used http 1.1 - deno used http 2.0