amigozz_•4w ago

For Loops hangs if any iterations fail

Hi! Thank you for the great tool! I was testing the following flow: - it fetches rss articles - processes articles in a for loop - fetches article content - summarizes the article - collects articles and summary in a single object from a for loop The fetches article content step might fail sometime and in a script I don't have any error handling; my intuition was that the for loop should skip the failures and the resulting collection simply won't contain them (as I have skip failures enabled in a for loop) Instead of this I'm seeing the flow hanging and not proceeding because of the failures. Should I add error handling to the script to avoid that or is there a better way to tell the for loop to ignore the failures?

12 Replies

rubenf•4w ago

what do you mean by the for loop hangs? also is it a parallel for loop?

amigozz_OP•4w ago

hangs

the flow never completes, and it showing that it's currently executing the for loop it's a parallel (2 threads) for loop

rubenf•4w ago

do you have a minimal reproduction, i'm not following the issue

amigozz_OP•4w ago

let me grab the code

rubenf•4w ago

if you can run that on app.windmill.dev and share the resulting run that'd be great

amigozz_OP•4w ago

let me know if you can see it https://app.windmill.dev/flows/get/u/stepan/benevolent_flow?workspace=demo the problematic node is this one inside the foor loop

import { JSDOM } from 'npm:jsdom';
import { Defuddle } from 'npm:defuddle/node';

type Input = {
  url: string;
}

export async function main(input: Input) {
  // Parse HTML from a URL
  const dom = await JSDOM.fromURL(input.url);
  const result = await Defuddle(dom);
  return result;
}

import { JSDOM } from 'npm:jsdom';
import { Defuddle } from 'npm:defuddle/node';

type Input = {
  url: string;
}

export async function main(input: Input) {
  // Parse HTML from a URL
  const dom = await JSDOM.fromURL(input.url);
  const result = await Defuddle(dom);
  return result;
}

Here, JSDOM.fromURL can fail with network error and the for loop seems to wait something and do not stop

rubenf•4w ago

It's not hanging for me:

rubenf•4w ago

you have one particular url that makes it hang but that's user code specific:

rubenf•4w ago

anyway, not a windmill issue

amigozz_OP•4w ago

one particular url that makes it hang

yeah, this is what I meant the URL retreival fails with 403

rubenf•4w ago

execute that script with deno locally, it will hang too it's up to you if you want to add code or windmill level timeout but windmill is working as intended

amigozz_OP•4w ago

execute that script with deno locally, it will hang too

oh this makes perfect thanks, thank you. The script itself is hanging I thought the script finishes with error, but if it hangs, then yes, it's expected I guess for for loop to hang

code or windmill level timeout

thank you, I will look into that. Have a great day!

For Loops hangs if any iterations fail

Did you find this page helpful?