brianthegreat•3mo ago

[Solved] DataFrame in Script and Flow Showing Incorrect Headers

I always got confused why my DataFrame header is not correct. You may see this sample code, I printed the logs also to see what Column Header it should have vs what being shown in Windmill. Please see the image. Let me know if the issue is on my side or Windmill's. I expect the column headers: ID, Amount, Status What being shown in Windmill: col0, col1, col2 Thanks.

import pandas as pd
from typing import Any
import csv
import io

def main() -> Any:
    # Create a dummy CSV data string with 3 columns
    csv_data = """ID;Amount;Status
    abc123;1000;completed
    def456;2000;pending
    ghi789;3000;failed"""

    # Use StringIO to treat the string as a file object
    csv_file_like_object = io.StringIO(csv_data)

    # Read the CSV data into a DataFrame directly
    df = pd.read_csv(csv_file_like_object, delimiter=';')

    # Debugging: Print DataFrame information
    print("DataFrame columns:", df.columns)
    print("DataFrame head:\n", df.head())
    
    # Return the DataFrame
    return df

import pandas as pd
from typing import Any
import csv
import io

def main() -> Any:
    # Create a dummy CSV data string with 3 columns
    csv_data = """ID;Amount;Status
    abc123;1000;completed
    def456;2000;pending
    ghi789;3000;failed"""

    # Use StringIO to treat the string as a file object
    csv_file_like_object = io.StringIO(csv_data)

    # Read the CSV data into a DataFrame directly
    df = pd.read_csv(csv_file_like_object, delimiter=';')

    # Debugging: Print DataFrame information
    print("DataFrame columns:", df.columns)
    print("DataFrame head:\n", df.head())
    
    # Return the DataFrame
    return df

2 Replies

wendrul•3mo ago

Hmm, I agree I would have expected it to work like that too. I'm not sure if it's an issue with how pandas serializes dataframes or if it has to do with windmill, but i can try checking. In any case there seems to be a workaround, you can just add this and it will behave as you expect:

  [...]
  return df.to_dict('records')

  [...]
  return df.to_dict('records')

brianthegreatOP•3mo ago

I'm not sure if it's pandas issue or windmill, but it's working fine in notebook, I assume it's Windmills (?) Thanks @Etienne ! It's working with your suggestion. Hi @wendrul there's a continuation of this issue, can you please help check my other post? https://discord.com/channels/930051556043276338/1278663127893807114