rubenfR
Windmill2y ago
13 replies
rubenf

apache iceberg / asset view

Hi @kimsia , you're correct that right now dagster focus on assets while windmill focus on compute. One way to see it is that windmill is lower level than dagster and you can build your own asset abstraction on top of it. We emphasize using object stores such as S3 by passing pointers/reference to it as input or output. When doing that, you can preview parquet files directly in the ui and cache based on etags.

The asset view is an abstraction on top of that we will build based on apache iceberg later but for now some of it you will have to decide for yourself. On the other hand, windmill is a lot more performant and flexible than dagster with respect to execution
Was this page helpful?