Workflows as Code | Windmill

Windmill•3y ago•

13 replies

We are releasing in beta Workflow as code for Python and Typescript. No more excuse to use Airflow or Prefect: https://www.windmill.dev/docs/core_concepts/workflows_as_code

from wmill import task

import pandas as pd
import numpy as np

# The pin is important since task is a decorator available only from 1.286.2

#extra_requirements:
#wmill>=1.286.2


##You can specify tag to run the task on a specific type of worker
@task(tag="highmem")
def heavy_compute(n: int):
    df = pd.DataFrame(np.random.randn(100, 4), columns=list('ABCD'))
    return df.sum().sum()


@task
def send_result(res: int, email: str):
    # logs of the subtask are available in the main task logs
    print(f"Sending result {res} to {email}")
    return "OK"

def main(n: int):
    l = []

    # to run things in parallel, simply use multiprocessing Pool map instead: https://docs.python.org/3/library/multiprocessing.html
    for i in range(n):
        l.append(heavy_compute(i))
    print(l)
    return send_result(sum(l), "example@example.com")

from wmill import task

import pandas as pd
import numpy as np

# The pin is important since task is a decorator available only from 1.286.2

#extra_requirements:
#wmill>=1.286.2


##You can specify tag to run the task on a specific type of worker
@task(tag="highmem")
def heavy_compute(n: int):
    df = pd.DataFrame(np.random.randn(100, 4), columns=list('ABCD'))
    return df.sum().sum()


@task
def send_result(res: int, email: str):
    # logs of the subtask are available in the main task logs
    print(f"Sending result {res} to {email}")
    return "OK"

def main(n: int):
    l = []

    # to run things in parallel, simply use multiprocessing Pool map instead: https://docs.python.org/3/library/multiprocessing.html
    for i in range(n):
        l.append(heavy_compute(i))
    print(l)
    return send_result(sum(l), "example@example.com")

Workflows as Code | Windmill

Flows are not the only way to write distributed programs that execute distinct jobs. Another approach is to write a program that defines the jobs and their dependencies, and then execute that program. This is known as workflows as code.

Windmill•3y ago•

13 replies

rubenf

We are releasing in beta Workflow as code for Python and Typescript. No more excuse to use Airflow or Prefect: https://www.windmill.dev/docs/core_concepts/workflows_as_code

from wmill import task

import pandas as pd
import numpy as np

# The pin is important since task is a decorator available only from 1.286.2

#extra_requirements:
#wmill>=1.286.2


##You can specify tag to run the task on a specific type of worker
@task(tag="highmem")
def heavy_compute(n: int):
    df = pd.DataFrame(np.random.randn(100, 4), columns=list('ABCD'))
    return df.sum().sum()


@task
def send_result(res: int, email: str):
    # logs of the subtask are available in the main task logs
    print(f"Sending result {res} to {email}")
    return "OK"

def main(n: int):
    l = []

    # to run things in parallel, simply use multiprocessing Pool map instead: https://docs.python.org/3/library/multiprocessing.html
    for i in range(n):
        l.append(heavy_compute(i))
    print(l)
    return send_result(sum(l), "example@example.com")

from wmill import task

import pandas as pd
import numpy as np

# The pin is important since task is a decorator available only from 1.286.2

#extra_requirements:
#wmill>=1.286.2


##You can specify tag to run the task on a specific type of worker
@task(tag="highmem")
def heavy_compute(n: int):
    df = pd.DataFrame(np.random.randn(100, 4), columns=list('ABCD'))
    return df.sum().sum()


@task
def send_result(res: int, email: str):
    # logs of the subtask are available in the main task logs
    print(f"Sending result {res} to {email}")
    return "OK"

def main(n: int):
    l = []

    # to run things in parallel, simply use multiprocessing Pool map instead: https://docs.python.org/3/library/multiprocessing.html
    for i in range(n):
        l.append(heavy_compute(i))
    print(l)
    return send_result(sum(l), "example@example.com")

Workflows as Code | Windmill

Similar Threads

Similar Threads

Similar Threads

Similar Threads