rubenf•16mo ago

Eval V2 are here!

Eval V2 are here! Apps go from already really fast to super blazing fast.. We now parse your eval and frontend scripts using the swc parser compiled to wasm to extract any references to outputs. It allows windmill to suggest dependencies for frontend scripts as shown in the video. More importantly, now evals are only evaluated on changes to the outputs that have been identified. It means that most likely a properly made windmill app could be as fast or faster than full code apps. Prior evals were recomputed for any change to any output. This scale really bad for complex apps and why we were careful about the usage of evals. Evals will not be converted automatically so you will have to press the toggle to go from Legacy Eval to New Eval. Since evals are now blazing fast, we made everything en eval 🙂

2 Replies

Sindre•16mo ago

Super nice, will eval now be the default and not static? I'm a huge eval fan, and if there is close to nothing in performance overhead, I would like to just to "" for static strings, and c.results for connect. It's much easiere to type then to find the compnent in my nested structure on the left... (feedback: be able to click on the component you want is prop a nice idea. my discgolf buddy was a at least a bit confused when using connect. ) I do not see any downside to eval... could you explain a bit more on why we need to opt in and this is not a internal upgrade? Auto add dependencies to frontend apps? Is that something you have looked into ? Are you able to find out if you have a assigment to a variable in the frontend script and auto add it to dependcies? I guess it could be hard to know when there is a mutation going on inside a function... so you cant be sure in all the cases but would be really nice to auto add dependencies you are sure about. Will flow also get this? Eval FTW! 🤩

rubenfOP•16mo ago

I agree we can likely merge connect and eval. I have to do a little bit of benchmarks because they do not work exactly the same way but I do not expect a big difference eval and static should not be merged, they are pretty different and for strings it might look somewhat the same (minus quotes) but for other types the static editor is quite different I rather do like for flows and just dynamically transform static to eval form when toggle is clicked There are 2 reasons for not doing this internally and requiring opt-in: The logic is completely different and I did not want to break existing apps. We actually need the wasm to run to evaluate your code and doing migration code to force upon edit entry is quite a bit of work considering the above point

<script lang="ts">
    import { accessPropertyByPath } from '$lib/components/apps/utils'

    function evalPath(value: any, path: string) {
        path = path.replace(/\[(\d+)\]/g, '.$1').replace(/\[\"(.*)\"\]/g, '.$1')
        let splitPoint = path.indexOf('.')
        if (splitPoint != -1) {
            const realPath = path.substring(splitPoint + 1)
            return accessPropertyByPath(value, realPath)
        }
    }

    const v = { foo: { bar: { foo: { bar: { bar: { bar: 23 } } } } }, bar: 23 }

    function createFunction(v: any, path: string) {
        return new Function(`
        let x = ${JSON.stringify(v)}
        return ${path}`) // this `x` refers to global `x`
    }

    function benchmark() {
        let x = Date.now()
        new Array(1000000000).forEach(() => {
            evalPath(v, 'x.bar')
        })
        let x1 = Date.now() - x
        let x2 = Date.now()
        new Array(1000000000).forEach(() => {
            createFunction(v, 'x.bar')()
        })
        let x3 = Date.now() - x2

        return [x1, x3]
    }

    let [a, b] = benchmark()
    function bench() {
        ;[a, b] = benchmark()
    }
</script>

<button on:click={bench}>Bench</button>
{a} - {b}
{evalPath(v, 'x.foo.bar.foo.bar.bar.bar')}
{createFunction(v, 'x.foo.bar.foo.bar.bar.bar')()}

<script lang="ts">
    import { accessPropertyByPath } from '$lib/components/apps/utils'

    function evalPath(value: any, path: string) {
        path = path.replace(/\[(\d+)\]/g, '.$1').replace(/\[\"(.*)\"\]/g, '.$1')
        let splitPoint = path.indexOf('.')
        if (splitPoint != -1) {
            const realPath = path.substring(splitPoint + 1)
            return accessPropertyByPath(value, realPath)
        }
    }

    const v = { foo: { bar: { foo: { bar: { bar: { bar: 23 } } } } }, bar: 23 }

    function createFunction(v: any, path: string) {
        return new Function(`
        let x = ${JSON.stringify(v)}
        return ${path}`) // this `x` refers to global `x`
    }

    function benchmark() {
        let x = Date.now()
        new Array(1000000000).forEach(() => {
            evalPath(v, 'x.bar')
        })
        let x1 = Date.now() - x
        let x2 = Date.now()
        new Array(1000000000).forEach(() => {
            createFunction(v, 'x.bar')()
        })
        let x3 = Date.now() - x2

        return [x1, x3]
    }

    let [a, b] = benchmark()
    function bench() {
        ;[a, b] = benchmark()
    }
</script>

<button on:click={bench}>Bench</button>
{a} - {b}
{evalPath(v, 'x.foo.bar.foo.bar.bar.bar')}
{createFunction(v, 'x.foo.bar.foo.bar.bar.bar')()}

ok benchmarks are surprising and connect and eval are approximately as fast or as slow 😄 👋 connect logic