Skip to content

[question] daemon concept #413

Description

@listvin

While caching compiled code is great,
one of the concerns I have while trying Kotlin scripting is startup time.

What's your opinion on possibility of JVM running as daemon, ready to pickup scripts on startup?
Similar in some manner to Gradle daemon.

Some properties to clarify concept:

  • start daemon when first script is started since system startup
    • limited lifetime
    • optional startup when system boots
  • dynamically load bytecode
  • one daemon per every runtime configuration
    • maybe several daemons per runtime if there are more than one script running

I believe it will give performance boost by utilizing JIT out of the box.

Activity

  1. aartiPl commented on Nov 26, 2023

    @aartiPl
    Collaborator

    Hi! This is an interesting idea. I am very curious about the real gains of daemon implementation compared to the implementation without daemon.

  2. ybznek commented on Dec 22, 2023

    @ybznek

    Startup time of compiled (.jar) script
    if it is an issue it could be compiled into native code (https://www.graalvm.org/latest/reference-manual/native-image/basics/), but the binary is roughly 8MB at minimum (maybe could be adjusted more?) and compilation is slow.
    Task of gradle daemon is "just" compilation. Shared daemon across various scripts would require some isolation - e.g. you don't want to kill daemon if one of scripts call exit(), etc.

    Initial startup time of not complied script:
    I've done experiment with my <1 day project similar to kscript.
    I do not call kotlinc, but ./gradlew and pass variables like srcDir, dependencies, .. via env. variables
    First start with kscript seems to be roughly 6x slower than my solution with running gradle daemon

    Significant slowdown of kscript seems to be due to resolution of dependency. My solution just collect
    runtimeClasspath from gradle (/home/z/.gradle/caches/modules-2/files-2.1/org.jsoup/jsoup/1.16.2/5d39ff6d1b8331c376a04c50c6716afbcdfe567d/jsoup-1.16.2.jar) - and let gradle to download that.

    Script:

    #!/usr/bin/env /home/z/projects/kotlin/kscript2/runner/kscript2
    @file:DependsOn("org.jsoup:jsoup:1.16.2")
    import org.jsoup.Jsoup
    println("a")
    println( Jsoup.parse("<html><body id=\"b\">sth</body></html>")!!.getElementById("b")!!.text())
    

    First run with kscript:

    time kscript /home/z/projects/kotlin/kscript2/runner/src/test/resources/source/main.kts a
    [kscript] Resolving org.jsoup:jsoup:1.16.2...
    a
    sth
    kscript  a  5,37s user 0,50s system 263% cpu 2,225 total
    

    First run with my experiment with running daemon:

    time /home/z/projects/kotlin/kscript2/runner/src/test/resources/source/main.kts a
    a
    sth
    /home/z/projects/kotlin/kscript2/runner/src/test/resources/source/main.kts a  0,86s user 0,16s system 98% cpu 1,027 total
    
    
  3. aartiPl commented on Dec 26, 2023

    @aartiPl
    Collaborator

    Nice - 6x improvement in start-up looks very promising 👍

  4. listvin commented on Jan 10, 2024

    @listvin
    Author

    @ybznek thanks for actual minifyed timing sample! Can you please share your project as repo?

    Shared daemon across various scripts would require some isolation - e.g. you don't want to kill daemon if one of scripts call exit(), etc.

    Nice concern. Do you have more ideas of isolation cases?
    Regarding .exit probably auto-restarting daemon, if it exits or crashes on its own (not via ~ $ kscriptd stop-daemon), would be sufficient at least for a poc-version?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions