Notebook E2E in a macOS GraalVM native image (#191 milestone 1, #249)

docs/SPEC.md

@@ -561,6 +561,17 @@ All in `PM/commonTest`, so they run wherever the interpreter loads.
   with the notebook's own state spelling (`main.App.messages.getValue()` / `setValue(...)`, #174)
   against pythonx-compose f6dd3e2. CI runs it on every push to `develop`
   (`desktop.yml`, job `notebook-e2e`, #213), against a wheel built from pythonx-compose `develop`.
+  **In a GraalVM native image on macOS** (#249, #191's first milestone): `NotebookE2eMain` runs the
+  same 20 scenarios by direct calls, and `notebookE2eNativeRun` requires 20 passed with `JAVA_HOME`
+  and `PYTHONHOME` removed and no `libjvm` linked (only system frameworks); the prefix
+  (`stageNativeImagePythonHome`, L-9), the wheel and the app are staged beside the executable. 20
+  passed, 0 failed on 2026-10-17 (Liberica NIK 25.0.4.1, a 129 MB image), also from a relocated copy
+  under `env -i`. The image takes `ui`, `ui-text` and `foundation` from compose-multiplatform-core-extended
+  (`composeExtended.*` in `gradle.properties`, one place), because upstream Compose dispatches its
+  main-thread work to skiko's Swing queue, which starts AWT, and AWT in an image needs the JDK's
+  `libawt`, which links `libjvm`; the fork sends it to `Dispatchers.Main` when
+  `compose.main.dispatcher=coroutines`, and the fixture supplies a single-thread one with no AWT.
+  `desktopTest` keeps upstream Compose. Not on CI yet: the fork's artefacts are in mavenLocal only.
 - **B-11** (#192) The artifact walker skips every declaration whose package has an `internal` segment
   (`*.internal` and `*.internal.*`, as in `androidx.compose.ui.internal`), on jars and on klibs. That is the
   Kotlin `.api` dump convention: a library that puts a declaration in such a package has declared it

gradle.properties

@@ -34,3 +34,12 @@ pythonAppleSupportBuild=b11
 # toolchain any existing task resolves to. See AGENTS.md §19 for why tooling lives
 # under /Volumes/macMini rather than the internal disk.
 org.gradle.java.installations.paths=/Volumes/macMini/toolchains/bellsoft-liberica-vm-full-openjdk25-25.0.4/Contents/Home
+
+# compose-multiplatform-core-extended (#191, #249): the Compose modules a native-image build takes from
+# the fork instead of upstream, and where from. Read in one place, by the notebook E2E's native image;
+# the fork's coordinates and version line are changing (to 1.11.1-ext.N), and only these lines follow.
+composeExtended.version=1.11.1001
+# The group the fork publishes under, as a prefix of upstream's (the fork moves to org.thisisthepy.compose).
+composeExtended.groupPrefix=org.jetbrains.compose
+composeExtended.modules=org.jetbrains.compose.ui:ui,org.jetbrains.compose.ui:ui-desktop,org.jetbrains.compose.ui:ui-text,org.jetbrains.compose.ui:ui-text-desktop,org.jetbrains.compose.foundation:foundation,org.jetbrains.compose.foundation:foundation-desktop
+composeExtended.mavenLocal=true

python-multiplatform-compose/fixtures/notebook-e2e/README.md

@@ -50,6 +50,30 @@ the wheel from pythonx-compose `develop` with uv and runs this module on every p
 interpreter has no installer of its own. The tests then require `pythonx.compose.__file__` to be
 inside that directory, so a stray source checkout on `sys.path` cannot stand in for the wheel.
 
+## In a GraalVM native image (macOS, #249)
+
+The same 20 scenarios, built into one executable and run on their own:
+
+```bash
+JAVA_HOME=<JDK 21> ./gradlew :python-multiplatform-compose:fixtures:notebook-e2e:notebookE2eNativeRun \
+    -PpythonxComposeWheel=.tmp/pythonx-compose-dist --console=plain
+```
+
+- `NotebookE2eMain` calls each scenario after its class's `@BeforeTest`, with no JUnit, and prints
+  `NOTEBOOK_E2E: n passed, m failed of 20`. `notebookE2eJvm` runs it on the JVM first.
+- `notebookE2eNativeCompile` calls `native-image` from the JDK 25 toolchain (Liberica NIK Full). The
+  configuration Compose and skiko need is `src/desktopTest/native-image`, recorded by
+  `notebookE2eNativeAgent`; rerun that task after a dependency changes.
+- Beside the executable: `python-multiplatform-home/` (the plugin's `stageNativeImagePythonHome`),
+  `site-packages/` (the wheel) and `app/`. The image finds them there with no environment.
+- `notebookE2eNativeRun` removes `JAVA_HOME` and `PYTHONHOME`, requires 20 passed, and fails if
+  `otool -L` shows `libjvm`.
+- The image takes `ui`, `ui-text` and `foundation` from compose-multiplatform-core-extended's mavenLocal
+  build (`composeExtended.*` in the root `gradle.properties`): upstream Compose sends its main-thread
+  work to skiko's Swing queue, which needs AWT, and AWT needs `libjvm`. The fork sends it to
+  `Dispatchers.Main` under `compose.main.dispatcher=coroutines`, which `NotebookE2eMainDispatcherFactory`
+  supplies (registered by `src/desktopTest/native-image-main/` on the image's classpath only).
+
 ## Scenarios
 
 | Notebook cells | Test | What must hold |

python-multiplatform-compose/fixtures/notebook-e2e/build.gradle.kts

@@ -1,3 +1,5 @@
+import java.io.ByteArrayOutputStream
+import org.apache.tools.ant.util.TeeOutputStream
 import org.jetbrains.kotlin.gradle.ExperimentalKotlinGradlePluginApi
 import org.jetbrains.kotlin.gradle.dsl.JvmTarget
 
@@ -181,3 +183,167 @@ tasks.named<Test>("desktopTest") {
     }
     enabled = !notebookSkipped
 }
+
+// ---- The notebook E2E in a GraalVM native image (issue #249, #191's first milestone) ----------
+//
+// `NotebookE2eMain` runs the same 20 scenarios as `desktopTest`, by direct calls, so they can run in a
+// native image. The scenes are offscreen (`ImageComposeScene`), so no window is involved. Built the way
+// `:sample:nativeCompile` builds its image: `native-image` called directly, because the GraalVM Gradle
+// plugin registers no tasks for a Kotlin Multiplatform `jvm()` target. The Compose and Skiko
+// configuration a closed world needs is recorded by `notebookE2eNativeAgent` into
+// `src/desktopTest/native-image` and committed; python-multiplatform's own FFM registrations come
+// from its jar's `META-INF/native-image`.
+
+val e2eTestCompilation = kotlin.targets.getByName("desktop").compilations.getByName("test")
+// `native-image-main/` registers the AWT-free `Dispatchers.Main` (`NotebookE2eMainDispatcher.kt`), first on
+// the path so it is found before kotlinx-coroutines-swing's; only the agent run and the image use it.
+val e2eNativeMainResources = layout.projectDirectory.dir("src/desktopTest/native-image-main")
+// The fork's Compose (#191): its `ui-desktop` sends the scene's main-thread work to `Dispatchers.Main`
+// when it is built with `-Dcompose.awt=false` (`ExtendedMainDispatcher`), where upstream's always uses
+// skiko's `SwingDispatcher`, which starts AWT; and AWT in a native image needs the JDK's `libawt`,
+// which links `libjvm`. So the image takes the modules `composeExtended.modules` names from the fork,
+// at `composeExtended.version` (root gradle.properties); `desktopTest` keeps upstream Compose.
+val composeExtendedVersion = providers.gradleProperty("composeExtended.version").orNull
+// The fork's group prefix: `org.jetbrains.compose` (its old coordinates) or its own, e.g.
+// `org.thisisthepy.compose`; a module `org.jetbrains.compose.ui:ui-desktop` is then taken as
+// `<prefix>.ui:ui-desktop`.
+val composeExtendedGroupPrefix = providers.gradleProperty("composeExtended.groupPrefix").orNull ?: "org.jetbrains.compose"
+val composeExtendedModules = providers.gradleProperty("composeExtended.modules").orNull
+    ?.split(',')?.map { it.trim() }?.filter { it.isNotEmpty() }?.toSet().orEmpty()
+if (providers.gradleProperty("composeExtended.mavenLocal").orNull?.toBoolean() == true && composeExtendedVersion != null) {
+    // A project's own repositories replace the settings' ones, so those are repeated here.
+    repositories {
+        mavenLocal {
+            mavenContent { includeVersionByRegex(Regex.escape(composeExtendedGroupPrefix) + ".*", ".*", Regex.escape(composeExtendedVersion)) }
+        }
+        google {
+            mavenContent {
+                includeGroupAndSubgroups("androidx")
+                includeGroupAndSubgroups("com.android")
+                includeGroupAndSubgroups("com.google")
+            }
+        }
+        mavenCentral()
+    }
+}
+val e2eNativeRuntime: Configuration = configurations.getByName("desktopTestRuntimeClasspath").copyRecursive().apply {
+    resolutionStrategy.eachDependency {
+        if (composeExtendedVersion != null && "${requested.group}:${requested.name}" in composeExtendedModules) {
+            useTarget("${requested.group.replaceFirst("org.jetbrains.compose", composeExtendedGroupPrefix)}:${requested.name}:$composeExtendedVersion")
+            because("compose-multiplatform-core-extended: an AWT-free main dispatcher for a native image (#249)")
+        }
+    }
+}
+// The copied configuration carries the dependencies only; the main compilation's output and the
+// walker's composable thunks (generated bytecode, not a dependency) are added as they are for desktopTest.
+val e2eMainCompilation = kotlin.targets.getByName("desktop").compilations.getByName("main")
+val e2eClasspath = files(
+    e2eNativeMainResources,
+    e2eTestCompilation.output.allOutputs,
+    e2eMainCompilation.output.allOutputs,
+    layout.buildDirectory.dir("generated/pythonComposableThunks/desktopMain"),
+    e2eNativeRuntime,
+)
+val e2eMainClass = "fixture.notebook.NotebookE2eMainKt"
+val e2eNativeConfig = layout.projectDirectory.dir("src/desktopTest/native-image")
+val e2eNativeDir = layout.buildDirectory.dir("native/notebook-e2e")
+val e2eAppDir = layout.projectDirectory.dir("src/desktopTest/python")
+val e2eJdk25 = javaToolchains.launcherFor { languageVersion.set(JavaLanguageVersion.of(25)) }
+
+fun JavaExec.notebookE2eOnTheJvm() {
+    dependsOn(e2eTestCompilation.compileTaskProvider, installPythonxComposeWheel)
+    classpath = e2eClasspath
+    mainClass.set(e2eMainClass)
+    javaLauncher.set(e2eJdk25)
+    jvmArgs("--enable-native-access=ALL-UNNAMED", "-Djava.awt.headless=true", "-Dcompose.awt=false")
+    systemProperty("notebookE2e.appDir", e2eAppDir.asFile.absolutePath)
+    systemProperty("notebookE2e.sitePackages", pythonxComposeSitePackages.get().asFile.absolutePath)
+    pythonxComposeWheel?.let { systemProperty("notebookE2e.wheel", it.absolutePath) }
+    pythonxComposeWheelError?.let { systemProperty("notebookE2e.wheelError", it) }
+}
+
+tasks.register<JavaExec>("notebookE2eJvm") {
+    group = "verification"
+    description = "Runs NotebookE2eMain on the JVM: the 20 scenarios outside JUnit, before a native image is built from it."
+    notebookE2eOnTheJvm()
+}
+
+tasks.register<JavaExec>("notebookE2eNativeAgent") {
+    group = "native"
+    description = "Runs NotebookE2eMain under native-image's tracing agent and writes the configuration it records into src/desktopTest/native-image."
+    notebookE2eOnTheJvm()
+    doFirst { e2eNativeConfig.asFile.mkdirs() }
+    jvmArgs("-agentlib:native-image-agent=config-output-dir=${e2eNativeConfig.asFile.absolutePath}")
+}
+
+pythonBindings {
+    nativeImageDirectory.set(e2eNativeDir)
+}
+
+val notebookE2eNativeStage by tasks.registering(Sync::class) {
+    group = "native"
+    description = "Puts the pythonx-compose wheel and the notebook's app beside the native image."
+    dependsOn(installPythonxComposeWheel)
+    from(pythonxComposeSitePackages) { into("site-packages") }
+    from(e2eAppDir) { into("app") }
+    into(layout.buildDirectory.dir("native/notebook-e2e-files"))
+}
+
+val notebookE2eNativeCompile by tasks.registering(Exec::class) {
+    group = "native"
+    description = "Builds the notebook E2E as a GraalVM native image (build/native/notebook-e2e/notebook-e2e)."
+    dependsOn(e2eTestCompilation.compileTaskProvider, "stageNativeImagePythonHome", notebookE2eNativeStage)
+    inputs.files(e2eClasspath)
+    inputs.dir(e2eNativeConfig)
+    outputs.file(e2eNativeDir.map { it.file("notebook-e2e") })
+    val launcher = e2eJdk25
+    val outDir = e2eNativeDir
+    val staged = layout.buildDirectory.dir("native/notebook-e2e-files")
+    doFirst { outDir.get().asFile.mkdirs() }
+    executable = launcher.get().metadata.installationPath.asFile.resolve("bin/native-image").absolutePath
+    args(
+        "-cp", e2eClasspath.asPath,
+        "--no-fallback",
+        "--enable-native-access=ALL-UNNAMED",
+        "-Djava.awt.headless=true",
+        // The fork's switch: `ExtendedAwt.available` is a constant false, so nothing reaches AWT.
+        "-Dcompose.awt=false",
+        "-H:ConfigurationFileDirectories=${e2eNativeConfig.asFile.absolutePath}",
+        "-o", outDir.get().asFile.resolve("notebook-e2e").absolutePath,
+        e2eMainClass,
+    )
+    doLast {
+        project.copy { from(staged); into(outDir) }
+    }
+}
+
+tasks.register<Exec>("notebookE2eNativeRun") {
+    group = "verification"
+    description = "Runs the native image on its own (no JAVA_HOME, no PYTHONHOME) and requires 20 of 20 scenarios to pass and no libjvm."
+    dependsOn(notebookE2eNativeCompile)
+    val dir = e2eNativeDir
+    workingDir(dir)
+    executable = dir.get().asFile.resolve("notebook-e2e").absolutePath
+    environment.remove("JAVA_HOME")
+    environment.remove("PYTHONHOME")
+    val captured = ByteArrayOutputStream()
+    standardOutput = TeeOutputStream(System.out, captured)
+    isIgnoreExitValue = true
+    doLast {
+        val out = captured.toString()
+        val summary = out.lineSequence().lastOrNull { it.startsWith("NOTEBOOK_E2E:") }
+            ?: throw GradleException("the native image printed no NOTEBOOK_E2E summary")
+        if (summary != "NOTEBOOK_E2E: 20 passed, 0 failed of 20") throw GradleException("notebook E2E in the native image: $summary")
+        val otool = ProcessBuilder("otool", "-L", executable).redirectErrorStream(true).start()
+        val linked = otool.inputStream.bufferedReader().readText().also { otool.waitFor() }
+        if ("libjvm" in linked) throw GradleException("the native image links libjvm:\n$linked")
+        logger.lifecycle("notebook E2E native image: $summary; no libjvm")
+    }
+}
+
+tasks.register("notebookE2eNativeClasspath") {
+    group = "native"
+    description = "Lists the Compose artefacts the notebook E2E's native image is built from."
+    val files = e2eNativeRuntime
+    doLast { files.files.map { it.name }.filter { it.startsWith("ui") || it.startsWith("foundation") || it.startsWith("skiko") }.sorted().forEach { println(it) } }
+}

python-multiplatform-compose/fixtures/notebook-e2e/src/desktopTest/kotlin/fixture/notebook/NotebookE2eMain.kt

@@ -0,0 +1,74 @@
+package fixture.notebook
+
+import java.io.File
+import kotlin.system.exitProcess
+
+/**
+ * The notebook E2E as one executable (issue #249, #191's first milestone): the same 20 scenarios the
+ * `desktopTest` suite runs, called directly, so they can run inside a GraalVM native image, where
+ * there is no JUnit and no reflection to find a test with.
+ *
+ * Each scenario is its class's `@BeforeTest` (`host()`) and then the test method, exactly what JUnit
+ * does for these classes (none has an `@AfterTest`). A scenario passes when it returns and fails when
+ * it throws; the summary line is what `notebookE2eNativeRun` checks, and the exit status is non-zero
+ * on any failure.
+ *
+ * The wheel's `site-packages` and the application directory are the system properties the suite
+ * reads (`NotebookHost`); when they are not set -- a native image started by hand -- they are
+ * `site-packages/` and `app/` beside the executable, where the build stages them.
+ */
+fun main() {
+    // The fork's switch (compose-multiplatform-core-extended `ExtendedMainDispatcher`): Compose's
+    // main-thread work goes to `Dispatchers.Main`, which `NotebookE2eMainDispatcherFactory` supplies
+    // with no AWT, instead of skiko's Swing event queue. Read once, before Compose starts; upstream
+    // Compose ignores it.
+    System.setProperty("compose.main.dispatcher", "coroutines")
+    besideTheExecutable("notebookE2e.sitePackages", "site-packages")
+    besideTheExecutable("notebookE2e.appDir", "app")
+
+    val scenarios: List<Pair<String, () -> Unit>> = listOf(
+        "cell05_theImportsResolveFromTheInstalledWheel" to { NotebookRootTest().run { host(); cell05_theImportsResolveFromTheInstalledWheel() } },
+        "cell06to07_importingMainDrawsTheAppItDeclares" to { NotebookRootTest().run { host(); cell06to07_importingMainDrawsTheAppItDeclares() } },
+        "cell09to13_theNotebookAndTheScreenShareOneState" to { NotebookRootTest().run { host(); cell09to13_theNotebookAndTheScreenShareOneState() } },
+        "cell15to16_redeclaringTheRootChangesTheScreenWithNoUpdateCall" to { NotebookRootTest().run { host(); cell15to16_redeclaringTheRootChangesTheScreenWithNoUpdateCall() } },
+        "negative_withAppStubbedOutTheRedeclarationDoesNotReachTheScreen" to { NotebookRootTest().run { host(); negative_withAppStubbedOutTheRedeclarationDoesNotReachTheScreen() } },
+        "negative_aHostHandedTheRootOnceDoesNotFollowARedeclaration" to { NotebookRootTest().run { host(); negative_aHostHandedTheRootOnceDoesNotFollowARedeclaration() } },
+        "cell19_text" to { NotebookPracticeTest().run { host(); cell19_text() } },
+        "cell20_textWithAFontSize" to { NotebookPracticeTest().run { host(); cell20_textWithAFontSize() } },
+        "cell22_anEmptyBlackButton" to { NotebookPracticeTest().run { host(); cell22_anEmptyBlackButton() } },
+        "cell23_aButtonHoldingText" to { NotebookPracticeTest().run { host(); cell23_aButtonHoldingText() } },
+        "cell25_aCard" to { NotebookPracticeTest().run { host(); cell25_aCard() } },
+        "cell28_defaultIconsAddIsAnImageVector" to { NotebookPracticeTest().run { host(); cell28_defaultIconsAddIsAnImageVector() } },
+        "cell29_anIcon" to { NotebookPracticeTest().run { host(); cell29_anIcon() } },
+        "cell30_anIconInsideAButton" to { NotebookPracticeTest().run { host(); cell30_anIconInsideAButton() } },
+        "cell35_aColumnOfTwoButtons" to { NotebookPracticeTest().run { host(); cell35_aColumnOfTwoButtons() } },
+        "cell36_aColumnAlignedToTheEnd" to { NotebookPracticeTest().run { host(); cell36_aColumnAlignedToTheEnd() } },
+        "cell37_aRowWithAnArrangement" to { NotebookPracticeTest().run { host(); cell37_aRowWithAnArrangement() } },
+        "cell38_aRow" to { NotebookPracticeTest().run { host(); cell38_aRow() } },
+        "cell40_aCardSpacedBySpacers" to { NotebookPracticeTest().run { host(); cell40_aCardSpacedBySpacers() } },
+        "cell32to33_aTextFieldKeepsInputMethodCompositionInComposeAndTheNotebookReadsTheResult" to {
+            NotebookTextFieldTest().run { host(); cell32to33_aTextFieldKeepsInputMethodCompositionInComposeAndTheNotebookReadsTheResult() }
+        },
+    )
+
+    var failed = 0
+    for ((name, scenario) in scenarios) {
+        try {
+            scenario()
+            println("PASS $name")
+        } catch (t: Throwable) {
+            failed++
+            println("FAIL $name: $t")
+            t.printStackTrace(System.out)
+        }
+    }
+    println("NOTEBOOK_E2E: ${scenarios.size - failed} passed, $failed failed of ${scenarios.size}")
+    System.out.flush()
+    exitProcess(if (failed == 0) 0 else 1)
+}
+
+private fun besideTheExecutable(property: String, directory: String) {
+    if (System.getProperty(property) != null) return
+    val executable = ProcessHandle.current().info().command().orElse(null) ?: return
+    System.setProperty(property, File(File(executable).absoluteFile.parentFile, directory).path)
+}