Skip to content

Binder: pin virtual threads in withGIL; crossinline block (#161 option B) - #337

Merged
b-re-w merged 1 commit into
developfrom
feat/withgil-crossinline-pinned
Oct 25, 2026
Merged

b-re-w merged 1 commit into
developfrom
feat/withgil-crossinline-pinned

Conversation

@b-re-w

@b-re-w b-re-w commented Oct 25, 2026

Copy link
Copy Markdown
Member

Closes #161 (option B, user decision; replaces option A's refusal from #303).

  • Public API change: withGIL(crossinline block) and Python3.withPython(crossinline block). A non-local return from those blocks no longer compiles.
  • Breakage list:
    • This repository: no call site broke. Every target compiles: desktop main and test, androidNative, iOS main and test, wasmJs, Android, compose, the ksp/compose/artifact fixtures and sample. The only other change is the private helper PyObject.callWithTuple, whose fill block also became crossinline.
    • Other repositories: no Kotlin use of withGIL or withPython in pythonx-compose, pythonx-concurrent, pythonx-graphics, pythonx-platform, toolchain or torchnative (grep).
  • Pin: on a JVM virtual thread, desktop only, the outermost scope runs as the comparator of C's bsearch over one element. That is an FFM upcall inside a downcall, so the native frame pins the virtual thread from PyGILState_Ensure to Release. An upcall must not throw, so the result or exception is carried out and rethrown. Platform threads and other targets run the block directly.
  • Panama gains createUpcallStubII_I, (long, long) -> int, in both backends, with an @UpcallShape marker; the generated reachability metadata includes it. The bsearch downcall (5 x long -> long) is already a registered descriptor.
  • SPEC T-1a is rewritten.
  • Tests: VirtualThreadGILTest.
    • 16 virtual threads park 5 times each inside withGIL, and pthread_self must stay the same.
    • An exception passes through the pinned scope and the GIL is released.
    • Nesting works, and platform threads are unchanged.
    • Mutant: without the pin, the test JVM aborts (exit 134) in PyGILState_Release on the wrong OS thread, which is the corruption Binder: withGIL on a JVM virtual thread can release the GIL from another carrier #161 described.
    • ReachabilityMetadataTest is updated for the new shape. python-multiplatform desktopTest 644/644; ksp app 80, compose 129, artifact 50.
  • Not verified: running the pin inside a GraalVM native image (rule 13). Only the descriptors are registered; no native-image build was run for this path.
  • pythonx-concurrent: notified on Binder requests from pythonx-concurrent (tracking) #164.

…using it (#161 option B)

withGIL's and withPython's blocks become crossinline (a public API change, user-approved) so the
outermost scope on a virtual thread can run inside a native frame: C's bsearch over one element calls
its comparator, an FFM upcall that runs the scope, so the virtual thread cannot unmount between
PyGILState_Ensure and Release. The result or exception is carried out of the upcall. Platform threads,
Android, native and wasm run the block directly. Panama gains the (long, long) -> int upcall shape;
its reachability descriptor is generated from the @UpcallShape marker.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant