You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Support a pthread and dynamic link build for wasm #96254
The build rule is omitted on purpose. Last time I checked builds with dynamic linking and pthread proxying were unstable and crashed. Did you check that the target is passing all tests?
I'm using this build rule to try and get https://github.com/Microsoft/DEBUGPY working in some form of a WASM build. Debugpy requires dynamic linking and pthreads.
That comment is outdated already :). wasm64-emscripten is working and all tests are passing -- or were passing Emscripten broke two days ago. Sam is working on a fix, emscripten-core/emscripten#17720
test_capi test_code test_imp test_importlib test_xxlimited are literally all tests that load a shared extension. The other tests use statically linked extensions. I guess dlopen() is still broken and corrupts memory with threading.
I would be willing to accept a emscripten-node-dl-pthreads-debug if you get it working, report problems to Emscripten project, and work with Emscripten upstream to get the bugs fixed. A non-debug build is ok once tests are passing and stable.
For the record, pthreads and dynamic linking are experimental and currently broken for CPython. Every attempt to load a shared extension module leads to an immediate crash:
emcc: warning: -sMAIN_MODULE + pthreads is experimental [-Wexperimental]
Error [RuntimeError]: memory access out of bounds
at PyType_IsSubtype (<anonymous>:wasm-function[2366]:0x1fd5c4)
at _PyImport_LoadDynamicModuleWithSpec (<anonymous>:wasm-function[3777]:0x2c0f80)
at _imp_create_dynamic (<anonymous>:wasm-function[3770]:0x2c076e)
at cfunction_vectorcall_FASTCALL (<anonymous>:wasm-function[1987]:0x1e9a8d)
at _PyVectorcall_Call (<anonymous>:wasm-function[880]:0x19c433)
at _PyObject_Call (<anonymous>:wasm-function[883]:0x19c754)
at PyObject_Call (<anonymous>:wasm-function[884]:0x19c801)
at _PyEval_EvalFrameDefault (<anonymous>:wasm-function[3269]:0x2855b1)
at _PyEval_Vector (<anonymous>:wasm-function[3270]:0x28b334)
at _PyFunction_Vectorcall (<anonymous>:wasm-function[887]:0x19c976)
Emitted 'error' event on process instance at:
at emitUnhandledRejectionOrErr (internal/event_target.js:579:11)
at MessagePort.[nodejs.internal.kHybridDispatch] (internal/event_target.js:403:9)
at MessagePort.exports.emitMessage (internal/per_context/messageport.js:18:26)
RuntimeError: invalid index into function table
at module_dealloc (<anonymous>:wasm-function[2027]:0x1eb420)
at _Py_Dealloc (<anonymous>:wasm-function[2113]:0x1ef34b)
at PyModule_FromDefAndSpec2 (<anonymous>:wasm-function[2015]:0x1eabfa)
at _PyImport_LoadDynamicModuleWithSpec (<anonymous>:wasm-function[3777]:0x2c0fdf)
at _imp_create_dynamic (<anonymous>:wasm-function[3770]:0x2c076e)
at cfunction_vectorcall_FASTCALL (<anonymous>:wasm-function[1987]:0x1e9a8d)
at _PyVectorcall_Call (<anonymous>:wasm-function[880]:0x19c433)
at _PyObject_Call (<anonymous>:wasm-function[883]:0x19c754)
at PyObject_Call (<anonymous>:wasm-function[884]:0x19c801)
at _PyEval_EvalFrameDefault (<anonymous>:wasm-function[3269]:0x2855b1)
Pthread 0x008c0948 sent an error! undefined:undefined: invalid index into function table
/home/heimes/dev/python/cpython/builddir/emscripten-node-dl-pthreads-debug/python.js:156
throw ex;
^
Error [RuntimeError]: invalid index into function table
at module_dealloc (<anonymous>:wasm-function[2027]:0x1eb420)
at _Py_Dealloc (<anonymous>:wasm-function[2113]:0x1ef34b)
at PyModule_FromDefAndSpec2 (<anonymous>:wasm-function[2015]:0x1eabfa)
at _PyImport_LoadDynamicModuleWithSpec (<anonymous>:wasm-function[3777]:0x2c0fdf)
at _imp_create_dynamic (<anonymous>:wasm-function[3770]:0x2c076e)
at cfunction_vectorcall_FASTCALL (<anonymous>:wasm-function[1987]:0x1e9a8d)
at _PyVectorcall_Call (<anonymous>:wasm-function[880]:0x19c433)
at _PyObject_Call (<anonymous>:wasm-function[883]:0x19c754)
at PyObject_Call (<anonymous>:wasm-function[884]:0x19c801)
at _PyEval_EvalFrameDefault (<anonymous>:wasm-function[3269]:0x2855b1)
Emitted 'error' event on process instance at:
at emitUnhandledRejectionOrErr (internal/event_target.js:579:11)
at MessagePort.[nodejs.internal.kHybridDispatch] (internal/event_target.js:403:9)
at MessagePort.exports.emitMessage (internal/per_context/messageport.js:18:26)
Activity
tiran commentedon Aug 25, 2022
Thanks for your interest. 👍
The build rule is omitted on purpose. Last time I checked builds with dynamic linking and pthread proxying were unstable and crashed. Did you check that the target is passing all tests?
rchiodo commentedon Aug 25, 2022
There are some tests that fail (5 out of 435) but I thought this is all experimental anyway?
Here's the failing tests:
I'm using this build rule to try and get https://github.com/Microsoft/DEBUGPY working in some form of a WASM build. Debugpy requires dynamic linking and pthreads.
rchiodo commentedon Aug 25, 2022
Would it be possible to add it and put a comment like this one?
cpython/Tools/wasm/wasm_build.py
Line 492 in caa2a97
tiran commentedon Aug 25, 2022
That comment is outdated already :). wasm64-emscripten is working and all tests are passing -- or were passing Emscripten broke two days ago. Sam is working on a fix, emscripten-core/emscripten#17720
test_capi test_code test_imp test_importlib test_xxlimitedare literally all tests that load a shared extension. The other tests use statically linked extensions. I guess dlopen() is still broken and corrupts memory with threading.I would be willing to accept a
emscripten-node-dl-pthreads-debugif you get it working, report problems to Emscripten project, and work with Emscripten upstream to get the bugs fixed. A non-debug build is ok once tests are passing and stable.tiran commentedon Aug 29, 2022
For the record, pthreads and dynamic linking are experimental and currently broken for CPython. Every attempt to load a shared extension module leads to an immediate crash:
emcc: warning: -sMAIN_MODULE + pthreads is experimental [-Wexperimental]brettcannon commentedon Jun 8, 2023
Closing as a duplicate of #101538 .