Repository navigation
test_dynamic fails on wasm32 WASI 8Core 3.x buildbot #117645
Description
Example of recent failing build: https://buildbot.python.org/all/#/builders/1344/builds/1372
build:
0:00:42 load avg: 4.04 [ 50/473/1] test_dynamic worker non-zero exit code (Exit code 134)
test_cannot_change_globals_or_builtins_with_eval (test.test_dynamic.RebindBuiltinsTests.test_cannot_change_globals_or_builtins_with_eval) ... ok
test_cannot_change_globals_or_builtins_with_exec (test.test_dynamic.RebindBuiltinsTests.test_cannot_change_globals_or_builtins_with_exec) ... ok
test_cannot_replace_builtins_dict_between_calls (test.test_dynamic.RebindBuiltinsTests.test_cannot_replace_builtins_dict_between_calls) ... ok
test_cannot_replace_builtins_dict_while_active (test.test_dynamic.RebindBuiltinsTests.test_cannot_replace_builtins_dict_while_active) ... ok
test_eval_gives_lambda_custom_globals (test.test_dynamic.RebindBuiltinsTests.test_eval_gives_lambda_custom_globals) ... ok
test_globals_shadow_builtins (test.test_dynamic.RebindBuiltinsTests.test_globals_shadow_builtins) ... ok
test_load_global_specialization_failure_keeps_oparg (test.test_dynamic.RebindBuiltinsTests.test_load_global_specialization_failure_keeps_oparg) ...
Error: failed to run main module `python.wasm`
Caused by:
0: failed to invoke command default
1: error while executing at wasm backtrace:
0: 0x1af790 - <unknown>!compiler_visit_expr
1: 0x1afaf9 - <unknown>!compiler_visit_expr
2: 0x1afaf9 - <unknown>!compiler_visit_expr
3: 0x1afaf9 - <unknown>!compiler_visit_expr
(...)
339: 0x1afaf9 - <unknown>!compiler_visit_expr
340: 0x1afe9e - <unknown>!compiler_visit_expr
341: 0x1a8f50 - <unknown>!compiler_codegen
342: 0x1a8873 - <unknown>!_PyAST_Compile
343: 0x20f198 - <unknown>!run_mod
344: 0x20f8b2 - <unknown>!PyRun_StringFlags
345: 0x1876d4 - <unknown>!builtin_eval
346: 0xb92f8 - <unknown>!cfunction_vectorcall_FASTCALL
347: 0x60876 - <unknown>!PyObject_Vectorcall
348: 0x1a125a - <unknown>!_PyEval_EvalFrameDefault
349: 0x1a1d21 - <unknown>!_PyEval_Vector
(...)
407: 0x233859 - <unknown>!Py_BytesMain
408: 0x5967 - <unknown>!main
409: 0x3ca4e3 - <unknown>!__main_void
410: 0x5940 - <unknown>!_start
note: using the `WASMTIME_BACKTRACE_DETAILS=1` environment variable may show more debugging information
2: memory fault at wasm address 0x1000005b0 in linear memory of size 0xa00000
3: wasm trap: out of bounds memory access
test.pythoninfo:
_testcapi.LONG_MAX: 2147483647
_testcapi.Py_C_RECURSION_LIMIT: 500
build.NDEBUG: ignore assertions (macro defined)
build.Py_DEBUG: No (sys.gettotalrefcount() missing)
sysconfig[HOSTRUNNER]: wasmtime run --wasm max-wasm-stack=8388608 --wasi preview2 --env PYTHONPATH=/$(shell realpath --relative-to /opt/buildbot/kushaldas-wasm/3.x.kushaldas-wasi.wasi.nondebug/build/build_oot/host/../.. /opt/buildbot/kushaldas-wasm/3.x.kushaldas-wasi.wasi.nondebug/build/build_oot/host)/$(shell cat pybuilddir.txt):/Lib --dir ../..::/
sysconfig[OPT]: -DNDEBUG -g -O3 -Wall
sysconfig[PY_CFLAGS]: -fno-strict-overflow -Wsign-compare -Wunreachable-code -DNDEBUG -g -O3 -Wall
sysconfig[PY_CFLAGS_NODIST]: -std=c11 -Wextra -Wno-unused-parameter -Wno-int-conversion -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden -I../../Include/internal -I../../Include/internal/mimalloc
sysconfig[PY_CORE_LDFLAGS]: -z stack-size=524288 -Wl,--stack-first -Wl,--initial-memory=10485760
sysconfig[PY_LDFLAGS_NODIST]: -z stack-size=524288 -Wl,--stack-first -Wl,--initial-memory=10485760
sysconfig[PY_STDMODULE_CFLAGS]: -fno-strict-overflow -Wsign-compare -Wunreachable-code -DNDEBUG -g -O3 -Wall -std=c11 -Wextra -Wno-unused-parameter -Wno-int-conversion -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden -I../../Include/internal -I../../Include/internal/mimalloc -IObjects -IInclude -IPython -I. -I../../Include
Python is built with 512 KiB stack and run with 8 MiB stack and Py_C_RECURSION_LIMIT=500.
Activity
vstinner commentedon Apr 8, 2024
Latest successful build (4 days ago):
060a96f1a9a901b01ed304aa82b886d248ca1cb6First failing build:
04697bcfaf5dd34c9312f4f405083b6d33b3511fSo the buildbot starts failing since the change: commit 04697bc
Maybe it's just that this commit changed how many stack memory Python uses on the WASI build.
pythongh-117645: Skip test_dynamic global specialization on WASI
pythongh-117645: Skip test_dynamic global specialization on WASI
pythongh-117645: Skip test_dynamic global specialization on WASI
gh-117645: Skip test_dynamic global specialization on WASI (#117646)
pythongh-117645: Increase WASI stack size from 512 KiB to 8 MiB
pythongh-117645: Increase WASI stack size from 512 KiB to 8 MiB
pythongh-117645: Increase WASI stack size from 512 KiB to 8 MiB
gh-117645: Increase WASI stack size from 512 KiB to 8 MiB (#117674)
pythongh-117645: Skip test_dynamic global specialization on WASI (pyt…
pythongh-117645: Increase WASI stack size from 512 KiB to 8 MiB (pyth…
markshannon commentedon May 3, 2024
WASI CI jobs still seem rather fragile: https://github.com/python/cpython/actions/runs/8937187388/job/24548976147?pr=118322
vstinner commentedon May 3, 2024
test_glob failed on this job. Please open a new issue. This issue was about test_dynamic.
markshannon commentedon May 3, 2024
test_glob or test_dynamic doesn't matter. The issue is the WASI stack depth, it has little to do with the particular tests that fail.
vstinner commentedon May 3, 2024
The error message is different.
markshannon commentedon May 3, 2024
I suspect that is just an artifact of the emscriptem compiler and where it puts large
autovariables. The root problem is the same.vstinner commentedon May 3, 2024
I cannot reproduce the test_glob issue on the main branch or on the your #118322 PR. You should open an issue with more details explaining how to reprodudce the issue, if Python was built in debug or release mode, etc.