Repository navigation
forkserver.set_forkserver_preload() silent preload import failures when sys.path is required #117378
Description
Bug report
Bug description:
Bug in def main(): https://github.com/python/cpython/blob/main/Lib/multiprocessing/forkserver.py#L167
The param sys_path is ignored. Result: ModuleNotFoundError for preloaded modules.
a) Using sys_path fixes this issue
b) Maybe better remove "pass" and report and error to simplify problem solving
if preload:
if '__main__' in preload and main_path is not None:
process.current_process()._inheriting = True
try:
spawn.import_main_path(main_path)
finally:
del process.current_process()._inheriting
+ if sys_path:
+ sys.path = sys_path
for modname in preload:
try:
__import__(modname)
except ImportError:
- pass
+ warnings.warn('forkserver: preloading module failed %s' % modname)CPython versions tested on:
3.12
Operating systems tested on:
No response
Linked PRs
- gh-117378: Fix multiprocessing forkserver preload sys.path inheritance. #126538
- [3.13] gh-117378: Fix multiprocessing forkserver preload sys.path inheritance. (GH-126538) #126632
- [3.12] gh-117378: Fix multiprocessing forkserver preload sys.path inheritance. (GH-126538) #126633
- gh-117378: Clear up the NEWS entry wording #126634
- gh-117378: Only run the new multiprocessing SysPath test when appropriate #126635
- [3.13] gh-117378: Only run the new multiprocessing SysPath test when appropriate (GH-126635) #126652
- [3.12] gh-117378: Only run the new multiprocessing SysPath test when appropriate (GH-126635) #126653
- [3.13] gh-117378: Clear up the NEWS entry wording (GH-126634) #126668
- [3.12] gh-117378: Clear up the NEWS entry wording (GH-126634) #126669
Metadata
Metadata
Assignees
Labels
Projects
- Status
Activity
pythongh-117378: Fix multiprocessing forkserver preload sys.path inhe…
gpshead commentedon Nov 7, 2024
Thanks! This is an ironic bug. It has been there from the start for the forkserver. I suspect in a lot of environments it just didn't matter as the default path or PYTHONPATH from the environment was correct. But I can easily imagine situations where this would defeat the purpose of preloading or even potentially cause subtle bugs based on "" vs the absolute path determined at parent multiprocessing import time being in that preload's sys.path.
doublex commentedon Nov 7, 2024
@gpshead
Thank you very much for your efforts!
gh-117378: Fix multiprocessing forkserver preload sys.path inheritanc…
pythongh-117378: Fix multiprocessing forkserver preload sys.path inhe…
[3.13] gh-117378: Fix multiprocessing forkserver preload sys.path inh…
[3.12] pythongh-117378: Fix multiprocessing forkserver preload sys.pa…
6 remaining items
[3.12] gh-117378: Only run the new multiprocessing SysPath test when …
[3.13] gh-117378: Only run the new multiprocessing SysPath test when …
gh-117378: Clear up the NEWS entry wording (GH-126634)
pythongh-117378: Clear up the NEWS entry wording (pythonGH-126634)
pythongh-117378: Clear up the NEWS entry wording (pythonGH-126634)
[3.13] gh-117378: Clear up the NEWS entry wording (GH-126634) (#126668)
[3.12] gh-117378: Clear up the NEWS entry wording (GH-126634) (#126669)
pythongh-117378: Fix multiprocessing forkserver preload sys.path inhe…
pythongh-117378: Only run the new multiprocessing SysPath test when a…
pythongh-117378: Clear up the NEWS entry wording (pythonGH-126634)
pythongh-117378: Fix multiprocessing forkserver preload sys.path inhe…
pythongh-117378: Only run the new multiprocessing SysPath test when a…
pythongh-117378: Clear up the NEWS entry wording (pythonGH-126634)