Repository navigation
REPL: AttributeError: module __mp_main__ has no attribute is_prime in ProcessPoolExecutor example #132898
Description
Bug report
Bug description:
python version:3.12.9
An error occurred when running as a standalone script.
An error message is reported when running the sample code of the python document:
Traceback (most recent call last):
File "C:\Users\wen\Desktop\test.py", line 32, in
main()
File "C:\Users\wen\Desktop\test.py", line 28, in main
for number, prime in zip(PRIMES, executor.map(is_prime, PRIMES)):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Miniconda3\Lib\concurrent\futures\process.py", line 636, in _chain_from_iterable_of_lists
for element in iterable:
^^^^^^^^
File "C:\Miniconda3\Lib\concurrent\futures_base.py", line 619, in result_iterator
yield _result_or_cancel(fs.pop())
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Miniconda3\Lib\concurrent\futures_base.py", line 317, in _result_or_cancel
return fut.result(timeout)
^^^^^^^^^^^^^^^^^^^
File "C:\Miniconda3\Lib\concurrent\futures_base.py", line 456, in result
return self.__get_result()
^^^^^^^^^^^^^^^^^^^
File "C:\Miniconda3\Lib\concurrent\futures_base.py", line 401, in __get_result
raise self._exception
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.
import concurrent.futures
import math
PRIMES = [
112272535095293,
112582705942171,
112272535095293,
115280095190773,
115797848077099,
1099726899285419]
def is_prime(n):
if n < 2:
return False
if n == 2:
return True
if n % 2 == 0:
return False
sqrt_n = int(math.floor(math.sqrt(n)))
for i in range(3, sqrt_n + 1, 2):
if n % i == 0:
return False
return True
def main():
with concurrent.futures.ProcessPoolExecutor() as executor:
for number, prime in zip(PRIMES, executor.map(is_prime, PRIMES)):
print('%d is prime: %s' % (number, prime))
if __name__ == '__main__':
main()
CPython versions tested on:
3.12
Operating systems tested on:
Windows 10
Linked PRs
- gh-132898: Add a note in
multiprocessing.Processdocs about creating a process in a REPL session #137118 - [3.14] gh-132898: Add a note in
multiprocessing.Processdocs about creating a process in a REPL (GH-137118) #137154 - [3.13] gh-132898: Add a note in
multiprocessing.Processdocs about creating a process in a REPL (GH-137118) #137155
Metadata
Metadata
Assignees
Labels
Projects
- Status
Activity
picnixz commentedon Apr 25, 2025
Question: did this occur because you hit Ctrl+C or killed the process somehow?Nevermind, it occurred on Linux as well with just running the example.
[-]A process in the process pool was terminated abruptly while the future was running or pending[/-][+]AttributeError: module `__mp_main__` has no attribute `is_prime` in `ProcessPoolExecutor` example[/+][-]AttributeError: module `__mp_main__` has no attribute `is_prime` in `ProcessPoolExecutor` example[/-][+]REPL: AttributeError: module `__mp_main__` has no attribute `is_prime` in `ProcessPoolExecutor` example[/+]picnixz commentedon Apr 25, 2025
Ok, I only found the issue when the above code is being tested inside the REPL and not as a script (namely
python test.pyshould work but not when I C/C the above code). The best fix for now would be to just indicate in the docs "run it as a standalone script, not inside the REPL", but I don't know the exact reason why (well I can guess: it has something with pickling and probably globals that are not made available because they are only available in the REPL andProcessPoolExecutoris not able to find the correct "main" module).cc @pablogsal @gpshead
picnixz commentedon Apr 25, 2025
@wenming21 Your latest edit indicates that the error also occurs when running the example as a standalone script. What is the full traceback? (namely, what's above the "Traceback (most recent call last):"? because I do have other stuff written when I encounter the bug)
StanFromIreland commentedon Apr 25, 2025
Why was this closed? @wenming21
StanFromIreland commentedon Apr 25, 2025
I can also add that this only occurs on the main branch, on 3.13 it works fine.
picnixz commentedon Apr 25, 2025
3.14 has seen changes with the default start method so this could be the cause by the way (at least on Linux)
YvesDup commentedon Apr 30, 2025
What's happen when you change the
start_method?I am on Mac, with a 3.12.8 version and I ran that script:
/bin/python3 -i /Users/yves/Desktop/Bugs/gh-132898.pyand in REPL, I callmain(), I have got errors:Ouptput
if I changed the
start_methodtoforkserver, I have got the same errors.And finally if the start method is
forkeverything is fine./usr/local/bin/python3 -i /Users/yves/Desktop/Bugs/gh-132898.pyIMO, this issue is also causing by
start_methodbut I don't figure out how REPL is involved.And a last remark, if you replace the
is_primefunction withmath.sqrt, the script succeeds in anystart_methodcase .27 remaining items
Alter chaos multiprocessing start method
Alter chaos multiprocessing start method
Alter chaos multiprocessing start method (#3149)
Alter chaos multiprocessing start method (RedHatQE#3149)
Alter chaos multiprocessing start method (RedHatQE#3149)
fix: move multiprocessing_start_method_fork fixture to shared conftest
fix: move multiprocessing_start_method_fork fixture to shared conftest
fix: move multiprocessing_start_method_fork fixture to shared conftest
fix: move multiprocessing_start_method_fork fixture to shared conftest
fix: move multiprocessing_start_method_fork fixture to shared conftest
fix: move multiprocessing_start_method_fork fixture to shared conftest
fix: move multiprocessing_start_method_fork fixture to shared conftest
fix: move multiprocessing_start_method_fork fixture to shared conftest
Alter chaos multiprocessing start method (RedHatQE#3149)
Alter chaos multiprocessing start method (RedHatQE#3149)