Repository navigation
Calling map on maps many times causes segfault #103503
Description
Crash report
When the following code is ran, It causes a Segmentation Fault:
a = []
for x in range(100_000):
a = map(int, a)
list(a)Now the list call calls Python/bltinmodule.c:map_next.
If we run the code with gdb, we get the following:
Program received signal SIGSEGV, Segmentation fault.
But we also get:
0x00005555557bbd6e in map_next (lz=0x7ffff7099ba0) at Python/bltinmodule.c:1342
To inspect the call stack, we can type where into gdb and we get loads of function calls, all seeming to be coming from line 1363, which is:
PyObject *val = Py_TYPE(it)->tp_iternext(it);And the value of Py_TYPE(it) is a map and it calls map_next which causes the recursion
Error message
Segmentation Fault
Core file
Core file: Google Drive Link
Possible fix
We could add a recursion limit like we have with regular python functions.
Your environment
- CPython versions tested on: 3.9, 3.10, 3.12.0a5+
- Operating system and architecture: Windows & Linux, x86-64
Activity
sunmy2019 commentedon Apr 13, 2023
Looks like a stack overflow.
On my computer, the final call stack is number 65510, very close to 65536, and it seems that one stack frame uses 64 Bytes, which is exactly ~4MB .
This can be verified by increasing the stack size. It works on my computer when I increase the stack size to 100MB.
https://stackoverflow.com/questions/15887787/increase-stack-size-for-a-program-to-run
sunmy2019 commentedon Apr 13, 2023
Calling
__next__is handled in C. Thus only the C stack increases, not the Python stack. I don't think any checks were performed.error-3317 commentedon Apr 13, 2023
Yes this is evident as the call stack is near 65k, as it crashed showing there are no checks involved, we could add a check if it goes over 65k calls and raise a
RecursionErrorlike:pochmann commentedon Apr 13, 2023
Old relevant issue with links to several others and discussion.
error-3317 commentedon Apr 13, 2023
But it has not been solved
pochmann commentedon Apr 13, 2023
Does it need to be solved? Is it a problem? What real code does that? Such deep nesting is highly inefficient, you wouldn't want to do that anyway.
terryjreedy commentedon Apr 13, 2023
@rhettinger This seems similar to the nested itertools issue that you closed (as won't fix).
rhettinger commentedon Apr 14, 2023
@pochmann As you noted, this doesn't seem to arise in practice, so I'm don't worry about it.
If someone really cares about this, this best solution would be for the interpreter to monitor the actual stack usage and respond appropriately when running low on space. This is similar to what people do to monitor free disk space on servers. Implementing this would be real challenge though as it requires substantial expertise with compilers and operating systems. It is in the doable but hard category.
The other challenge would be to define what we mean by "responding appropriately". To get the system back to a usable state, the stack would have to be cleared-off without breaking invariants. The only way I know to do this is by snapshotting. That would be a somewhat heavy handed solution to a problem that almost no user ever has.
carljm commentedon Apr 14, 2023
If anyone wants to pursue stack overflow prevention, it would be a good idea to start with understanding the rejected PEP 651 and the reasons for its rejection. (Note that one reason for rejection was concern about multiple Python call frames in a single C frame, which we already now have.)
error-3317 commentedon Apr 15, 2023
We could do some crude pointer arithmetic
picnixz commentedon Aug 18, 2025
I'm going to close this one as the others were closed. Preventing those cases is probably not worth the cost of slowing down regular cases. Unless a concrete real-world example is impeded by this, similar issues should be closed as well.
markshannon commentedon Jul 29, 2026
We now have fast (a few machine instructions) stack overflow checking available, so let's fix this.
pythongh-103503: add NEWS entry