Segfault from calling StringIO methods in threads on free-threading debug build

@vstinner

Crash report

What happened?

It's possible to segfault or abort the interpreter in a free-threading debug build by calling StringIO methods from threads:

from io import StringIO
from threading import Thread

for _ in range(10):
    alive = []

    sio = StringIO(newline="")
    def call_stringio_methods():
        try:
            sio.write("\x00" * 10 ** 5)
            sio.seek(0)  # Comment this line out for a different segfault
            sio.readlines()
        except Exception:
            pass

    for _ in range(5):
        alive.append(Thread(target=call_stringio_methods))
    for t in alive:
        t.start()

The segfault for this MRE can happen in different places, all seemingly related to _stringio_readline. If you comment out the sio.seek(0) line, a different segfault happens.

Backtrace:

Thread 14 "Thread-13 (call" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffb1ffb640 (LWP 3706081)]
0x00005555560a4b15 in _stringio_readline (self=self@entry=0x7fffb4301090, limit=300000, limit@entry=-1) at ./Modules/_io/stringio.c:370
370         old_char = *end;

#0  0x00005555560a4b15 in _stringio_readline (self=self@entry=0x7fffb4301090, limit=300000, limit@entry=-1) at ./Modules/_io/stringio.c:370
#1  0x00005555560aa0b4 in stringio_iternext (op=<_io.StringIO at remote 0x7fffb4301090>) at ./Modules/_io/stringio.c:418
#2  0x0000555555a5dfc7 in list_extend_iter_lock_held (self=self@entry=0x7fffc0040070,
    iterable=iterable@entry=<_io.StringIO at remote 0x7fffb4301090>) at Objects/listobject.c:1262
#3  0x0000555555a5e6a6 in _list_extend (self=0x7fffc0040070, iterable=<_io.StringIO at remote 0x7fffb4301090>) at Objects/listobject.c:1451
#4  0x0000555555a61168 in list_extend_impl (self=<optimized out>, iterable=<optimized out>) at Objects/listobject.c:1470
#5  0x0000555555a61189 in list_extend (self=<optimized out>, iterable=<optimized out>) at Objects/clinic/listobject.c.h:145
#6  0x00005555559ce5fb in method_vectorcall_O (func=<optimized out>, args=0x7fffb1ff89a0, nargsf=<optimized out>, kwnames=<optimized out>)
    at Objects/descrobject.c:476
#7  0x000055555599674e in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=2, args=0x7fffb1ff89a0,
    callable=<method_descriptor at remote 0x7fffb4201120>, tstate=0x6290000aa210) at ./Include/internal/pycore_call.h:169
#8  object_vacall (tstate=tstate@entry=0x6290000aa210, base=base@entry=[], callable=<optimized out>, vargs=vargs@entry=0x7fffb1ff8ab0)
    at Objects/call.c:819
#9  0x0000555555996b34 in PyObject_CallMethodObjArgs (obj=[], name=<optimized out>) at Objects/call.c:886
#10 0x0000555556055b9f in _io__IOBase_readlines_impl (self=self@entry=<_io.StringIO at remote 0x7fffb4301090>, hint=-1)
    at ./Modules/_io/iobase.c:729
#11 0x000055555605627b in _io__IOBase_readlines (self=<optimized out>, args=0x7fffb1ff93a0, nargs=0) at ./Modules/_io/clinic/iobase.c.h:369
#12 0x00005555559cda46 in method_vectorcall_FASTCALL (func=<optimized out>, args=0x7fffb1ff9398, nargsf=<optimized out>, kwnames=<optimized out>)
    at Objects/descrobject.c:402
#13 0x00005555559956af in _PyObject_VectorcallTstate (tstate=0x6290000aa210, callable=<method_descriptor at remote 0x7fffb45359c0>,
    args=0x7fffb1ff9398, nargsf=9223372036854775809, kwnames=0x0) at ./Include/internal/pycore_call.h:169
#14 0x000055555599580a in PyObject_Vectorcall (callable=callable@entry=<method_descriptor at remote 0x7fffb45359c0>,
    args=args@entry=0x7fffb1ff9398, nargsf=<optimized out>, kwnames=kwnames@entry=0x0) at Objects/call.c:327
#15 0x0000555555d82a18 in _PyEval_EvalFrameDefault (tstate=tstate@entry=0x6290000aa210, frame=0x6290000af3a8, frame@entry=0x6290000af328,
    throwflag=throwflag@entry=0) at Python/generated_cases.c.h:1619
#16 0x0000555555de46bc in _PyEval_EvalFrame (throwflag=0, frame=0x6290000af328, tstate=0x6290000aa210) at ./Include/internal/pycore_ceval.h:119
#17 _PyEval_Vector (tstate=<optimized out>, func=<optimized out>, locals=locals@entry=0x0, args=<optimized out>, argcount=1,
    kwnames=<optimized out>) at Python/ceval.c:1975
#18 0x0000555555994c83 in _PyFunction_Vectorcall (func=<optimized out>, stack=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>)
    at Objects/call.c:413
#19 0x00005555559a0259 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=1, args=0x7fffb1ff9b90, callable=<function at remote 0x7fffb44ab0d0>,
    tstate=0x6290000aa210) at ./Include/internal/pycore_call.h:169
#20 method_vectorcall (method=<optimized out>, args=0x7fffb1ffa388, nargsf=<optimized out>, kwnames=<optimized out>) at Objects/classobject.c:72
#21 0x0000555555e2dd35 in _PyObject_VectorcallTstate (tstate=tstate@entry=0x6290000aa210, callable=<method at remote 0x7fffc0040130>,
    args=args@entry=0x7fffb1ffa388, nargsf=nargsf@entry=0, kwnames=kwnames@entry=0x0) at ./Include/internal/pycore_call.h:169

Similar backtrace:

Thread 15 "Thread-14 (call" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffb37fe640 (LWP 3712552)]
0x00005555560a0d24 in PyUnicode_READ (index=0, data=0x7fffba270000, kind=4) at ./Include/cpython/unicodeobject.h:343
343         return _Py_STATIC_CAST(const Py_UCS4*, data)[index];
(gdb) bt
#0  0x00005555560a0d24 in PyUnicode_READ (index=0, data=0x7fffba270000, kind=4) at ./Include/cpython/unicodeobject.h:343
#1  _PyIO_find_line_ending (translated=<optimized out>, universal=1, readnl='', kind=kind@entry=4, start=start@entry=0x7fffba170010 "",
    end=end@entry=0x7fffba233510 "\315\315\315\315\315\315\315\315", '\375' <repeats 24 times>, '\315' <repeats 168 times>...,
    consumed=0x7fffb37fb330) at ./Modules/_io/textio.c:2139
#2  0x00005555560a4bb4 in _stringio_readline (self=self@entry=0x7fffb4301090, limit=200000, limit@entry=-1) at ./Modules/_io/stringio.c:372
#3  0x00005555560aa0b4 in stringio_iternext (op=<_io.StringIO at remote 0x7fffb4301090>) at ./Modules/_io/stringio.c:418
#4  0x0000555555a5dfc7 in list_extend_iter_lock_held (self=self@entry=0x7fffb60c00d0,
    iterable=iterable@entry=<_io.StringIO at remote 0x7fffb4301090>) at Objects/listobject.c:1262
#5  0x0000555555a5e6a6 in _list_extend (self=0x7fffb60c00d0, iterable=<_io.StringIO at remote 0x7fffb4301090>) at Objects/listobject.c:1451
#6  0x0000555555a61168 in list_extend_impl (self=<optimized out>, iterable=<optimized out>) at Objects/listobject.c:1470
#7  0x0000555555a61189 in list_extend (self=<optimized out>, iterable=<optimized out>) at Objects/clinic/listobject.c.h:145
#8  0x00005555559ce5fb in method_vectorcall_O (func=<optimized out>, args=0x7fffb37fba60, nargsf=<optimized out>, kwnames=<optimized out>)
    at Objects/descrobject.c:476
#9  0x000055555599674e in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=2, args=0x7fffb37fba60,
    callable=<method_descriptor at remote 0x7fffb4201120>, tstate=0x6290000be210) at ./Include/internal/pycore_call.h:169
#10 object_vacall (tstate=tstate@entry=0x6290000be210, base=base@entry=[], callable=<optimized out>, vargs=vargs@entry=0x7fffb37fbb70)
    at Objects/call.c:819
#11 0x0000555555996b34 in PyObject_CallMethodObjArgs (obj=[], name=<optimized out>) at Objects/call.c:886
#12 0x0000555556055b9f in _io__IOBase_readlines_impl (self=self@entry=<_io.StringIO at remote 0x7fffb4301090>, hint=-1)
    at ./Modules/_io/iobase.c:729
#13 0x000055555605627b in _io__IOBase_readlines (self=<optimized out>, args=0x7fffb37fc460, nargs=0) at ./Modules/_io/clinic/iobase.c.h:369
#14 0x00005555559cda46 in method_vectorcall_FASTCALL (func=<optimized out>, args=0x7fffb37fc458, nargsf=<optimized out>, kwnames=<optimized out>)
    at Objects/descrobject.c:402
#15 0x00005555559956af in _PyObject_VectorcallTstate (tstate=0x6290000be210, callable=<method_descriptor at remote 0x7fffb45359c0>,
    args=0x7fffb37fc458, nargsf=9223372036854775809, kwnames=0x0) at ./Include/internal/pycore_call.h:169
#16 0x000055555599580a in PyObject_Vectorcall (callable=callable@entry=<method_descriptor at remote 0x7fffb45359c0>,
    args=args@entry=0x7fffb37fc458, nargsf=<optimized out>, kwnames=kwnames@entry=0x0) at Objects/call.c:327
#17 0x0000555555d82a18 in _PyEval_EvalFrameDefault (tstate=tstate@entry=0x6290000be210, frame=0x6290000cd3a8, frame@entry=0x6290000cd328,
    throwflag=throwflag@entry=0) at Python/generated_cases.c.h:1619

Backtrace with indicated line commented out:

Thread 1 "python" received signal SIGSEGV, Segmentation fault.
0x0000555555b17d3b in mi_block_nextx (keys=0x555556751cd0 <_PyRuntime+365968>, block=0x210162caff47ebfd, null=0x555556751190 <_PyRuntime+363088>) at ./Include/internal/mimalloc/mimalloc/internal.h:637
637       next = (mi_block_t*)mi_ptr_decode(null, mi_atomic_load_relaxed(&block->next), keys);

#0  0x0000555555b17d3b in mi_block_nextx (keys=0x555556751cd0 <_PyRuntime+365968>, block=0x210162caff47ebfd,
    null=0x555556751190 <_PyRuntime+363088>) at ./Include/internal/mimalloc/mimalloc/internal.h:637
#1  _mi_heap_delayed_free_partial (heap=heap@entry=0x555556751190 <_PyRuntime+363088>) at Objects/mimalloc/page.c:331
#2  0x0000555555b208ce in _mi_malloc_generic (heap=heap@entry=0x555556751190 <_PyRuntime+363088>, size=size@entry=56, zero=zero@entry=false,
    huge_alignment=huge_alignment@entry=0) at Objects/mimalloc/page.c:943
#3  0x0000555555b21586 in _mi_page_malloc (heap=heap@entry=0x555556751190 <_PyRuntime+363088>, page=0x7fffb4006148, size=size@entry=56,
    zero=zero@entry=false) at Objects/mimalloc/alloc.c:44
#4  0x0000555555b2237f in mi_heap_malloc_small_zero (heap=0x555556751190 <_PyRuntime+363088>, size=48, zero=<optimized out>)
    at Objects/mimalloc/alloc.c:127
#5  0x0000555555b231db in _mi_heap_malloc_zero_ex (huge_alignment=0, zero=false, size=48, heap=0x555556751190 <_PyRuntime+363088>)
    at Objects/mimalloc/alloc.c:156
#6  _mi_heap_malloc_zero (zero=false, size=48, heap=0x555556751190 <_PyRuntime+363088>) at Objects/mimalloc/alloc.c:179
#7  mi_heap_malloc (size=48, heap=0x555556751190 <_PyRuntime+363088>) at Objects/mimalloc/alloc.c:183
#8  _PyMem_MiMalloc (ctx=<optimized out>, size=48) at Objects/obmalloc.c:209
#9  0x0000555555afad90 in _PyMem_DebugRawAlloc (use_calloc=use_calloc@entry=0, ctx=0x5555566f8b58 <_PyRuntime+1048>, nbytes=24)
    at Objects/obmalloc.c:2788
#10 0x0000555555afadf8 in _PyMem_DebugRawMalloc (ctx=<optimized out>, nbytes=<optimized out>) at Objects/obmalloc.c:2821
#11 0x0000555555afae15 in _PyMem_DebugMalloc (ctx=<optimized out>, nbytes=<optimized out>) at Objects/obmalloc.c:2986
#12 0x0000555555b29338 in PyMem_Malloc (size=size@entry=24) at Objects/obmalloc.c:1007
#13 0x0000555555999f59 in _PyStack_UnpackDict (tstate=tstate@entry=0x555556750de8 <_PyRuntime+362152>, args=args@entry=0x7fffffffc460,
    nargs=nargs@entry=1, kwargs=kwargs@entry={'target': <function at remote 0x7fffb5139150>}, p_kwnames=p_kwnames@entry=0x7fffffffc370)
    at Objects/call.c:988
#14 0x000055555599aaaa in _PyObject_VectorcallDictTstate (tstate=tstate@entry=0x555556750de8 <_PyRuntime+362152>,
    callable=callable@entry=<function at remote 0x7fffb4ea8970>, args=args@entry=0x7fffffffc460, nargsf=nargsf@entry=1,
    kwargs=kwargs@entry={'target': <function at remote 0x7fffb5139150>}) at Objects/call.c:140
#15 0x000055555599adc3 in _PyObject_Call_Prepend (tstate=tstate@entry=0x555556750de8 <_PyRuntime+362152>, callable=<optimized out>,
    obj=obj@entry=<Thread() at remote 0x7fffb45620a0>, args=args@entry=(), kwargs=kwargs@entry={'target': <function at remote 0x7fffb5139150>})
    at Objects/call.c:504
#16 0x0000555555ba4972 in call_method (self=<Thread() at remote 0x7fffb45620a0>, attr=<optimized out>, args=<optimized out>, kwds=<optimized out>)
    at Objects/typeobject.c:3042
#17 0x0000555555ba4b68 in slot_tp_init (self=<optimized out>, args=<optimized out>, kwds=<optimized out>) at Objects/typeobject.c:10720
#18 0x0000555555b875f8 in type_call (self=self@entry=<type at remote 0x7fffb4d8e110>, args=args@entry=(),
    kwds=kwds@entry={'target': <function at remote 0x7fffb5139150>}) at Objects/typeobject.c:2426
#19 0x000055555599504b in _PyObject_MakeTpCall (tstate=tstate@entry=0x555556750de8 <_PyRuntime+362152>,
    callable=callable@entry=<type at remote 0x7fffb4d8e110>, args=args@entry=0x7fffffffcf58, nargs=0, keywords=keywords@entry=('target',))
    at Objects/call.c:242

Found using fusil by @vstinner.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.15.0a0 experimental free-threading build (heads/main:ac7511062bf, Jun 9 2025, 15:09:10) [GCC 11.4.0]

Linked PRs

This seems very similar to #134908 we should add critical section around iteration

gh-135410: use a critical section around `StringIO.__next__` (#135412)

pythongh-135410: use a critical section around `StringIO.__next__` (p…

[3.14] gh-135410: use a critical section around `StringIO.__next__` (G…

We broke the refleak buildbots.

Revert "pythongh-135410: use a critical section around `StringIO.__ne…

Documentation expectations
Documentation must be clearly written and maintained.
For Python users: What guarantees exist, and how are they affected by the free-threaded build on all APIs in all modules of the standard library.

Approval of PEP 779 requires the recording the status of thread-safety at the docs.python.org
So then, Python users only rely on the C implementation from now on? At least from the current fix should guarantee the status of thread safe.

My simple question would be if we do not guarantee both implementations into the same implementation, then how will we record it on docs.python.org?

The docs are for io.StringIO. We should document that behavior. _pyio is, as far as I can tell, undocumented.

I am not sure why you separate the fallback implementation, io.StringIO should be documented as to whether it is thread-safe or not, if we consider the next step and this is very important.
(IIUC it includes a fallback implementation)
Similar concerns were issued with other PR #137021 (comment)

I am fine with making a pure Python implementation slower (I need to find, but I approved at least one PR with that change.)

Here is the PR that I approved for the same condition. #136218 (review)

I think most modules that have been made thread-safe under FT do not have a safe pure-Python implementation.

I am not sure why you separate the fallback implementation

It's not a "fallback" in the normal sense of the word. The only way to get _pyio is to explicitly the import the undocumented _pyio module. import io never gives you _pyio. The C extension isn't optional and io.py only uses the C extension:

cpython/Lib/io.py

Line 53 in 525784a

import _io

Similar concerns were issued with other PR #137021 (comment)

I don't think this is a legitimate concern. This is not new behavior. It's long been the case -- since _pyio was split from io in 3.1 -- that io.StringIO is thread-safe and _pyio.StringIO is not. That's not currently true in the FT build and we need to fix that. We shouldn't expand scope to _pyio.StringIO.

that io.StringIO is thread-safe and _pyio.StringIO is not

I don't get it, it was thread-safe because GIL exists? It's not?
I think that this is why https://docs.python.org/3/library/io.html#multi-threading we have not recorded it as thread-safe even in the default build. One thing we have to do from now on is write this status explicitly, even if it has not been documented.

It's not a "fallback" in the normal sense of the word. The only way to get _pyio is to explicitly the import the undocumented _pyio module. import io never gives you _pyio. The C extension isn't optional and io.py only uses the C extension:

Okay, but it still needs that we should record the thread-safe status somewhere in docs.python.org
Then, can we document that the FT build does not guarantee the thread safety of _pyio implementation? (Maybe just top of the io page)

I think most modules that have been made thread-safe under FT do not have a safe pure-Python implementation.

Yeah, that's one issue that I have tackled recently. If the fallback implementation exists, then we need to make them have the same behavior... because it's related to the documentation issue.

The easiest solution would be removing pure Python implementation from FT build 😆

The easiest solution would be removing pure Python implementation from FT build 😆

You might add a sentence in _pyio classes docstrings to mention that they are not thread-safe.

pythongh-135410: use a critical section around `StringIO.__next__` (p…

Revert "pythongh-135410: use a critical section around `StringIO.__ne…

So what's the io.StringIO and _pyio.StringIO status at the end?

Navigation Menu

Repository navigation

Segfault from calling `StringIO` methods in threads on free-threading debug build #135410

Crash report

What happened?

CPython versions tested on:

Operating systems tested on:

Output from running 'python -VV' on the command line:

Linked PRs

21 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Segfault from calling StringIO methods in threads on free-threading debug build #135410

Description

Crash report

What happened?

CPython versions tested on:

Operating systems tested on:

Output from running 'python -VV' on the command line:

Linked PRs

Activity

kumaraditya303 commented on Jun 12, 2025

ZeroIntensity commented on Jun 12, 2025

21 remaining items

corona10 commented on Aug 5, 2025

corona10 commented on Aug 5, 2025

colesbury commented on Aug 5, 2025

corona10 commented on Aug 5, 2025

corona10 commented on Aug 5, 2025

ZeroIntensity commented on Aug 5, 2025

colesbury commented on Aug 5, 2025

corona10 commented on Aug 5, 2025

corona10 commented on Aug 5, 2025

corona10 commented on Aug 5, 2025

vstinner commented on Aug 6, 2025

vstinner commented on Feb 9, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

Segfault from calling `StringIO` methods in threads on free-threading debug build #135410