Recursion check in 3.13.8 in CALL_BOUND_METHOD_GENERAL leaves stack in inconsistent state

Bug report

Bug description:

I had the same experience with this bug as with a compiler bug: I first suspected and debugged everything else than cpython. But:

I could really not find anything wrong anywhere else
The cpython bisect unambiguously and deterministically points to this 3.13.8 backport
ebccd1de88d. All 3.13 git commits before that backport pass, all 3.13 versions after this backport fail.
cpython 3.14 with the original gh-132744: Check recursion limit in _PY_FRAME_GENERAL #132746 works fine. Maybe the 3.13.8 backport is missing some subtle dependency found only in 3.14? EDIT: confirmed below.
The behavior is the same across Windows, Linux and macOS:
test_manifest.py: parametrize loop_detection test for more coverage zephyrproject-rtos/west#919 (comment)

Reverting the 3.13.8 backport makes all failing test configurations pass again:

cd cpython
git checkout v3.13.12
git revert ebccd1de88d4 # surprisingly zero conflict

Explain this?

   sys.setrecursionlimit(1000) # PASS
   sys.setrecursionlimit(1001) # FAIL
   sys.setrecursionlimit(1002) # PASS
   sys.setrecursionlimit(1003) # FAIL
   sys.setrecursionlimit(1004) # PASS
   sys.setrecursionlimit(1005) # FAIL

While it's not clear to me why some test configurations fails and others do not, the
corruption that seems to be happening is 100% deterministic. When a test configuration fails,
then it fails 100% of the time. Test configurations that pass, pass 100% of the time.

So, I really think this is a cpython issue now.

Simple and short reproduction steps simplified from all the trial and error in zephyrproject-rtos/west#908

git clone https://github.com/zephyrproject-rtos/west
cd west
git checkout 2d883de32f4f # today's latest version

# You MUST turn off the @pytest.mark.skipif((3, 13, 8) <= sys.version_info) I had to add in test_manifest.py not to break CI.
# See where in https://github.com/zephyrproject-rtos/west/pull/919#issuecomment-3900130946

uv python install 3.12   3.13.7 3.13.8 3.13.12   3.14
for i in          3.12   3.13.7 3.13.8 3.13.12   3.14; do
  uv run --python "$i"  pytest -k loop_detect
done

Versions from 3.13.8 to 3.13.12 FAIL odd recursion limits. All other Python versions pass.

The test fails with a very long (by design) stack trace seen here:
zephyrproject-rtos/west#908 (comment)

Then, the last frames of the stack observed with pytest --pdb makes no sense:

The srp() method (Stream Reader.peek()) is called without any parameter
Its default index argument is index : int = 0
Inside the peek() method, the index: int parameter becomes equal to... self!!

        def peek(self, index: int = 0) -> Text:

(Pdb) p self
<ruamel.yaml.reader.Reader object at 0x7ff5ed1b9010>
(Pdb) p index
<ruamel.yaml.reader.Reader object at 0x7ff5ed1b9010>  # !!!!!

CPython versions tested on:

3.13

Operating systems tested on:

Linux, Windows, macOS

Linked PRs

[3.13] gh-145008: Maintain stack consistency in CALL_BOUND_METHOD_EXACT_ARGS #145015

cpython 3.14 with the original works fine

Doesn't that suggest that this isn't an issue with the recursion checker, but possibly something else? 3.14 had better stack protection mechanisms introduced, so that likely means one of two things:

There is a bug in cpython 3.13's stack detection mechanism.
There is a bug in the extension code. Just checking, does it do any sort of clever stuff at the C level?

Again, the recursion checker is just a symptom, not a cause. It's highly unlikely for the recursion check to cause stack corruption. The _PY_FRAME_GENERAL is used in Python's specializing interpreter. This means the default behavior must follow un-specialized calls in Python, which check for the recursion limit.

Unless you provide a better description or issue title, I'm planning to close the issue, as pointing it to _PY_FRAME_GENERAL is misleading.

A big red herring can be found in zephyrproject-rtos/west#908: xdist.

This issue was initially found while trying to run the whole test suite faster. It took me a very long time to realize xdist was a red herring and to reproduce without it. I'm guessing xdist is a messenger because it offsets the stack by an odd number?

Remember:

   sys.setrecursionlimit(1000) # PASS
   sys.setrecursionlimit(1001) # FAIL
   sys.setrecursionlimit(1002) # PASS
   sys.setrecursionlimit(1003) # FAIL
   sys.setrecursionlimit(1004) # PASS
   sys.setrecursionlimit(1005) # FAIL

xdist and debugging don't like each other, which made the whole experience especially painful:

support pdb.set_trace() with pytest-xdist pytest-dev/pytest#390

Off-topic now, sorry.

There is a bug in the extension code. Just checking, does it do any sort of clever stuff at the C level?

Absolutely not, which is why I filed this here.

Unless you provide a better description or issue title, I'm planning to close the issue, as pointing it to _PY_FRAME_GENERAL is misleading.

Please edit the title in any better way you see fit. I don't see why that would require closing the issue.

On the other hand, I spent a lot of time carefully crafting the description and keeping it as factual as possible and I don't see what is misleading in it?

There is a bug in cpython 3.13's stack detection mechanism.

I think this is more likely now, as I went to look at your issue and saw that you are using uv's Python. Just so you're aware:

3.13 uv Python builds with default Clang and computed gotos for Linux, which is notoriously known for consuming a huge amount of C stack
3.14 uv Python builds with default Clang and tail calling interpreter for Linux, which significantly reduces the C stack space required.

So this is very much pointing in the direction of the C stack problems causing corruption.

On the other hand, I spent a lot of time carefully crafting the description and keeping it as factual as possible and I don't see what is misleading in it?

Thanks for taking the time to craft it, but the title is

Stack corruption and 3.13.8 regression caused _PY_FRAME_GENERAL

Which states as a matter of fact that X is caused by Y, however the evidence doesn't seem to necessarily point to that, so I think it's misleading. I will change the issue title.

I think this is more likely now, as I went to look at your issue and saw that you are using uv's Python.

I mentioned uv for reproduction convenience but none of my bisects and debug used uv. I invoked pytest directly the vast majority of the time and uv was not involved. I did not spot any difference caused by uv yet.

3.13 uv Python builds with default Clang and computed gotos for Linux, which is notoriously known for consuming a huge amount of C stack

Test duration aside, the size of the stack never made any functional difference. Try it; it's a one-line change in the test!


   sys.setrecursionlimit(200) # PASS
   sys.setrecursionlimit(201) # FAIL
   sys.setrecursionlimit(202) # PASS
   sys.setrecursionlimit(203) # FAIL
   sys.setrecursionlimit(204) # PASS
   sys.setrecursionlimit(205) # FAIL

   sys.setrecursionlimit(1000) # PASS
   sys.setrecursionlimit(1001) # FAIL
   sys.setrecursionlimit(1002) # PASS
   sys.setrecursionlimit(1003) # FAIL
   sys.setrecursionlimit(1004) # PASS
   sys.setrecursionlimit(1005) # FAIL

   sys.setrecursionlimit(10000) # PASS
   sys.setrecursionlimit(10001) # FAIL
   sys.setrecursionlimit(10002) # PASS
   sys.setrecursionlimit(10003) # FAIL
   sys.setrecursionlimit(10004) # PASS
   sys.setrecursionlimit(10005) # FAIL

I already cloned it and tried it locally, with a fix.

Ok thanks, that gives me enough information. I narrowed it down to a discrepancy between how we deal with the stack in Python 3.13 and 3.14/15:

On Python 3.13, we do not necessarily leave the stack in a consistent state between uops
On Python 3.14/15, we guarantee to leave the stack in a consistent state between uops.

I have a fix up soon, however I don't know how to write a test for this. The fix is not so nice in my opinion, but as a bandaid for 3.13 it should work fine.

Stack corruption and 3.13.8 regression caused _PY_FRAME_GENERAL [truncated]

Which states as a matter of fact that X is caused by Y,

I meant the backport was the cause because that was the result of my bisect. I honestly don't understand the change itself so I wasn't trying to blame anything specific in the code. Thanks for editing the title.

I have a fix up soon, however I don't know how to write a test for this.

BTW I tried to craft a simpler, ruamel-only and "west-free" test in ruamel/_test/test_infinite_rec.py. But my attempts could unfortunately never reproduce.

Details

_test/test_infinite_rec.py

import pytest
import sys

from roundtrip import round_trip

from ruamel.yaml import YAML

_def_rec_limit = sys.getrecursionlimit()

_def_rec_limit = 1000

@pytest.mark.parametrize("py_rec_limit", range(_def_rec_limit, _def_rec_limit + 10))
def test_infinite_recursion(py_rec_limit):
    sys.setrecursionlimit(py_rec_limit)

    def infi_rec():
        yml = YAML()
        yml.load('''
        manifest:
          projects: []
          self:
            import: foo.yml
        ''')
        infi_rec()

    with pytest.raises(RecursionError):
        infi_rec()

--- a/tox.ini
+++ b/tox.ini
@@ -1,6 +1,6 @@
 [tox]
 # toxworkdir = /data1/DATA/tox/ruamel.yaml 
-envlist = cs,py311,py310,py39,py38,py37,py312
+envlist = py313
 
 [testenv]
 allowlist_externals = /bin/bash

So reproduction seems to require "west" for now :-( It's not that bad: west has by design very few dependencies, which keeps the reproduction steps still very basic IMHO (still too big for a unit test in cpython of course)

I bisected ruamel too and found that reverting this ruamel commit is another way to avoid this failure: https://sourceforge.net/p/ruamel-yaml/code/ci/f57c3e16091a6dec8cde656d22afba305f03bf77 (ruamel revert with conflicts fixed in marc-hb/ruamel-yaml@88a38c7) Sharing for completeness and in case that helps with test ideas?

EDIT: the only thing this ruamel commit does is moving one "yaml_version" field from one class to another... baffling.

PS: because of this ruamel bisect, I thought for a long time that this ruamel commit was "guilty" and the cpython "innocent", with the 3.13.8 backport just a messenger... I changed my mind and filed this for all the reasons listed in the description, and especially when I spotted that the index default parameter becomes equal to self...

Aligning all those planets has been quite the journey!

@marc-hb

@marc-hb sorry for the curt response earlier. Thanks again for the bug report.

Don't worry - I can relate and we live "interesting" times:

https://daniel.haxx.se/blog/2025/07/14/death-by-a-thousand-slops/
[PERF] Replace np.column_stack with np.vstack().T matplotlib/matplotlib#31132 (comment) /https://lwn.net/Articles/1058643/ "Do androids dream of accepted pull requests?"
https://lwn.net/Articles/1008897/ "Fighting the AI scraperbot scourge"

After all, how do you know I'm not some OpenClaw AI? ;-)

And to be really honest I had even "lower" expectations: I was afraid this issue would understandably be drown in the 5000+ cpython issue currently opened. I can't believe there's a tentative fix floating less than 1h after I filed this - incredible!

@marc-hb

@marc-hb can you please verify the fix works for your test case?

test_manifest.py: re-enable RecursionError test for python 3.13.13

[3.13] gh-145008: Maintain stack consistency in CALL_BOUND_METHOD_EXA…

Navigation Menu

Repository navigation

Recursion check in 3.13.8 in CALL_BOUND_METHOD_GENERAL leaves stack in inconsistent state #145008

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

3 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Recursion check in 3.13.8 in CALL_BOUND_METHOD_GENERAL leaves stack in inconsistent state #145008

Description

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

Activity

Fidget-Spinner commented on Feb 19, 2026

marc-hb commented on Feb 19, 2026

marc-hb commented on Feb 19, 2026

Fidget-Spinner commented on Feb 19, 2026

marc-hb commented on Feb 19, 2026

marc-hb commented on Feb 19, 2026

Fidget-Spinner commented on Feb 19, 2026

marc-hb commented on Feb 19, 2026

3 remaining items

marc-hb commented on Feb 19, 2026

marc-hb commented on Feb 19, 2026

Fidget-Spinner commented on Feb 19, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions