linecache.cache sometimes has an entry for <string> under Python 3.13.0a5

Bug report

Bug description:

I noticed this via Hypothesis' pretty-printer for lambda functions, and tracked the divergence through the inspect module to linecache:

import linecache

def test():
    print(linecache.cache["<string>"])  # expected to raise KeyError
    assert False

If I run this snippet with python3.13 -Wignore -m pytest repro.py, or any older Python version, you'll get the expected KeyError. But if I append -n2 to the command, it prints (44, None, ['import sys;exec(eval(sys.stdin.readline()))\n'], '<string>')!

(python3.13 -m pip install pytest pytest-xdist will get the dependencies for this)

That's the popen bootstrap line from execnet, which handles running code across multiple processes for pytest-xdist. At this point I've found linecache.cache.pop("<string>", None) to be an acceptable workaround, and since I don't have any particular knowledge of either the CPython or execnet code that's as far as I investigated.

CPython versions tested on:

3.8, 3.9, 3.10, 3.11, 3.12, 3.13

Operating systems tested on:

Linux

Linked PRs

@pablogsal

I believe this is a feature from #110805 which allows the traceback to show source code with python -c.

python -c "import linecache;print(linecache.cache);1/0"

3.12:

{}
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ZeroDivisionError: division by zero

3.13:

{'<string>': (44, None, ['import linecache;print(linecache.cache);1/0\n'], '<string>')}
Traceback (most recent call last):
  File "<string>", line 1, in <module>
    import linecache;print(linecache.cache);1/0
                                            ~^~
ZeroDivisionError: division by zero

However, I can see that the secret <string> key in the cache causing issues for inspect because that module heavily relies on it. An example:

./python -c "import inspect;x=eval('lambda x: x');print(inspect.getsource(x))"

The code above should trigger an exception because we should not be able to get the source of x (it's dynamically generated) but now it gives the full string.

I think a proper fix would be to change the code object name <string> for code passed with -c. <string> is commonly used for many dynamically generated code objects and if we want to do something special with the source code from -c we should name it differently. It might not be 100% backward compatible but I don't see a giant gap there. We can even keep the <string> in the traceback like what we did for python inputs.

A simpler way would be just disable this feature for -c - it's designed for interactive inputs and I don't even know if this is intentional or a pleasant side effect.

Or we could do even more aggressive and do similar numbering for dynamically compiled code and always give source code, which is a bit ambitious and definitely beyond this issue.

@pablogsal thoughts on this? I can do the PR if you want.

pythongh-117174: Rename <string> to <__interactive_string__> for inte…

I have opened #117500 to do a renaming of the string to something less common. We still want to preserve this behaviour for -c since this still leads to better error locations and tracebacks.

pythongh-117174: Rename <string> to <__interactive_string__> for inte…

pythongh-117174: Respect tests that require environment variables wit…

gh-117174: Add a new route in linecache to fetch interactive source c…

pythongh-117174: Add a new route in linecache to fetch interactive so…

pythongh-117174: Adapt test_multiple_statements_fail_early now that w…

gh-117174: Adapt test_multiple_statements_fail_early now that we have…

[3.13] gh-117174: Add a new route in linecache to fetch interactive s…

After this was merged I'm seeing failures like this:

======================================================================
ERROR: test_frames (test.test_gdb.test_pretty_print.PrettyPrintTests.test_frames)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.lto/build/Lib/test/test_gdb/test_pretty_print.py", line 426, in test_frames
            gdb_output = self.get_stack_trace('''
                         ~~~~~~~~~~~~~~~~~~~~^^^^
    import sys
    ^^^^^^^^^^
    ...<7 lines>...
                                              cmds_after_breakpoint=['print (PyFrameObject*)x']
                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                                              )
                                              ^
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.lto/build/Lib/test/test_gdb/util.py", line 242, in get_stack_trace
    out, err = run_gdb(*args, PYTHONHASHSEED=PYTHONHASHSEED)
               ~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.lto/build/Lib/test/test_gdb/util.py", line 67, in run_gdb
    raise Exception(f"{cmd_text} failed with exit code {proc.returncode}, "
    ...<2 lines>...
                    f"stderr={stderr!r}")
Exception: /usr/bin/gdb --batch -nx --init-eval-command 'add-auto-load-safe-path /home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.lto/build/python-gdb.py' '--eval-command=set breakpoint pending yes' '--eval-command=break _typing__idfunc' '--eval-command=set print address off' --eval-command=run '--eval-command=set print entry-values no' '--eval-command=print (PyFrameObject*)x' --args /home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.lto/build/python -bb -E -Wdefault -S -c '
import sys
from _typing import _idfunc
def foo(a, b, c):
    return sys._getframe(0)

f = foo(3, 4, 5)
_idfunc(f)' failed with exit code 1, expected exit code 0:
stdout='Breakpoint 1 at 0x43c9e0: file ./Include/object.h, line 270.\n[Thread debugging using libthread_db enabled]\nUsing host libthread_db library "/lib64/libthread_db.so.1".\n\nBreakpoint 1, _typing__idfunc () at ./Include/object.h:270\n270\t        return ob->ob_type;\n'
stderr='No symbol "x" in current context.\n'

How this was not detected in the CI?

I will take a look later today

[3.13] gh-117174: Fix reference leak and gdb tests (GH-131095) (#131120)

@vstinner

@vstinner, do you have any ideas?

pythongh-117174: Skip test_gdb if Python is built with LTO

@pablogsal

tl; dr I created PR gh-131143 to skip test_gdb if Python is built with LTO.

There are two important things on the AMD64 RHEL 8 buildbot:

It uses an old GCC version: gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-24)
LTO is always used on RHEL buildbots

I already noticed weird gdb behaviors when Python is built with LTO.

@pablogsal:

Looking at the failures this is super wrong: (...)
_typing__idfunc is certainly not at ./Include/object.h:270

It's maybe time to declare game over and skip test_gdb if Python is built with LTO :-( We already skip test_gdb if Python is built with PGO or BOLT:

if support.check_cflags_pgo():
    raise unittest.SkipTest("test_gdb is not reliable on PGO builds")

if support.check_bolt_optimized():
    raise unittest.SkipTest("test_gdb is not reliable on BOLT optimized builds")

By the way, we recently modified test_gdb to also skip it if Python is built with clang -Og (commit 129db32).

@pablogsal

tl; dr I created PR gh-131143 to skip test_gdb if Python is built with LTO.

There are two important things on the AMD64 RHEL 8 buildbot:

It uses an old GCC version: gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-24)

LTO is always used on RHEL buildbots

I already noticed weird gdb behaviors when Python is built with LTO.

@pablogsal:

Looking at the failures this is super wrong: (...)
_typing__idfunc is certainly not at ./Include/object.h:270

It's maybe time to declare game over and skip test_gdb if Python is built with LTO :-( We already skip test_gdb if Python is built with PGO or BOLT:
if support.check_cflags_pgo():
    raise unittest.SkipTest("test_gdb is not reliable on PGO builds")

if support.check_bolt_optimized():
    raise unittest.SkipTest("test_gdb is not reliable on BOLT optimized builds")
By the way, we recently modified test_gdb to also skip it if Python is built with clang -Og (commit 129db32).

Please let's not do it. The whole point of test gdb is to test that gdb works with released versions of CPython. If we skip it then gdb will be broken and we never know that the functionality works with the versions that people use it

I fixed the test failures by using a different function that doesn't get super optimised to wrong DWARF so at least we are fine for now

Oh, I didn't see that you already fixed the issue. That's why I was unable to reproduce it :-)

Ok, I abandon my PR skipping test_gdb is Python is built with LTO.

Adapt test to new REPL behavior (follow-up pythongh-117174)

gh-117174: Adapt `test_multiple_statements_fail_early` to new REPL be…

pythongh-117174: Adapt `test_multiple_statements_fail_early` to new R…

pythongh-117174: Adapt test_multiple_statements_fail_early now that w…

[3.13] gh-117174: Adapt test_multiple_statements_fail_early now that …

[3.13] gh-117174: Adapt `test_multiple_statements_fail_early` to new …

Copy `code.co_qualname` on Python >= 3.11

Copy `code.co_qualname` on Python >= 3.11 (#423)

Navigation Menu

Repository navigation

`linecache.cache` sometimes has an entry for `<string>` under Python 3.13.0a5 #117174

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

8 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

linecache.cache sometimes has an entry for <string> under Python 3.13.0a5 #117174

Description

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

Activity

gaogaotiantian commented on Mar 26, 2024

pablogsal commented on Apr 3, 2024

encukou commented on Mar 11, 2025

pablogsal commented on Mar 11, 2025

8 remaining items

encukou commented on Mar 12, 2025

vstinner commented on Mar 12, 2025

pablogsal commented on Mar 12, 2025

pablogsal commented on Mar 12, 2025

vstinner commented on Mar 12, 2025

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

`linecache.cache` sometimes has an entry for `<string>` under Python 3.13.0a5 #117174