Skip to content

gh-153838: Skip non-regular source files in the heatmap exporter - #153839

Merged
pablogsal merged 4 commits into
python:mainfrom
tonghuaroot:heatmap-confine-source-reads
Oct 5, 2026
Merged

pablogsal merged 4 commits into
python:mainfrom
tonghuaroot:heatmap-confine-source-reads

Conversation

@tonghuaroot

@tonghuaroot tonghuaroot commented Jul 17, 2026 •

Copy link
Copy Markdown
Contributor

The heatmap exporter can hang or read indefinitely when a sampled filename points to a FIFO or device. Check that the source is a regular file before reading it and use the existing unavailable-source placeholder otherwise.

Tests require generated HTML and verify that missing files, directories, and FIFOs are never opened. Regular-file exports remain covered. There is no size cap, following the review discussion in #153838. The profiling test suite passes.

Closes #153838.

@bedevere-app bedevere-app Bot added the type-feature A feature request or enhancement label Oct 5, 2026
@pablogsal pablogsal changed the title gh-153838: Skip non-regular and oversized files in heatmap exporter gh-153838: Skip non-regular source files in the heatmap exporter Oct 5, 2026
@pablogsal pablogsal added the needs backport to 3.15 pre-release feature fixes, bugs and security fixes label Oct 5, 2026
@pablogsal
pablogsal enabled auto-merge (squash) October 5, 2026 14:07
@bedevere-app

bedevere-app Bot commented Oct 5, 2026

Copy link
Copy Markdown

GH-158853 is a backport of this pull request to the 3.15 branch.

@pablogsal
pablogsal merged commit 7226870 into python:main Oct 5, 2026
102 of 107 checks passed
@pablogsal pablogsal added type-bug An unexpected behavior, bug, or error and removed type-feature A feature request or enhancement labels Oct 5, 2026
pablogsal added a commit that referenced this pull request Oct 5, 2026
…er (GH-153839) (#158853)

Backport of GH-153839.

Co-authored-by: tonghuaroot <tonghuaroot@gmail.com>
pablogsal added a commit to pablogsal/cpython that referenced this pull request Oct 5, 2026
pablogsal added a commit that referenced this pull request Oct 11, 2026
…58882)

* [3.15] gh-153364: Make frame, coroutine, and task-waiter chain walks iterative and bounded (GH-153365) (#158813)

gh-153364: Make frame, coroutine, and task-waiter chain walks iterative and bounded (GH-153365)

* let me declare single limit

* use our new limit in process_frame_chain()

* add it in parse_async_frame_chain()

* parse_coro_chain()

* NEWS

* async in the message?

* test

* no race

* process_task_awaited_by

* process_task_awaited_by limit test

* NEWS

* MAX_TASK_WAITER_CHAIN_DEPTH

* TASK_WAITER_CHAIN_DEPTH in test

* TASK_WAITER_CHAIN_DEPTH 256

* prevent the drift with the comment

* better naming, better style

* MAX_TASK_WAITER_CHAIN_DEPTH comment

* task-waiter iterative bfs walk

* iterative coro-walk

* nicer news

* 1 << 14

* comment

* unused read_Py_ssize_t

* fix tombstones

* simplify

* correct msg

* better test

* news for tombstones

* left-over from when testing buggy version

* redundant new line
(cherry picked from commit e0861c6)

Co-authored-by: Maurycy Pawłowski-Wieroński <maurycy@maurycy.com>
(cherry picked from commit 9e401cf)

* [3.15] gh-155811: Add a seqcount to `gc_stats` to prevent torn reads (GH-155828) (#158829)

* update_seq

* no need for XCHGL, MOVL is enough?

* gh-155811: Retry an inconsistent GC snapshot once

---------
(cherry picked from commit 5fecd44)

Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
(cherry picked from commit 0281240)

* [3.15] gh-151292: `_remote_debugging`: Do not corrupt the binary file when hitting `OverflowError` (GH-152892) (#158830)

gh-151292: `_remote_debugging`: Do not corrupt the binary file when hitting `OverflowError` (GH-152892)

* the kolektor

* test

* better test

* news

* =Q, move const to the base, not self.running

* gh-151292: Track binary writer finalization state

---------
(cherry picked from commit f839c06)

Co-authored-by: Maurycy Pawłowski-Wieroński <maurycy@maurycy.com>
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
(cherry picked from commit 8a7c23f)

* [3.15] gh-158583: Fix uninitialized memory read in bytes.fromhex() (GH-158584) (#158691)

gh-158583: Fix uninitialized memory read in bytes.fromhex() (GH-158584)
(cherry picked from commit 9d22a53)

Co-authored-by: Victor Stinner <vstinner@python.org>
(cherry picked from commit 4f7af46)

* [3.15] gh-154194: Degrade frames in Tachyon instead of failing the sample (GH-154195) (#158831)

* gh-154194: Degrade frames in Tachyon instead of failing the sample (#154195)

* degrade gracefully

* news

* better NEWS wording

* do not raise on MAX_REMOTE_STR_READ

* bye MAX_REMOTE_STR_READ

* fix -m asyncio ps|pstree

* test truncation and linetable sentinel

* simpler

* simpler

* redundant now

* respect #157790 in the news

---------

Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
(cherry picked from commit 7d25916)

* Preserve the stable ABI when creating fallback frame names

---------

Co-authored-by: Maurycy Pawłowski-Wieroński <maurycy@maurycy.com>
(cherry picked from commit c27f494)

* [3.15] Add MSan to CI (GH-158625) (#158832)

Run a job that the test suite with MSan to the CI (#158625)

* Run the test suite with MSan in CI

* Additional fixes

* Add `_Py_MSAN_UNPOISON_STRING`

* Apply Victor's suggestions

* Apply Victor's suggestions

---------

(cherry picked from commit b93fb19)

Co-authored-by: Stan Ulbrych <stan@python.org>
Co-authored-by: Victor Stinner <victor.stinner@gmail.com>
(cherry picked from commit 1bc78de)

* [3.15] gh-156810: Write the profiler's collapsed-stack export as UTF-8 (GH-156811) (#156814)

gh-156810: Write the profiler's collapsed-stack export as UTF-8 (GH-156811)
(cherry picked from commit c3706f4)

Co-authored-by: tonghuaroot (童话) <tonghuaroot@gmail.com>
(cherry picked from commit fce28da)

* [3.15] gh-158552: Wait for Windows threads to suspend before blocking sampling (GH-158802) (#158845)

gh-158552: Wait for Windows threads to suspend before blocking sampling (GH-158802)

* gh-158552: Wait for Windows threads to suspend before blocking sampling

* Use a named Windows thread enumeration status constant
(cherry picked from commit 1643525)

Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
(cherry picked from commit 31288da)

* [3.15] gh-152721: Fix quadratic RLE replay time in the profiling binary reader (GH-152722) (#158850)

Backport of GH-152722.

Co-authored-by: tonghuaroot <tonghuaroot@gmail.com>
(cherry picked from commit ebaca2a)

* [3.15] gh-156545: Fix flamegraph export RecursionError on deeply recursive programs (GH-156546) (#158851)

Backport of GH-156546.

Co-authored-by: tonghuaroot (童话) <tonghuaroot@gmail.com>
(cherry picked from commit 0ac7217)

* [3.15] gh-158540: Add the profiled script's directory to sys.path (GH-158548) (#158844)

gh-158540: Add the profiled script's directory to sys.path (GH-158548)

* gh-158540: Add the profiled script's directory to sys.path

When a script is profiled with ``python -m profiling.sampling run
script.py`` from another directory, the script cannot import modules
placed next to it, because ``_sync_coordinator._execute_script()``
executes it with the working directory (added by
``_setup_environment()`` for the module case) as ``sys.path[0]``
instead of the script's own directory.

Make the script's directory importable in ``_execute_script()``,
matching the behavior of ``python script.py``.

Add a regression test that runs the coordinator on a script importing
a sibling module.

* Update _sync_coordinator.py Comment simplified

* gh-158540: Resolve symlinks when adding the script directory to sys.path

``python script.py`` resolves symlinks when computing ``sys.path[0]``, so
a script reached through a symlink (``link.py -> sub/where.py``) imports
modules from the real script's directory.  Apply ``os.path.realpath()``
before taking the directory name, and make sure the result is placed at
the front of ``sys.path`` even if it was already listed.

Add a regression test for a symlinked script.

---------
(cherry picked from commit 3f02aab)

Co-authored-by: he_tao <53343436+hetaozdh@users.noreply.github.com>
Co-authored-by: Eduardo Villalpando Mello <eduardo.villalpando.mello@gmail.com>
(cherry picked from commit f52d831)

* [3.15] gh-153838: Skip non-regular source files in the heatmap exporter (GH-153839) (#158853)

Backport of GH-153839.

Co-authored-by: tonghuaroot <tonghuaroot@gmail.com>
(cherry picked from commit 48998df)

* [3.15] gh-158539: Fix exception mode missing handlers in generators/coroutines (GH-158581) (#158852)

* [3.15] gh-158539: Fix exception mode missing handlers in generators/coroutines (GH-158581)

Backport of GH-158581.

Co-authored-by: LucasZhou <donghao.zhou@outlook.com>

* [3.15] gh-158539: Use portable static assertion messages

* [3.15] gh-158539: Keep layout assertions with debug-offset validation

* [3.15] gh-158539: Use the platform guard for in-process inspection tests

---------

Co-authored-by: LucasZhou <donghao.zhou@outlook.com>
(cherry picked from commit d625ecb)

* [3.15] gh-158522: Fix truncated stack for a task whose coroutine recurses (GH-158526) (#158870)

Co-authored-by: Timofei Ivankov <128279579+deadlovelll@users.noreply.github.com>
(cherry picked from commit a904b39)

* gh-156545, gh-158539: Fix deep flamegraph export on small C stacks and macOS runtime lookup (#158874)

(cherry picked from commit 114de19)

Include the C-stack test helper from main, introduced by
ce5ae29, which the regression test
requires but 3.15 does not yet provide.

---------

Co-authored-by: Miss Islington (bot) <31488909+miss-islington@users.noreply.github.com>
Co-authored-by: Maurycy Pawłowski-Wieroński <maurycy@maurycy.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Stan Ulbrych <stan@python.org>
Co-authored-by: Victor Stinner <victor.stinner@gmail.com>
Co-authored-by: tonghuaroot (童话) <tonghuaroot@gmail.com>
Co-authored-by: he_tao <53343436+hetaozdh@users.noreply.github.com>
Co-authored-by: Eduardo Villalpando Mello <eduardo.villalpando.mello@gmail.com>
Co-authored-by: LucasZhou <donghao.zhou@outlook.com>
Co-authored-by: Timofei Ivankov <128279579+deadlovelll@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type-bug An unexpected behavior, bug, or error

Projects

None yet

Development

Successfully merging this pull request may close these issues.

profiling.sampling heatmap reads non-regular and oversized files

2 participants