multi-threading + fork warning when threads are stopped before fork

Bug report

Bug description:

If all threads are stopped before a fork, and then restarted in the parent (and optionally the child) process, the new multi-threading with fork warning is given

import os
import threading


stop = False


def target():
    while not stop:
        pass


thread = threading.Thread(target=target)


def stop_thread():
    global stop
    stop = True
    thread.join()


def restart_thread():
    global thread, stop
    stop = False
    (thread := threading.Thread(target=target)).start()


os.register_at_fork(before=stop_thread, after_in_child=restart_thread, after_in_parent=restart_thread)


thread.start()

if os.fork() == 0:
    os._exit(0)

If the thread is not restarted in the parent, no warning is given. I believe warn_about_fork_with_threads should be refactored so that a warning is given to the user only if there are threads detected after the call to PyOS_BeforeFork. If it is not possible to give a warning there, as stated in the comments, perhaps one can just do the thread count check there and take note of the result, then warn after PyOS_AfterFork_Parent, when safe to do so.

CPython versions tested on:

3.13

Operating systems tested on:

macOS

Linked PRs

@ZeroIntensity

Let's remove the callback in the child. The order of execution would be:

Start thread.
Request to fork.
Join thread.
Do the fork.
Restart the thread in the parent.

I think the problem is with the fact that thread from the parent process is not entirely joined. And so it's effectively the case that we're still multi-threaded even though it's not the case maybe.

cc @ZeroIntensity as someone who has better threading knowledge.

It looks to me that warn_about_fork_with_threads is warning on the new thread in the parent, because it is restarted by PyOS_AfterFork_Parent, before warn_about_fork_with_threads is called. So the warning is about the new thread running in the parent after the fork, rather than the original thread (that has actually been stopped and joined before the call to the fork syscall).

Yeah, it'd be better to call the after-fork-parent calls after we check and warn about threads. That would avoid the spurious warning.

Code paths that warn_about_fork_with_thread can take depend on PyOS_AfterFork_Parent() having been called already to avoid deadlock.

A workaround for this could be for it to be called by PyOS_AfterFork_Parent() itself right before the run_at_forkers(...) call.

But that'd still leave deadlocks possible as the Python world it may need to call into could rely on other afterfork parent calls that have been registered.

Okay I think I have a solution for the most common case. The warn function can be split up into the OS specific APIs to get the number of threads (which can be called before AfterFork_Parent) and the Python APIs that must wait until after. This won't fix it in situations where to lack platform specific thread detection code or when those APIs fail, but will in the most widely used scenarios. PR coming after testing.

pythongh-137109: refactor warning about threads when forking

re: race conditions, it may be OS specific as to if the thread could still show up in OS APIs as existing in the process after the internal C pthread_join() that the Python thread.join() did has returned. I'd like to think not, but that is up to the OS as there are no posix APIs to get a list of threads for a process so it is undefined behavior and up to the kernel implementation as to if it has finished tearing down that state and cleaning it up synchronously or not.

gh-137109: refactor warning about threads when forking (#141438)

[3.14] pythongh-137109: refactor warning about threads when forking (p…

[3.13] gh-137109: refactor warning about threads when forking (GH-141438

The PRs merged should've fixed this for the most common case.

pythongh-137109: refactor warning about threads when forking (python#…

fixed the last three ci found: 3.13 retainers, the fork warning, windows

asked the interpreter about the fork warning in the run being judged

stopped a race in cpython's own bookkeeping failing the fork parity

DEBUG: bracket gen_log.info()/os.fork() and add gc.collect()+sleep probe

DEBUG: measure OS-level thread count, not just PyThreadStates

test: ignore the spurious fork() multi-threading warning on macOS

test: ignore the spurious fork() multi-threading warning on x86_64 macOS

fixed the last three ci found: 3.13 retainers, the fork warning, windows

asked the interpreter about the fork warning in the run being judged

stopped a race in cpython's own bookkeeping failing the fork parity

Navigation Menu

Repository navigation

multi-threading + fork warning when threads are stopped before fork #137109

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

7 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

multi-threading + fork warning when threads are stopped before fork #137109

Description

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

Activity

picnixz commented on Jul 25, 2025

P403n1x87 commented on Jul 25, 2025

colesbury commented on Jul 28, 2025

gpshead commented on Nov 12, 2025

gpshead commented on Nov 12, 2025

gpshead commented on Nov 12, 2025

gpshead commented on Nov 12, 2025

gpshead commented on Nov 12, 2025

7 remaining items

gpshead commented on Nov 24, 2025

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions