forkserver not reused in child processes

BPO	42188
Files	test.py

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2020-10-29.00:40:06.686>
labels = ['3.8', 'type-bug', 'library']
title = 'forkserver not reused in child processes'
updated_at = <Date 2020-10-29.23:36:31.572>
user = 'https://bugs.python.org/cchomey'

bugs.python.org fields:

activity = <Date 2020-10-29.23:36:31.572>
actor = 'cchomey'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2020-10-29.00:40:06.686>
creator = 'cchomey'
dependencies = []
files = ['49549']
hgrepos = []
issue_num = 42188
keywords = []
message_count = 1.0
messages = ['379851']
nosy_count = 1.0
nosy_names = ['cchomey']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue42188'
versions = ['Python 3.8']

Linked PRs

gh-86354: Fix child processes not reusing forkserver created by parent #139537

The docstring on Forkserver.ensure_running states that a child process will use the forkserver process started by its parent:

"""
Make sure that a fork server is running.

This can be called from any process. Note that usually a child
process will just reuse the forkserver started by its parent, so
ensure_running() will do nothing.
"""

Link to definition: https://github.com/python/cpython/blob/master/Lib/multiprocessing/forkserver.py#L105

That is not the behavior that I am seeing, however. I'm seeing each child (that start a process itself) will first spawn a *new* forkserver.

pythongh-86354: Fix child processes not reusing forkserver created by…

Also seeing this issue. I'm getting around it by monkey-patching:

def _patch_forkserver_reuse():
    """Patch ForkServer.ensure_running so children reuse the parent's daemon.

    Forkserver-forked children inherit _forkserver_address and _forkserver_alive_fd
    but NOT _forkserver_pid. Without this patch, ensure_running() sees pid=None
    and starts a new daemon, wasting memory.

    This patch checks _forkserver_alive_fd: if it's set and the fd is valid,
    the parent's daemon is alive and we skip starting a new one.

    Workaround for https://github.com/python/cpython/issues/86354
    Upstream fix: https://github.com/python/cpython/pull/139537
    """
    import multiprocessing.forkserver as _fs

    _orig_ensure_running = _fs.ForkServer.ensure_running

    def _patched_ensure_running(self):
        with self._lock:
            if self._forkserver_pid is None and self._forkserver_alive_fd is not None:
                try:
                    os.fstat(self._forkserver_alive_fd)
                except OSError:
                    # fd is invalid — daemon is dead, clear and fall through
                    self._forkserver_alive_fd = None
                    self._forkserver_address = None
                else:
                    # fd is valid — daemon is alive, reuse it
                    from multiprocessing import resource_tracker

                    resource_tracker.ensure_running()
                    return
        _orig_ensure_running(self)

    _fs.ForkServer.ensure_running = _patched_ensure_running


_patch_forkserver_reuse()

Navigation Menu

Repository navigation

forkserver not reused in child processes #86354

Linked PRs

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

forkserver not reused in child processes #86354

Description

Linked PRs

Activity

cchomey commented on Oct 29, 2020

dodamih commented on Apr 12, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions