Can't import concurrent.futures.thread at shutdown

Bug report

Bug description:

Simple reproducer (real-world bug report is a bit more complex):

import threading


def use_cf():
    import concurrent.futures.thread


def main():
    t = threading.Thread(target=use_cf)
    t.start()


if __name__ == "__main__":
    main()

CPython versions tested on:

3.10, 3.11, 3.12, 3.13

Operating systems tested on:

No response

@gpshead

I don't know if we want to do anything about this, I'm just posting just in case.

@gpshead

In general we consider this working as intended (from the linked bug):

$ python main.py 
Traceback (most recent call last):
  File "pyarrow/pandas-shim.pxi", line 100, in pyarrow.lib._PandasAPIShim._check_import
  File "pyarrow/pandas-shim.pxi", line 48, in pyarrow.lib._PandasAPIShim._import_pandas
  File "/home/jleibs/pyarrow-repro/venv/lib/python3.10/site-packages/pyarrow/pandas_compat.py", line 24, in <module>
    import concurrent.futures.thread  # noqa
  File "/usr/lib/python3.10/concurrent/futures/thread.py", line 37, in <module>
    threading._register_atexit(_python_exit)
  File "/usr/lib/python3.10/threading.py", line 1504, in _register_atexit
    raise RuntimeError("can't register atexit after shutdown")
RuntimeError: can't register atexit after shutdown

After the main interpreter thread has exited, launching new threads isn't really supported.

This has been true since 3.9 thanks to b61b818 which resolved Issue #83993 via PR #19149.

The problem isn't launching new threads here, it's simply importing the module. In other words, c.f.thread is imported as a side effect of the user importing another module after shutdown has started, even though the executor functionality wouldn't have been used anyway. It is of course possible to workaround by catching the RuntimeError, but it's a bit of a pointless annoyance.

In general, I think it's a good idea for module import to be graceful, even though it can be complicated to achieve in some cases (and concurrent.futures is of course a complex module).

It isn't reasonable to expect all non-trivial imports to work in a finalization context.

…

On Thu, Dec 14, 2023, 1:19 AM Antoine Pitrou ***@***.***> wrote: In general, I think it's a good idea for module import to be graceful, even though it can be complicated to achieve in some cases (and concurrent.futures is of course a complex module). — Reply to this email directly, view it on GitHub <#113060 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAQXCZ7H5M23XR63EPF57LYJLAARAVCNFSM6AAAAABATUVWJ6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNJVGQ3TAMZVGE> . You are receiving this because you modified the open/close state.Message ID: ***@***.***>

Navigation Menu

Repository navigation

Can't import `concurrent.futures.thread` at shutdown #113060

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Can't import concurrent.futures.thread at shutdown #113060

Description

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Activity

pitrou commented on Dec 13, 2023

gpshead commented on Dec 14, 2023

pitrou commented on Dec 14, 2023

pitrou commented on Dec 14, 2023

gpshead commented on Dec 14, 2023

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

Can't import `concurrent.futures.thread` at shutdown #113060