Repository navigation
Improve import time of various stdlib modules #118761
Description
Activity
Sure!
Reacted by Ofek LevI've opened a PR over at the
importlib_metadatarepo that avoids importinginspect. python/importlib_metadata#499importlib.metadata imports zipfile at the top for a function that won't be called in the vast majority of cases.
Some other less frequently-used imports which are only accessed once or twice, such as json, could also be tucked away in their calling functions.@layday were you planning on tackling these?
It also imports importlib.abc, which in turn imports importlib.resources, to subclass an ABC with a single, non-abstract method
This seems to be solved on main,
importlib.abcno longer importsimportlib.resources.- changed the title
[-]Further improve import time of `importlib.metadata`[/-][+]Improve import time of various stdlib modules[/+]on Aug 6, 2024 - added a commit that references this issue
on Aug 7, 2024 I've opened a PR over at the importlib_metadata repo that avoids importing inspect. python/importlib_metadata#499
This has been merged and released in version 8.4 of
importlib_metadata🎉importlib.metadata imports zipfile at the top for a function that won't be called in the vast majority of cases. It also imports importlib.abc, which in turn imports importlib.resources, to subclass an ABC with a single, non-abstract method - I assume redefining the method in importlib.metadata would be harmless. Some other less frequently-used imports which are only accessed once or twice, such as json, could also be tucked away in their calling functions.
I've submitted python/importlib_metadata#502 that defers zip import, and python/importlib_metadata#503 which defers
jsonandplatform.Reacted by Hugo van Kemenade(removing the 3.14 label since features always target the main branch)
- added a commit that references this issue
on Nov 21, 2024 110 remaining items
- added a commit that references this issue
on Apr 7, 2025 - added a commit that references this issue
on Apr 8, 2025 #132614 adds a test helper
test.support.import_helper.ensure_lazy_importsthat can be used to assert that when one module is imported, it doesn't import specific other modules as a side effect. This makes sure we don't accidentally regress if we've added a lazy import somewhere.If anyone reading this is interested, it'd be helpful to add test cases using this new helper to other modules that we've previously optimized by deferring imports.
Reacted by Ofek Lev, Cody Maloney, Hugo van Kemenade, Michael H, Jakub Kuczys and Daniel Hollas- added a commit that references this issue
on Apr 24, 2025 If anyone reading this is interested, it'd be helpful to add test cases using this new helper to other modules that we've previously optimized by deferring imports.
I've submitted PR for this in #133057
Python 3.14 is now in release candidate (please test!), so no more of these are allowed in 3.14. Thanks everyone for work here, some 22 modules have been improved!
Would someone like to open a new issue for 3.15? The pending #131796 and #132046 PRs and new improvements can be covered by that.
And we can also backport
ensure_lazy_importstests for the 3.14 improvements, to help guard against regressions.Reacted by Daniel Hollas and layday
Feature or enhancement
Proposal:
Following on from #109653, further improvements can be made to import times.
Links to previous discussion of this feature:
https://discuss.python.org/t/deferred-computation-evalution-for-toplevels-imports-and-dataclasses/34173
For example:
importlib.metadatais often used for tasks that need to happen at import, e.g. to enumerate/load entry point plug-ins, so it might be worth seeing if we can cut down its own import time a bit more.importlib.metadataimportszipfileat the top for a function that won't be called in the vast majority of cases. It also importsimportlib.abc, which in turn importsimportlib.resources, to subclass an ABC with a single, non-abstract method - I assume redefining the method inimportlib.metadatawould be harmless. Some other less frequently-used imports which are only accessed once or twice, such asjson, could also be tucked away in their calling functions.Linked PRs
pprint#122725socketby writingsocket.errorTabas a constant and lazy import modules #121424mimetypes#126979pickle#128732reimport inbase64.b16decodefor a more efficient alternative #128736secrets#128738csv#128858tomllib#128907pstatsandzipfileby removing imports totyping#128981sqlite3#129118subprocess#129427warningsinthreading#129428subprocess(GH-129427) #129447subprocess(GH-129427) #129448warnings#129765traceback#129811reinlocale#129860dataclasses#129925cmdmodule #130056subprocess(GH-129427)" #130201subprocess(GH-129427)" (GH-130201) #130204subprocess(GH-129427)" (GH-130201) #130205sqlite3#131796ast#132024annotationlib#132028enum._simple_enumforannotationlib.Format#132031shlex#132036string#132037email.quoprimimeremovingreimport #132046_types#132103