Repository navigation
Un-deprecate functional API for importlib resources & add subdirectory support #116608
Description
Activity
- added a commit that references this issue
on Mar 11, 2024 Wouldn't this be a bit late for that? We already went through the deprecation period, and removed the feature in the alpha releases, bringing them back now would be a bit confusing.
The
importlib.resourcesfunctions{open,read}_{text,binary},path,is_resourceandcontents, deprecated in 3.11 and removed in 3.13 alphas, are, anecdotally, missed by quite a few users.Can you actually show a couple examples of this affecting users downstream? I think that's the most viable argument to bring that API back.
taking multiple path components as positional arguments
Why not just take multiple components with separators in a single argument? It's easy enough to require forward slash, disallow
..and even to normalise backslashes on Windows if you feel like it.If they didn't allow subdirectories before (I never noticed, tbh), then presumably using a slash here would have either failed completely or worked. Either way, we can enable them in a new release.
(And add me to the anecdotal list of people who missed them. It's easy enough to add a few lines of code to bring them back, which is how I have been handling it so far, but I'd be happier to have those few lines in the stdlib.)
At least one audience that would like to keep the legacy APIs is in mesonbuild/meson#12401.
I admit, I prefer this approach over keeping the legacy APIs with the cruft that it still had lying around. It adds a mostly-compatible layer and restores these wrappers in a supported way.
On one hand, this approach violates the "preferably one way" to do things; users will need to decide which way works best for them, creating a variety of supported approaches. On the other hand, I do appreciate that it offers a friendlier interface for certain operations (esp.
path(...)).FFY00 and I put a lot of work into this deprecation process, so it'll be disappointing to now see this reversed at the last minute, but it does feel like the right thing to do, especially since someone else is willing to own the implementation (thanks encukuo!). We will have to backport the change to importlib_resources, but that should be fairly straightforward.
Overall, I'm +0 on the change. I'd really like to see more vocal support from other core devs before committing to this approach.
Reacted by Petr Viktorin and Daniël van NoordReacted by Wim Jeantine-GlennI've made the encoding argument mandatory for
_textfunctions when multiple path names are given.Wouldn't this be a bit late for that?
Yes, sorry. Previously I couldn't commit to supporting this API.
Why not just take multiple components with separators in a single argument?
I'd rather not derail discussion on this issue. Support for separators can be added later if necessary. If they will, allowing multiple arguments will still be useful.
FFY00 and I put a lot of work into this deprecation process
Sorry to hear that. Sunk costs suck :(
This makes it seem that implementing the deprecation process was similarly (or more) time-consuming as keeping the API working. That's not a good situation to be in, especially considering all the work users need to put in to update their code.(And add me to the anecdotal list of people who missed them. It's easy enough to add a few lines of code to bring them back, which is how I have been handling it so far, but I'd be happier to have those few lines in the stdlib.)
I’ll add a “me too” here as well. Being able to do simple things simply is an advantage.
+1 from me on the (updated) proposed API as well as un-deprecating these -- for reasons that have been discussed on the d.p.o thread as well as mentioned here by others.
Their main drawback -- not allowing subdirectories -- can be solved by taking multiple path components as positional arguments, for example:
importlib.resources.read_text('modulename', 'subdirectory', 'subsubdir', 'resource.txt')
I've always sort of wondered why this is a drawback at all, compared to simply doing this:
with importlib.resources.path('modulename.subdirectory.subsubdir', 'resources.txt') as f: ...
I'm not objecting to the new API! It's more ergonomic than pretending everything is a namespace module. But for backwards compatibility with python < 3.13 it seems practical to use the two-argument form, and the lack of a new API doesn't seem like it should have been a killer problem before now.
I don't understand the reason we can't reimplement it as:
def read_text(module, filename, *args, **kwargs): #use proper args if you want here, I just don't know them all off the top of my head with (path(module) / filename).open("r", *args, **kwargs) as f: return f.read()Why do we need the module and filename as multiple args instead of just two?
Reacted by Barney GaleI'd rather not derail discussion on this issue.
How is it derailing this issue? You're bringing back an API, which I like, and changing the design in a potentially backwards-incompatible way in the process, which I don't. Why is it derailing to ask why it has to have a different design now?
Why do we need the module and filename as multiple args instead of just two?
I'm catching myself up on this. I think the answer is (somewhere) in this thread: https://gitlab.com/python-devs/importlib_resources/-/issues/58
Reacted by Steve Dower, Petr Viktorin and Wim Jeantine-GlennWhy do we need the module and filename as multiple args instead of just two?
I'm catching myself up on this. I think the answer is (somewhere) in this thread: https://gitlab.com/python-devs/importlib_resources/-/issues/58
Which was migrated to python/importlib_resources#58.
IIUC:
- Support for resources in subdirectories required new APIs, e.g. something resembling
Traversible.iterdir(). - The pathlib API was considered a good fit and chosen for the task.
- The functional interface wasn't enhanced to support subdirectories; instead it was earmarked for eventual removal.
So there's perhaps four levels of support we could offer for the functional APIs:
- Deprecate + remove (@jaraco's original plan, already landed)
- Restore old APIs + functionality as it was.
- Restore old APIs, and add support for subdirectories to existing functions (this issue and @encukou's PR)
- Restore old APIs, add add support for subdirectories, including adding new functions where needed (e.g. functional equivalent of
traversable.iterdir())
Personally I'd lean towards option 2. If folks need subdirectory support they can use the OOP API - that's it's whole reason to exist!
Reacted by Gregory P. Smith- Support for resources in subdirectories required new APIs, e.g. something resembling
27 remaining items
This was undeprecated in 3.13 but the undeprecation should be backported to 3.12 and 3.11
- added a commit that references this issue
on Apr 7, 2025 - added a commit that references this issue
on Apr 8, 2025 - added a commit that references this issue
on May 12, 2025 I can see this was un-deprecated by https://github.com/python/cpython/pull/132206/changes in Python 3.12.10.
Can I suggest updating the documentation (at https://docs.python.org/3/library/importlib.resources.html) to add a note next to un-deprecated functions along the lines of
Changed in version 3.12.10: No longer raises DeprecationWarningso that programs targeting older Python versions know not to use those functions (especially because programs might expect compatibility with all Python 3.12 versions if they run their CI on the latest minor version of Python 3.12)
Feature or enhancement
Proposal:
The
importlib.resourcesfunctions{open,read}_{text,binary},path,is_resourceandcontents, deprecated in 3.11 and removed in 3.13 alphas, are, anecdotally, missed by quite a few users.They provide a simple API for simple tasks, while the full-featured
TraversableAPI is better suited for complex ones -- especially for implementing new resources-aware loaders.I'm now in a position where I can add these functions back and support them.
Their main drawback -- not allowing subdirectories -- can be solved by taking multiple path components as positional arguments, for example:
The additional arguments (encoding and errors) would become keyword-only.
There is a wrinkle in this: in Python 3.9-3.11, the above would mean:
I believe that this is acceptable, since:pragmatically: typical file names do not match typical encoding/errorhandler nameslawyerly: the functions have already been deprecated for 2 releases; no one is using them now, right?However, if this is a problem, I can[edit: This is solved by:]
encodingargument required if a text-reading function more than one path component is given.Has this already been discussed elsewhere?
I have already discussed this feature proposal on Discourse
Links to previous discussion of this feature:
https://discuss.python.org/t/deprecating-importlib-resources-legacy-api/11386/29
Linked PRs