Repository navigation
Reduce the number of materialized instance __dict__s #106485
Description
Accessing the __dict__ attribute of an object causes the dictionary to be materialized.
This has a negative impact on performance.
We should provide means to access the __dict__ without forcing materialization, which can be used in copy, pickle and any third party code that chooses to.
I propose two new methods on object:
__dict_copy__(self), which will be equivalent toself.__dict__.copy(), but without materializing the dictionary of objects with virtual dictionaries.__dict_update__(self, arg), which will be equivalent toself.__dict__.update(arg), but without materializing the dictionary of objects with virtual dictionaries.
Linked PRs
- GH-106485: "Un-materialize"
__dict__s inLOAD_ATTR_WITH_HINT#106496 - GH-106485: Dematerialize instance dictionaries when possible #106539
- GH-106485: Handle dict subclasses correctly when dematerializing
__dict__#107837 - GH-106485: Create object's dict-values instead of creating __dict__, when we can. #107843
Activity
brandtbucher commentedon Jul 6, 2023
I opened a draft PR for an alternate approach, which aggressively "un-materializes"
__dict__s inLOAD_ATTR_WITH_HINT.[-]Avoid using `__dict__` in pickle and copy.[/-][+]Reduce the number of materialized instance `__dict__`s[/+]GH-106485: Dematerialize instance dictionaries when possible (GH-106539)
kumaraditya303 commentedon Aug 10, 2023
This change seems to be causing crashes on buildbots. See https://buildbot.python.org/all/#/builders/123/builds/813 as example.
markshannon commentedon Aug 10, 2023
Two more things to fix:
copy.deepcopydo not get their__dict__s dematerialized._PyObjectDict_SetItemfor objects with managed__dict__s if the key is in the shared keys.GH-106485: Handle dict subclasses correctly when dematerializing `__d…
GH-106485: Create object's dict-values instead of creating __dict__, …
hugovk commentedon Nov 10, 2023
Is there more to do here, or can we close this issue?