Repository navigation
Iterating over ChainMap.values() and ChainMap.items() updates underlying defaultdicts #93432
Description
Bug report
Hello,
I encountered some confusing behavior when combining ChainMap with defaultdicts. Might be related to #23534
Specifically, iterating over the values or items of the ChainMap updates one of the underlying dictionaries, and returns the wrong result.
Just iterating over they keys or the dictionary directly produces the right results still and doesn't update the underlying dictionaries.
from collections import ChainMap, defaultdict
d1 = defaultdict(int, {'a': 1, 'b': 2})
d2 = defaultdict(int, {'c': 3, 'd': 4})
d3 = ChainMap(d1, d2)
print('Prior to iter')
print(d1)
print(d2)
print('After iter')
a = list(d3.values()) # Same result when calling d3.items
print(a) # Should return: [1, 2, 3, 4]. Returns: [0, 0, 1, 2]
print(d1) # Now: {'a': 1, 'b': 2, 'c': 0, 'd': 0}
print(d2) # Unchanged
Your environment
Tested on Python version 3.10.4.
Linux 5.13.0-41-generic #46~20.04.1-Ubuntu SMP
x86_64 x86_64 x86_64 GNU/Linux
Activity
corona10 commentedon Jun 4, 2022
cc @serhiy-storchaka
corona10 commentedon Jun 4, 2022
The following patch will solve this issue, but I am waiting for @rhettinger and @serhiy-storchaka(Sorry for editing the previous comment: it will break other behavior.)
This behavior looks like it is intended because this is a matter of look-up order.
IMO, under the script,
d1did what it should do asdefaultdictwhen meeting the non-existence key.but I am waiting for both of the members whom I mentioned.
Footnotes
https://docs.python.org/3/library/collections.html?#collections.ChainMap ↩
serhiy-storchaka commentedon Jun 4, 2022
Yes, it looks like an intentional behavior. There is even a comment in the code about this:
cpython/Lib/collections/__init__.py
Line 1001 in 8150b8c
rhettinger commentedon Jun 5, 2022
To the extent that this is a problem, it lies with the defaultdict updating as a side-effect of lookups. If that isn't desired, it isn't hard to define a dict subclass with
__missing__that doesn't have the updating behavior (seecollections.Counterfor an example of how to do this).rhettinger commentedon Jul 6, 2022
FWIW, you can turn-off defaultdict's updating behavior prior to iteration: