Repository navigation
mypy should type infer variable as superclass when needed #10965
Description
Bug Report
from itertools import chain
arr = [1,2,3]
doubles = map(lambda x: 2 * x, arr)
doubles = chain(doubles, [8, 10])
print(list(doubles))➜ python3 test.py
[2, 4, 6, 8, 10]
➜ mypy test.py
test.py:4: error: Incompatible types in assignment (expression has type "chain[int]", variable has type "map[int]")
Found 1 error in 1 file (checked 1 source file)
Based on context, I think mypy should be able to infer that doubles should be a variable of type Iterator[int], since both chain[int] and map[int] are subclasses (Iterator is covariant I believe)
Can fix the above code by explicitly marking doubles: Iterator[int]
Here is a real life example
pypa/pip#10343 (comment)
Expected Behavior
Expected variable to be inferred as Iterator[int]
Activity
A5rocks commentedon Aug 10, 2021
If you want to be pedantic, you could say
object(or was ittype? Python subclassing is weird) is the superclass of everything here, which doesn't seem very useful...Where would you draw the line?
nipunn1313 commentedon Aug 10, 2021
Yep - fair point.
In a case like above,
map[int]is not a particularly useful class to distinguish from Iterator[int]. Its use comes from overloading the__init__constructor.If we were to solve this use case, I can think of two directions to solve it
Change something about the typing stub annotations of
class mapto make it clear that we want to return anIterator[T]rather thanmap[T]from construction.Change something about mypy - to help it infer in cases where relevant. "draw a line" as you mentioned. One idea that comes to mind is to draw the line directly below "object", but I can imagine that being more harmful than useful in some cases. Eg (pseudocode)
JelleZijlstra commentedon Aug 10, 2021
Does mypy's
--allow-redefinitionsflag help here?nipunn1313 commentedon Aug 10, 2021
allow-redefinition would certainly cause the error to go away, but I don't know if it's actually helping.
In solution 1) We'd want to disallow redefinitions, but somehow treat the results of
mapanditertools.chainas having the same return type ofIterator[int](is this doable in typeshed? Can we make theclass mapconstructor returnIterator[int]instead ofmap[int]?)In solution 2) We'd want to disallow redefinitions, but allow mypy to infer a superclass when the redefinition has a common non-trivial superclass (not
object).From this discussion, I'd probably lean toward something like solution 1 if possible
hauntsaninja commentedon Aug 10, 2021
re solution 1: This whole situation is actually caused by typeshed changes, see python/typeshed#5145 . I'd be in favour of going back to the way things were, but I was in the minority when things were changed.
re solution 2: I'd have to look at how mypy implements
--allow-redefinition, but something like joining the new inferred type with the previous type could work. You'd want to ensure that if there's an explicit type annotation it's always respected, but maybe more feasible than I thought when first reading the issue (my mind went first to collecting all constraints from a function and solving them all simultaneously, which would be a very hard change to make to mypy).nipunn1313 commentedon Aug 11, 2021
Wow I am amazed how python/typeshed#5219 (comment) actually found the exact issue that caused me to file this. Impressive!
It sounds like this has already been considered. Ok with closing this - typing it manually isn't such a big deal.
For
mapanditertoolsit might be nice to return an Iterator, since the class doesn't add anything on top, but there's a totally reasonable counterargument to it being a lie compared to runtime (eg messing up documentation).hauntsaninja commentedon Aug 11, 2021
That comment comes from https://github.com/hauntsaninja/mypy_primer :-) If you have any projects you'd want to surface mypy / typeshed regressions on, add them there!
AlexWaygood commentedon Mar 30, 2022
I'm closing this issue. I think "inferring variables as superclasses when needed" would be problematic, and quite difficult to generalise, for the reasons @A5rocks stated above.
I'm sympathetic to the argument that stubbing these classes "truthfully" in typeshed does more harm than good. I'd usually argue strongly for sticking as closely to the runtime as possible, but there sure do seem to be a lot of issues in this bug tracker as a result of stubbing them as classes over at typeshed.
But regardless, such discussions should be held over at the typeshed issue tracker, rather than here.