Failure to correctly infer tuple type through zip #8454

Activity

1. JonathanShor commented on Feb 28, 2020

   

   JonathanShor

   on Feb 28, 2020

   Author

   More actions

   Might be relevant, but when vals is 6 or more entries long, the error is not produced. Mypy is fine with the following.

   vals = (("zz", 1), ("yyy", 2), ("xxx", 3), ("w", 4), ("v", 5), ("u", 6))
EXAMPLE: Dict[str, List[Union[str, int]]] = { k: list(val) for k, val in zip(["A", "B"], zip(*vals)) }
   ...
   >>> EXAMPLE
{'A': ['zz', 'yyy', 'xxx', 'w', 'v', 'u'], 'B': [1, 2, 3, 4, 5, 6]}
2. hauntsaninja commented on Mar 1, 2020

   

   hauntsaninja

   on Mar 1, 2020

   Last edited by hauntsaninja

   Collaborator

   More actions

   This is basically because of how zip is typed in typeshed: https://github.com/python/typeshed/blob/master/stdlib/2and3/builtins.pyi#L1446

   As you can see, it treats its input arguments as Iterable[T] (a homogenous iterable). When mypy tries to figure out what the most appropriate T for Tuple[str, int] is, it correctly infers object (the common baseclass of str and int). Hence:

   ~/delete λ cat test37.py  
   vals = (("zz", 1), ("yyy", 8))
   reveal_type(vals)
   reveal_type(zip(*vals))
   
   ~/delete λ mypy test37.py
   test37.py:2: note: Revealed type is 'Tuple[Tuple[builtins.str, builtins.int], Tuple[builtins.str, builtins.int]]'
   test37.py:3: note: Revealed type is 'typing.Iterator[Tuple[builtins.object*, builtins.object*]]'
   

   You can also see from how zip is typed at the typeshed link that if you pass in enough arguments or splat something of length that's unknown to the type system, it doesn't list out the typevars and instead gives up and returns Any.
   "Something of unknown length to the type system" explains why your explicit type annotation works (would also work if you typed it as Iterable or made the thing a list).
   "Pass in enough arguments" explains your second comment.

   The output of something like zip(*[("a", 1), ("b", 2)]) is an Iterator with the first item Tuple[str] and the second item Tuple[int] and there isn't really a good way to type that.
   We could lie in typeshed for a couple of these, maybe with something like def zip(*iter: Tuple[_T1, _T2]) -> Tuple[Tuple[_T1, ...], Tuple[_T2, ...]]
3. hauntsaninja commented on Mar 1, 2020

   

   hauntsaninja

   on Mar 1, 2020

   Last edited by hauntsaninja

   Collaborator

   More actions

   So maybe we should move this issue to typeshed and see if people have objections to that lie^
4. ilevkivskyi commented on Mar 6, 2020

   

   ilevkivskyi

   on Mar 6, 2020

   Member

   More actions

   Yes, properly typing this would require heterogeneous iterables, it is unlikely it will be supported in foreseeable future. You can however propose the quick fix (adding few extra overloads) on typeshed tracker, as suggested.
5. ilevkivskyi

   closed this as completedon Mar 6, 2020
6. hauntsaninja

   added a commit that references this issue on Mar 6, 2020

   zip: add some overloads for heterogeneous tuples

   ...

   6385c3f
7. JelleZijlstra

   added a commit that references this issue on May 4, 2020

   zip: add some overloads for heterogeneous tuples (#3830)

   ...

   e857ad6
8. vishalkuo

   added a commit that references this issue on Jun 26, 2020

   zip: add some overloads for heterogeneous tuples (python#3830)

   ...

   22a5d58