Recursive types not working correctly #13786

Activity

1. Waszker

   added

   bugmypy got something wrongmypy got something wrong

   on Oct 1, 2022
2. erictraut commented on Oct 1, 2022

   

   erictraut

   on Oct 1, 2022

   More actions

   I think mypy is doing the correct thing here. In your first sample, some_data has no type annotation, so its type is inferred to be dict[str, str]. The type parameters for dict are invariant, so they must match exactly. Since some_data is dict[str, str], it is not compatible with dict[str, JSON]. If you add the type annotation for some_data as you do in your second example, then it will work fine. Also, if you inline the expression for some_data rather than creating a separate variable, mypy can use the type context to infer that its type should be dict[str, JSON] in that context.
3. hauntsaninja

   closed this as completedon Oct 1, 2022
4. Waszker commented on Oct 1, 2022

   

   Waszker

   on Oct 1, 2022

   Author

   More actions

   I think mypy is doing the correct thing here. In your first sample, some_data has no type annotation, so its type is inferred to be dict[str, str]. The type parameters for dict are invariant, so they must match exactly. Since some_data is dict[str, str], it is not compatible with dict[str, JSON]. If you add the type annotation for some_data as you do in your second example, then it will work fine. Also, if you inline the expression for some_data rather than creating a separate variable, mypy can use the type context to infer that its type should be dict[str, JSON] in that context.

   Thank you for the answer. How should I proceed with the JSON type definition then? There are some functions in my code that have annotated return value as dict[str, str] and I'd like them to be accepted as part of JSON structure. Should I just cast them explicitly?
5. erictraut commented on Oct 2, 2022

   

   erictraut

   on Oct 2, 2022

   More actions

   Casting is one option. Another option is possible if you don't need the resulting type to be mutable. You could define another type for a read-only JSON value.

   JSON_RO = Union[
       Mapping[str, "JSON_RO"], Sequence["JSON_RO"], str, int, float, bool, None
   ]
   
   some_data = {"eggs": "spam"}
   json_data: JSON_RO = {"foo": "bar", "python": some_data}

   This works because the type parameters for Mapping and Sequence are covariant.

   🎉React with 🎉2Reacted by Piotr Waszkiewicz and Cal Jacobson
6. smheidrich commented on Nov 18, 2022

   

   smheidrich

   on Nov 18, 2022

   Last edited by smheidrich

   More actions

   @erictraut The problem with that is that you often genuinely don't want generic mappings and sequences to be considered valid JSON-like types, because certain JSON libraries won't accept them.

   Python's own builtin json module, for example, insists on lists and dicts (or subclasses thereof) and will fail if you e.g. give it another kind of mapping:

   import json
   from typing import Mapping, Sequence, Union
   
   JSON_RO = Union[
       Mapping[str, "JSON_RO"], Sequence["JSON_RO"], str, int, float, bool, None
   ]
   
   class MyMapping(Mapping[str, int]):
       def __init__(self):
           self.data = {"a": 1, "b": 2}
   
       def __iter__(self):
           return iter(self.data)
   
       def __getitem__(self, key):
           return self.data[key]
   
       def __len__(self):
           return len(self.data)
   
   def dumps(j: JSON_RO):
       json.dumps(j)
   
   dumps(MyMapping())

   Mypy doesn't complain about this, of course, but it fails at runtime with

   Traceback (most recent call last):
     File "/home/smheidrich/json-type.py", line 29, in <module>
       dumps(MyMapping())
     File "/home/smheidrich/json-type.py", line 26, in dumps
       json.dumps(j)
     File "/usr/lib/python3.10/json/__init__.py", line 231, in dumps
       return _default_encoder.encode(obj)
     File "/usr/lib/python3.10/json/encoder.py", line 199, in encode
       chunks = self.iterencode(o, _one_shot=True)
     File "/usr/lib/python3.10/json/encoder.py", line 257, in iterencode
       return _iterencode(o, 0)
     File "/usr/lib/python3.10/json/encoder.py", line 179, in default
       raise TypeError(f'Object of type {o.__class__.__name__} '
   TypeError: Object of type MyMapping is not JSON serializable
   

   So in that case, using list and dict instead of Sequence and Mapping would be quite deliberate and allow Mypy to catch the invalid usage.

   Is there a way to fix the type alias while keeping it limited to lists and dicts (or subclasses) specifically? 🤔