Incorrect hints for TestCase.assertRaises

Pymongo's mypy tests pass with mypy==0.942 but fail with mypy==0.971 with this error:

test/test_bson.py: note: In member "assertInvalid" of class "TestBSON":
test/test_bson.py:120: error: Argument 2 to "assertRaises" of "TestCase" has incompatible type
"Callable[[Union[bytes, memoryview, mmap, array[Any]], Optional[CodecOptions[_DocumentType]]], _DocumentType]"; expected "Callable[..., object]" 
[arg-type]
            self.assertRaises(InvalidBSON, decode, data)
                                           ^

I believe this was a regression caused by the changes in #7012

What is _DocumentType here? The error message may indicate a mypy bug, since Callable is covariant in its return type and every type should be compatible with object.

decode() is defined here:

def decode(
    data: _ReadableBuffer, codec_options: "Optional[CodecOptions[_DocumentType]]" = None
) -> _DocumentType:
    ...

and _DocumentType is defined here:

class CodecOptions(Tuple, Generic[_DocumentType]):
    document_class: Type[_DocumentType]
    tz_aware: bool
    uuid_representation: int
    unicode_decode_error_handler: Optional[str]
    tzinfo: Optional[datetime.tzinfo]
    type_registry: TypeRegistry

    def __new__(
        cls: Type[CodecOptions],
        document_class: Optional[Type[_DocumentType]] = ...,
        tz_aware: bool = ...,
        uuid_representation: Optional[int] = ...,
        unicode_decode_error_handler: Optional[str] = ...,
        tzinfo: Optional[datetime.tzinfo] = ...,
        type_registry: Optional[TypeRegistry] = ...,
    ) -> CodecOptions[_DocumentType]: ...

Thanks! Maybe it has something to do with the TypeVar, but I'm not having much luck trying to reproduce this in a smaller example. https://mypy-play.net/?mypy=latest&python=3.10&gist=1f95669029e61cff3091eb3b75c4e2bd

Here's the smallest repro I could get:

import unittest
from typing import TypeVar, Any, Mapping, Optional

T = TypeVar("T", bound=Mapping[str, Any])

def raises(opts: Optional[T] = None) -> T:
    if opts is None:
        raise TypeError()
    return opts

class TestAssertRaises(unittest.TestCase):
    def test_assertRaises(self) -> None:
        self.assertRaises(TypeError, raises, None)

output:

mypy --strict repro-mypy-bug.py
repro-mypy-bug.py: note: In member "test_assertRaises" of class "TestAssertRaises":
repro-mypy-bug.py:13: error: Argument 2 to "assertRaises" of "TestCase" has incompatible type "Callable[[Optional[T]], T]"; expected
"Callable[..., object]"  [arg-type]
            self.assertRaises(TypeError, raises, None)
                                         ^
Found 1 error in 1 file (checked 1 source file)

@AlexWaygood

Thanks! Here's a repro for the mypy bug that doesn't rely on the stubs:

from typing import Callable, TypeVar, Any, Mapping, Optional
T = TypeVar("T", bound=Mapping[str, Any])
def raises(opts: Optional[T]) -> T: pass
def assertRaises(cb: Callable[..., object]) -> None: pass
assertRaises(raises)

main.py:11: error: Argument 1 to "assertRaises" has incompatible type "Callable[[Optional[T]], T]"; expected "Callable[..., object]"

@AlexWaygood given this mypy bug, what do you think of returning to Callable[..., Any] for now?

Opened python/mypy#13220 to track the mypy issue

@AlexWaygood

@AlexWaygood given this mypy bug, what do you think of returning to Callable[..., Any] for now?

Sounds good to me.

I merged #8373. Is there a risk that this mypy bug could hit users for any of the other functions that were changed in #7012? Or are we good to leave this closed now?

It could hit in other cases, I think the main ingredient is the constrained TypeVar. Not sure how likely it is to actually come up though.

Reopening. We should at least document why we use Any in the PR, instead of object. That said, I'd prefer if mypy would fix their bug, or alternatively that mypy patches their version of typeshed when shipping the next release.

I can confirm that the issue is the bound TypeVar (and possible constrained TypeVars have a similar problem). I tried several ways of fixing, but they all broke a bunch of stuff -- curious if anyone has ideas on alternate mypy fixes.

Is there a risk that this mypy bug could hit users for any of the other functions that were changed in #7012? Or are we good to leave this closed now?

My mistake, the other functions also need to be changed to use Any, for example addCleanup is also broken:

test/test_change_stream.py: note: In member "setFailPoint" of class "TestAllLegacyScenarios":
test/test_change_stream.py:1088: error: Argument 1 to "addCleanup" of
"TestCase" has incompatible type
"Callable[[Union[str, MutableMapping[str, Any]], Any, bool, Optional[Sequence[Union[str, int]]], Optional[_ServerMode], Optional[CodecOptions[_CodecDocumentType]], Optional[ClientSession], Optional[Any], KwArg(Any)], _CodecDocumentType]";
expected
"Callable[[Union[str, MutableMapping[str, Any]], Any, bool, Optional[Sequence[Union[str, int]]], Optional[_ServerMode], Optional[CodecOptions[_CodecDocumentType]], Optional[ClientSession], Optional[Any], KwArg(Any)], object]"
 [arg-type]
                client_context.client.admin.command,
                ^

Navigation Menu

Repository navigation

Incorrect hints for TestCase.assertRaises #8372

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Incorrect hints for TestCase.assertRaises #8372

Description

Activity

JelleZijlstra commented on Jul 23, 2022

ShaneHarvey commented on Jul 23, 2022

JelleZijlstra commented on Jul 23, 2022

ShaneHarvey commented on Jul 23, 2022

JelleZijlstra commented on Jul 23, 2022

hauntsaninja commented on Jul 23, 2022

AlexWaygood commented on Jul 23, 2022

AlexWaygood commented on Jul 23, 2022

JelleZijlstra commented on Jul 23, 2022

srittau commented on Jul 23, 2022

hauntsaninja commented on Jul 23, 2022

ShaneHarvey commented on Jul 25, 2022

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions