Skip to content

Can't use Callable[..., T] | Callable[..., Awaitable[T]] as a function argument #14669

Open
Open
Can't use Callable[..., T] | Callable[..., Awaitable[T]] as a function argument#14669
Labels
bugmypy got something wrong

Description

@patrick91
patrick91
opened on Feb 10, 2023

Code:

I was trying to define a callable that can be async or not and use a TypeVar as the return type, but it seem to be broken, here's the full example:

from typing import Union, Awaitable, Callable, TypeVar, Any

T = TypeVar("T")

_RESOLVER_TYPE = Union[
    Callable[..., T],
    Callable[..., Awaitable[T]],
]

def x() -> int:
    return 1
    
async def a_x() -> int:
    return 1
    
    
def field_ok(
    resolver: _RESOLVER_TYPE,
) -> Any:
    ...
    
def field_broken(
    resolver: _RESOLVER_TYPE[T],
) -> Any:
    ...
    
field_ok(x)
field_ok(a_x)

field_broken(x)
field_broken(a_x)

if I don't pass T it seem to work, no sure what is wrong, the error is:

main.py:31: error: Argument 1 to "field_broken" has 
    incompatible type "Callable[[], Coroutine[Any, Any, int]]"; 
    expected "Union[Callable[..., <nothing>], Callable[..., Awaitable[<nothing>]]]"  [arg-type]

Here's the playground url: https://mypy-play.net/?mypy=latest&python=3.11&gist=6d8ed4a4e55e0d6b2712eff23cd3e3b0

Activity

  1. patrick91
    added
    bugmypy got something wrong
    on Feb 10, 2023

  2. patrick91 commented on Feb 12, 2023

    @patrick91
    patrick91
    on Feb 12, 2023
    ContributorAuthor

    I don't know if this helps, but while trying to make a test for this I noticed that adding [builtins fixtures/tuple.pyi] makes the problem disappear. here's the test for reference:

    [case testCallableAsyncUnion]
[builtins fixtures/tuple.pyi]
from typing import Union, Awaitable, Callable, TypeVar, Any

T = TypeVar("T")

_RESOLVER_TYPE = Union[
    Callable[..., T],
    Callable[..., Awaitable[T]],
]

def x() -> int:
    return 1

async def a_x() -> int:
    return 1

def field_broken(
    resolver: _RESOLVER_TYPE[T],
) -> Any:
    ...

field_broken(x)
field_broken(a_x)

  3. A5rocks commented on Feb 13, 2023

    @A5rocks
    A5rocks
    on Feb 13, 2023
    Collaborator

    For test cases, put the fixtures at the end (that may be why, though it may be a difference in typeshed vs fixtures in which case we should fix that)

  4. patrick91 commented on Feb 13, 2023

    @patrick91
    patrick91
    on Feb 13, 2023
    ContributorAuthor

    @A5rocks moving it at the end doesn't seem to change anything :)

    I can send a PR with the test, if that helps 😊

    Also for now I changed my code to look like this:

    _RESOLVER_TYPE = Union[
    Callable[..., T],
    Callable[..., Coroutine[T, None, None]],
]

    which "fixed" my original problem :)

  5. fjarri commented on Feb 19, 2023

    @fjarri
    fjarri
    on Feb 19, 2023

    Unfortunately it does not fix things if the signature with Awaitable is in another library (currently hitting this problem with trio's Nursery.start_soon() which expects something returning an Awaitable). This worked in mypy==0.991.

  6. A5rocks commented on Feb 25, 2023

    @A5rocks
    A5rocks
    on Feb 25, 2023
    Collaborator

    @patrick91 Sorry for the delay -- I've finally tried out the test... and it seemed to work?

    I stuck this at the bottom of my test-data/unit/check-type-aliases.test:

    
[case testTestTestTest]
from typing import Union, Awaitable, Callable, TypeVar, Any

T = TypeVar("T")

_RESOLVER_TYPE = Union[
    Callable[..., T],
    Callable[..., Awaitable[T]],
]

def x() -> int:
    return 1
    
async def a_x() -> int:
    return 1
    
    
def field_ok(
    resolver: _RESOLVER_TYPE,
) -> Any:
    ...
    
def field_broken(
    resolver: _RESOLVER_TYPE[T],
) -> Any:
    ...
    
field_ok(x)
field_ok(a_x)

field_broken(x)
field_broken(a_x)
[builtins fixtures/tuple.pyi]

    And then I ran pytest -k testTestTestTest. pytest output:

    (venv) PS C:\Users\A5rocks\Documents\mypy> pytest -k testTestTestTest
================================================= test session starts =================================================
platform win32 -- Python 3.10.9, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: C:\Users\A5rocks\Documents\mypy, configfile: pytest.ini, testpaths: mypy/test, mypyc/test
plugins: cov-2.12.1, forked-1.3.0, xdist-1.34.0
gw0 [1] / gw1 [1] / gw2 [1] / gw3 [1] / gw4 [1] / gw5 [1] / gw6 [1] / gw7 [1] / gw8 [1] / gw9 [1] / gw10 [1] / gw11 [1]
F                                                                                                                [100%]
====================================================== FAILURES =======================================================
__________________________________________________ testTestTestTest ___________________________________________________
[gw0] win32 -- Python 3.10.9 C:\Users\A5rocks\Documents\mypy\venv\Scripts\python.exe
data: C:\Users\A5rocks\Documents\mypy\test-data\unit\check-type-aliases.test:1032:
..\..\..\..\Documents\mypy\mypy\test\testcheck.py:86: in run_case
    self.run_case_once(testcase)
..\..\..\..\Documents\mypy\mypy\test\testcheck.py:181: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
E   AssertionError: Unexpected type checker output (C:\Users\A5rocks\Documents\mypy\test-data\unit\check-type-aliases.test, line 1032)
------------------------------------------------ Captured stderr call -------------------------------------------------
Expected:
Actual:
  main:31: error: Argument 1 to "field_broken" has incompatible type "Callable[[], Coroutine[Any, Any, int]]"; expected "Union[Callable[..., <nothing>], Callable[..., Awaitable[<nothing>]]]" (diff)

=============================================== short test summary info ===============================================
FAILED mypy/test/testcheck.py::TypeCheckSuite::check-type-aliases.test::testTestTestTest
================================================== 1 failed in 7.00s ==================================================

    Do you have specifics on what you're doing? (A link to a branch would work, or a PR as you suggested)

  7. A5rocks commented on Feb 25, 2023

    @A5rocks
    A5rocks
    on Feb 25, 2023
    Collaborator

    My best guess is this is another impact of the type alias typevar change (the one which allowed paramspecs as type aliases, #14159). In which case, this is probably just a small missed detail. This hunch is based on the fact mypy seems to be inferring the typevar out and that it worked in 0.991 -- IDK any other disruptive change that impacted type aliases during that time but I may be completely misremembering :P

    cc @ilevkivskyi since you made that change (in case you want to correct me or take a closer look at this -- I haven't done so yet! All the above is simply conjecture)

    I should have verified before sending this -- this is present in the commit before that! I'll bisect this.

    This worked in mypy==0.991.

    I can't reproduce this working in mypy 0.991. This breakage doesn't seem recent.

  8. fjarri commented on May 11, 2023

    @fjarri
    fjarri
    on May 11, 2023

    I can't reproduce this working in mypy 0.991. This breakage doesn't seem recent.

    Sorry, just noticed your edit.

    This commit: fjarri/nucypher-async@25718f2 passes with mypy 0.991 (ok, there is one fail, but it is unrelated to this issue), but reports a number of problems with Awaitable with mypy 1.3.0:

    nucypher_async/utils/__init__.py:30: error: Argument 1 to "start_soon" of "Nursery" has incompatible type "Callable[[Event], Coroutine[Any, Any, None]]"; expected "Callable[[__T1], Awaitable[Any]]"  [arg-type]
nucypher_async/mocks/asgi.py:28: error: Argument 1 to "start_soon" of "Nursery" has incompatible type "Callable[[Union[HTTPScope, WebsocketScope, LifespanScope], Callable[[], Awaitable[Union[HTTPRequestEvent, HTTPDisconnectEvent, WebsocketConnectEvent, WebsocketReceiveEvent, WebsocketDisconnectEvent, LifespanStartupEvent, LifespanShutdownEvent]]], Callable[[Union[HTTPResponseStartEvent, HTTPResponseBodyEvent, HTTPServerPushEvent, HTTPEarlyHintEvent, HTTPDisconnectEvent, WebsocketAcceptEvent, WebsocketSendEvent, WebsocketResponseStartEvent, WebsocketResponseBodyEvent, WebsocketCloseEvent, LifespanStartupCompleteEvent, LifespanStartupFailedEvent, LifespanShutdownCompleteEvent, LifespanShutdownFailedEvent]], Awaitable[None]]], Awaitable[None]]"; expected "Callable[[__T1, __T2, __T3], Awaitable[Any]]"  [arg-type]
nucypher_async/p2p/learner.py:253: error: Argument 1 to "start_soon" of "Nursery" has incompatible type "Callable[[Contact], Coroutine[Any, Any, None]]"; expected "Callable[[__T1], Awaitable[Any]]"  [arg-type]
nucypher_async/p2p/learner.py:255: error: Argument 1 to "start_soon" of "Nursery" has incompatible type "Callable[[Contact], Coroutine[Any, Any, Optional[VerifiedUrsulaInfo]]]"; expected "Callable[[__T1], Awaitable[Any]]"  [arg-type]
nucypher_async/p2p/learner.py:264: error: Argument 1 to "start_soon" of "Nursery" has incompatible type "Callable[[VerifiedUrsulaInfo], Coroutine[Any, Any, None]]"; expected "Callable[[__T1], Awaitable[Any]]"  [arg-type]
nucypher_async/p2p/algorithms.py:132: error: Argument 1 to "start_soon" of "Nursery" has incompatible type "Callable[[Contact], Coroutine[Any, Any, Optional[VerifiedUrsulaInfo]]]"; expected "Callable[[__T1], Awaitable[Any]]"  [arg-type]
nucypher_async/p2p/algorithms.py:144: error: Argument 1 to "start_soon" of "Nursery" has incompatible type "Callable[[Contact], Coroutine[Any, Any, Optional[VerifiedUrsulaInfo]]]"; expected "Callable[[__T1], Awaitable[Any]]"  [arg-type]
nucypher_async/p2p/algorithms.py:218: error: Argument 1 to "start_soon" of "Nursery" has incompatible type "Callable[[Contact], Coroutine[Any, Any, None]]"; expected "Callable[[__T1], Awaitable[Any]]"  [arg-type]
nucypher_async/client/pre.py:154: error: Argument 1 to "start_soon" of "Nursery" has incompatible type "Callable[[Nursery, VerifiedUrsulaInfo], Coroutine[Any, Any, None]]"; expected "Callable[[__T1, __T2], Awaitable[Any]]"  [arg-type]

    (Nursery is a type from the trio package)

  9. patrick91 commented on Jun 15, 2023

    @patrick91
    patrick91
    on Jun 15, 2023
    ContributorAuthor

    @A5rocks sorry for the radio silence, I just tested this again and it seems to work with the latest version of mypy 😊

    my final type looks like this:

    _RESOLVER_TYPE = Union[
    Callable[..., T],
    Callable[..., Coroutine[T, Any, Any]],
    Callable[..., Awaitable[T]],
    "staticmethod[Any, T]",
    "classmethod[Any, Any, T]",
]

  10. A5rocks commented on Jun 15, 2023

    @A5rocks
    A5rocks
    on Jun 15, 2023
    Collaborator

    Hmm, wacky. I meant to check the repository but completely forgot to. It's good that it's fixed now, nonetheless!

    EDIT: I forgot that your original example still fails and has done so since... a while. That should still be fixed, I suppose.

  11. patrick91 commented on Jun 15, 2023

    @patrick91
    patrick91
    on Jun 15, 2023
    ContributorAuthor

    Hmm, wacky. I meant to check the repository but completely forgot to. It's good that it's fixed now, nonetheless!

    EDIT: I forgot that your original example still fails and has done so since... a while. That should still be fixed, I suppose.

    Yes, I needed to keep the Callable[..., Coroutine[T, Any, Any]], but also add the awaitable one after removing some parts of our plugin here: https://github.com/strawberry-graphql/strawberry/pull/2852/files#diff-00af9d43c9b59404ba170e72e470939f2a6dd50e2eae24d0e24ef105431e5414L46-L48

    I'll leave this issue open then 😊

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugmypy got something wrong

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions