Generic types that use ParamSpec aren't callable

from functools import partial
from typing import Callable, Generic, TypeVar

from typing_extensions import ParamSpec

R = TypeVar('R')
P = ParamSpec('P')

class custom_vjp(Generic[P, R]):
    def __init__(self, fun: Callable[P, R], x: int = 0):
        self.fun = fun

    def __call__(self, *args: P.args, **kwargs: P.kwargs) -> R:
        return self.fun(*args, **kwargs)

partial(custom_vjp, x=0)  # error: Argument 1 to "partial" has incompatible type "Type[custom_vjp[Any, Any]]"; expected "Callable[..., custom_vjp[P, R]]"  [arg-type]

Types are callable in the general case: https://mypy-play.net/?mypy=latest&python=3.10&gist=a18ffb3451fb8341f3f89341edebab5b

I'm not sure what's going on here; I suspect some weird interaction with ParamSpec, though.

Maybe due to existence of self? In such cases, we need to use typing.Concatenate introduced in Python 3.10, but mypy does not yet support it.

@achimnol

@achimnol How would you use Concatenate here?

Hm... I think I'm wrong about using Concatenate here.

The following code is well accepted and type-checked by mypy,

def myfun(x: int) -> str:
    return 'b' + str(x)

print('a' + custom_vjp(myfun, x=0)(123))  # accepted
print(1 + custom_vjp(myfun, x=0)(123))    # rejected

but here the problem seems to lie in the type inference of functools.partial against the class constructor.

pyright accepts your example while mypy shows an error as you reported.

Though, your example may be converted to:

def custom_vjp(func: Callable[P, R], x: int = 0) -> Callable[P, R]:
    def inner(*args: P.args, **kwargs: P.kwargs) -> R:
        return func(*args, **kwargs)
    return inner

partial(custom_vjp, x=0)  # accepted & type-checked well

unless you are using other states and methods in custom_vjp.

OK, investigated a bit and it looks like it's due to ParamSpecs not being treated right in subtype checking: (*Any, **Any) -> Any should probably be a supertype of an arbitrary P, I think.

I believe this is due to a missing special case in subtype checking here (is_subtype(left=def (*Any, **Any) -> Any, right=ParamSpec)):

mypy/mypy/subtypes.py

Lines 333 to 362 in ef43416

    
           def visit_callable_type(self, left: CallableType) -> bool: 
        
               right = self.right 
        
               if isinstance(right, CallableType): 
        
                   if left.type_guard is not None and right.type_guard is not None: 
        
                       if not self._is_subtype(left.type_guard, right.type_guard): 
        
                           return False 
        
                   elif right.type_guard is not None and left.type_guard is None: 
        
                       # This means that one function has `TypeGuard` and other does not. 
        
                       # They are not compatible. See https://github.com/python/mypy/issues/11307 
        
                       return False 
        
                   return is_callable_compatible( 
        
                       left, right, 
        
                       is_compat=self._is_subtype, 
        
                       ignore_pos_arg_names=self.ignore_pos_arg_names) 
        
               elif isinstance(right, Overloaded): 
        
                   return all(self._is_subtype(left, item) for item in right.items) 
        
               elif isinstance(right, Instance): 
        
                   if right.type.is_protocol and right.type.protocol_members == ['__call__']: 
        
                       # OK, a callable can implement a protocol with a single `__call__` member. 
        
                       # TODO: we should probably explicitly exclude self-types in this case. 
        
                       call = find_member('__call__', right, left, is_operator=True) 
        
                       assert call is not None 
        
                       if self._is_subtype(left, call): 
        
                           return True 
        
                   return self._is_subtype(left.fallback, right) 
        
               elif isinstance(right, TypeType): 
        
                   # This is unsound, we don't check the __init__ signature. 
        
                   return left.is_type_obj() and self._is_subtype(left.ret_type, right.item) 
        
               else: 
        
                   return False

Compare what will happen for is_subtype(left=typing.Any, right=TypeVar):

mypy/mypy/subtypes.py

Lines 220 to 221 in ef43416

    
           def visit_any(self, left: AnyType) -> bool: 
        
               return True

This might be an easy first PR for someone wanting that :P (just check that right is ParamSpecType and that left is def (*Any, **Any) -> Any, I believe?)

I ran into this in some code I was writing myself and can probably fix it (once my existing mypy PR gets merged, I really don't want to have to think about multiple PRs at once). If someone beats me to it, great!

Original example works on master, likely fixed by #15837

Navigation Menu

Repository navigation

Generic types that use ParamSpec aren't callable #11846

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

	def visit_callable_type(self, left: CallableType) -> bool:
	right = self.right
	if isinstance(right, CallableType):
	if left.type_guard is not None and right.type_guard is not None:
	if not self._is_subtype(left.type_guard, right.type_guard):
	return False
	elif right.type_guard is not None and left.type_guard is None:
	# This means that one function has `TypeGuard` and other does not.
	# They are not compatible. See https://github.com/python/mypy/issues/11307
	return False
	return is_callable_compatible(
	left, right,
	is_compat=self._is_subtype,
	ignore_pos_arg_names=self.ignore_pos_arg_names)
	elif isinstance(right, Overloaded):
	return all(self._is_subtype(left, item) for item in right.items)
	elif isinstance(right, Instance):
	if right.type.is_protocol and right.type.protocol_members == ['__call__']:
	# OK, a callable can implement a protocol with a single `__call__` member.
	# TODO: we should probably explicitly exclude self-types in this case.
	call = find_member('__call__', right, left, is_operator=True)
	assert call is not None
	if self._is_subtype(left, call):
	return True
	return self._is_subtype(left.fallback, right)
	elif isinstance(right, TypeType):
	# This is unsound, we don't check the __init__ signature.
	return left.is_type_obj() and self._is_subtype(left.ret_type, right.item)
	else:
	return False

Generic types that use ParamSpec aren't callable #11846

Description

Activity

A5rocks commented on Dec 26, 2021

achimnol commented on Dec 27, 2021

NeilGirdhar commented on Dec 27, 2021

achimnol commented on Dec 28, 2021

achimnol commented on Dec 28, 2021

A5rocks commented on Dec 29, 2021

A5rocks commented on Jan 26, 2022

ilevkivskyi commented on Aug 16, 2023

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions