object.__eq__ and other comparisons should have return type Any

@JukkaL

@JukkaL commented on Feb 1, 2017
I wonder if the return type of object.__eq__ in typeshed should be object, since the bool return type is not quite correct. The same applies to __lt__ and friends.

Given the implementation, I would suggest the return type should be Union[NoReturn, Any]. object.__eq__ always throws a NotImplementedError (hence the NoReturn) and Any allows sub-classes to override the return type to anything they like.

Originally posted by @Victor-Savu in python/mypy#2783 (comment)

Please fix the return type of __eq__ and others in typeshed as described in the comment above. __eq__ not returning a boolean is very common, most notably NumPy arrays performing elementwise comparisons.

Please see the thread linked above for details. The statement about raising NotImplementedError was corrected tot returning NotImplemented, so defining Any as the return type should be what we need.

I would be happy to open a PR …

I am honestly not convinced. I would argue that the default assumption is that __eq__ and other binary comparison methods should return a boolean value. If some classes return another type, they can always override the return type, potentially using # type: ignore.

@jonasrauber

@jonasrauber I very much agree that something needs to be done about __eq__, but there are many nuances around it so I'd recommend discussing with the mypy maintainers before putting in significant time to prep a PR. It seems that there may be some baked-in assumptions and special handling beyond what typeshed dictates.

Related discussions:
python/mypy#2783
python/mypy#5704
python/mypy#6710

This is a case where # type: ignore should be used if non-standard behavior is desired.

NoReturn should be a bottom type, a subtype of every type. So Union[NoReturn, Any] is just Any.

Any is not what you want because then you have the undesirable property that fully statically typed code will contain dynamic types (types containing Any). object is the return type you want because (1) it faithfully matches the implementation and (2) it does allow subclasses to correctly override the method.

This is a case where # type: ignore should be used if non-standard behavior is desired.

OK, but I think you should reconsider closing this instead of fixing it.

The non-standard behavior is the annotation in typeshed. It's non-standard because it disagrees with the runtime semantics via implementation of CPython (the only real standard we have for Python). This program has a type error:

def f(x: object) -> int:
  return x == 0

But a static type checker that uses the typing in typeshed will not catch it without special handling of the type annotations in class object. I would say instead that the annotations in typeshed should be correct and then that checkers could implement something else if they wanted to.

This program has a type error

No it doesn't. object.__eq__ returns a bool, and a bool is an int.

Are we both talking about the runtime behavior? x == 0 doesn't necessarily invoke object.__eq__. What matters is if the runtime ensures that x == 0 is always bool; and it doesn't. You can decide that the static type of x == 0 is bool but that seems like a bad description of the runtime behavior.

I suspect the point was that

def f(x: object) -> int:
  return x == 0

has no type errors assuming that the type of x contains a compatible overload of object.__eq__. However, this is not true for all classes, including the very widely used np.ndarray. Imagine we call f with an instance of such class

class A:
  def __eq__(self, other) -> str:
    return "YES"

f(A())

A.__eq__ is clearly incompatible with object.__eq__, because str is not a subtype of bool. If we hide that incompatibility via # type: ignore, we will force the type checker to accept an otherwise unsafe f(...) call. Is this WAI?

@kmillikin

This is a case where # type: ignore should be used if non-standard behavior is desired.

Rich comparisons have been in Python since 2.1. There's nothing non-standard about it, as @kmillikin said.

I have some snippets here to speed up debugging (I do that with pytest . --pdb) that will require tons of # type: ignore statements if I use this more:

class Falsy:
    def __init__(self, reason: str) -> None:
        """Record description."""
        self.reason = reason

    def __bool__(self) -> typing.Literal[False]:
        """This class is always False."""
        return False

    def __eq__(self, other: typing.Any) -> bool:
        """Equal to falsy values."""
        return not other

    def __repr__(self) -> str:
        """Print the reason instead of just False."""
        return f'Falsy("{self.reason}")'


Boolean = Falsy | bool

class SomeClass:
    ...
    def __eq__(self, other: Any) -> Boolean:  # type: ignore
        for attr in self._defining_attributes:
            try:
                selfattr = getattr(self, attr)
                otherattr = getattr(other, attr)
            except AttributeError:
                return NotImplemented
            if selfattr != otherattr:
                return Falsy(f"{selfattr} != {otherattr}")
        return True

I also don't understand how this is supposed to work with Numpy.

Perhaps we need something like collections.abc.Boolean?

I'm here because SQLAlchemy also has __eq__ (and others) returning non-bools. I had to convince myself this works with a little test program and some googling, but I'm seeing SQLAlchemy return a BinaryExpression object for use in Object Relational Mapper (ORM) queries:
https://docs.sqlalchemy.org/en/20/core/sqlelement.html#sqlalchemy.sql.expression.BinaryExpression

Actually, I believe that Brython uses <= a little interestingly too:
https://brython.info/static_doc/en/faq.html

My first reaction to Brython's was sadness, but in the larger context of SQLAlchemy, Numpy and Brython, I'm getting over it. :)

Here's my test program:
$ cat /tmp/denb
below cmd output started 2023 Mon May 08 01:20:39 PM PDT

#!/usr/bin/env python3

class Foo:
    def __init__(self):
        pass

    def __eq__(self, other):
        return "abc"


foo1 = Foo()
foo2 = Foo()
print(foo1 == foo2)

above cmd output done    2023 Mon May 08 01:20:39 PM PDT
dstromberg@zareason2:~ x86_64-pc-linux-gnu 1421091

$ pythons --file /tmp/denb
below cmd output started 2023 Mon May 08 01:20:45 PM PDT
/usr/local/cpython-0.9/bin/python (unknown) bad
    Parsing error: file /tmp/denb, line 3:
    class Foo:
              ^
    Unhandled exception: run-time error: syntax error
/usr/local/cpython-1.0/bin/python (1.0.1) good 0
/usr/local/cpython-1.1/bin/python (1.1) good 0
/usr/local/cpython-1.2/bin/python (1.2) good 0
/usr/local/cpython-1.3/bin/python (1.3) good 0
/usr/local/cpython-1.4/bin/python (1.4) good 0
/usr/local/cpython-1.5/bin/python (1.5.2) good 0
/usr/local/cpython-1.6/bin/python (1.6.1) good 0
/usr/local/cpython-2.0/bin/python (2.0.1) good 0
/usr/local/cpython-2.1/bin/python (2.1.0) good abc
/usr/local/cpython-2.2/bin/python (2.2.0) good abc
/usr/local/cpython-2.3/bin/python (2.3.0) good abc
/usr/local/cpython-2.4/bin/python (2.4.0) good abc
/usr/local/cpython-2.5/bin/python (2.5.6) good abc
/usr/local/cpython-2.6/bin/python (2.6.9) good abc
/usr/local/cpython-2.7/bin/python (2.7.16) good abc
/usr/local/cpython-3.0/bin/python (3.0.1) good abc
/usr/local/cpython-3.1/bin/python (3.1.5) good abc
/usr/local/cpython-3.2/bin/python (3.2.5) good abc
/usr/local/cpython-3.3/bin/python (3.3.7) good abc
/usr/local/cpython-3.4/bin/python (3.4.8) good abc
/usr/local/cpython-3.5/bin/python (3.5.5) good abc
/usr/local/cpython-3.6/bin/python (3.6.13) good abc
/usr/local/cpython-3.7/bin/python (3.7.0) good abc
/usr/local/cpython-3.8/bin/python (3.8.0) good abc
/usr/local/cpython-3.9/bin/python (3.9.0) good abc
/usr/local/cpython-3.10/bin/python (3.10.0) good abc
/usr/local/cpython-3.11/bin/python (3.11.0) good abc
/usr/local/pypy3-7.3.9/bin/pypy3 (3.9.12) good abc

 1 bad  **
28 good ************************************************************

So apparently this has worked since 2.1.0.

I have encountered the same problem. I am making a query builder for specific needs and it is strange that __eq__ should return bool while other operators shouldn't and there are really popular projects like SQLAlchemy and numpy using rich comparison.

Additionally, I want to quote pep-0207

This PEP proposes several new features for comparisons:

Allow separately overloading of <, >, <=, >=, ==, !=, both in classes and in C extensions.

Allow any of those overloaded operators to return something else besides a Boolean result.

One thing that really stands out about Python compared to other languages, is that it is a great host for embedded domain specific languages (DSLs). This bug means that type checkers that use typeshed for typing == will give false positives for such code, which is too bad.

Put another way: this typing is (gratuitously) unsound with respect to the runtime semantics.

This bug means that type checkers that use typeshed for typing == will give false positives for such code, which is too bad.

As mentioned before, type checkers will happily accept those comparisons if the LSP violation in the overloading classes is # type: ignored.

class Foo:
    def __eq__(self, other: object) -> str:  # type: ignore[override]
        return "Hey!"
    
reveal_type(Foo() == Foo())  # revealed type is `str`

While overloading the operators in common, it is an LSP violation and unsound. Using # type: ignore explicitly to signal intent is the best option. I don't see use changing this. The only completely safe option would be to change the return type to object, which is unreasonable.

@srittau

it is an LSP violation and unsound.

@srittau Can you please give us a reference to prove it? According to PEP207 it is not required to return bool for any comparison operator. I do not see there any difference between == and <= operators there

__le__ (and the other comparison operators) is not defined on object in typeshed, which is likely an oversight and should probably be fixed, although that might prove too disruptive. Apart from that I'm not interested in discussing this any further.

Navigation Menu

Repository navigation

object.eq and other comparisons should have return type Any #3685

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

object.__eq__ and other comparisons should have return type Any #3685

Description

Activity

jonasrauber commented on Jan 29, 2020

jonasrauber commented on Jan 29, 2020

srittau commented on Jan 29, 2020

llchan commented on Feb 6, 2020

srittau commented on Feb 8, 2022

kmillikin commented on Feb 8, 2022

kmillikin commented on Feb 8, 2022

JelleZijlstra commented on Feb 8, 2022

kmillikin commented on Feb 8, 2022

superbobry commented on Feb 9, 2022

joooeey commented on Nov 18, 2022

dstromberg commented on May 8, 2023

Tishka17 commented on Nov 21, 2024

kmillikin commented on Nov 21, 2024

srittau commented on Nov 21, 2024

Tishka17 commented on Nov 21, 2024

srittau commented on Nov 21, 2024

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

object.eq and other comparisons should have return type Any #3685