Exhaustive Matching on Enums / Literals

@bluetech

Are you reporting a bug, or opening a feature request?

Asking a question. I'd like to know if there is a standard idiom for enforcing exhaustive matching of an Enum / Literal type.

Please insert below the code you are checking with mypy

Suppose there are three types of pets: cats, dogs and iguanas. I want to write a function that makes the appropriate pet sound, depending on the type of pet. I'd also like to make sure that I've caught every possible type of pet. Here was my first attempt, using the NoReturn trick mentioned by @bluetech here. It looks like this works for Unions, but not for Enums / Literals? Apologies if this has been covered before.

def assert_never(x: Any) -> NoReturn:
    assert False, "Unhandled type: {}".format(type(x).__name__)

PetLiteral = Literal['dog', 'cat', 'iguana']

def make_sound_literal(pet: PetLiteral) -> str:
    if pet == 'dog':
        return 'woof'
    elif pet == 'cat':
        return 'meow'
    # elif pet == 'iguana':
    #     return 'confused silence'
    else:
        assert_never(pet)

If I comment out the iguana branch and check this file with:

$ mypy --strict  pet_sounds.py

mypy has no complaints. If I try the same deal with an enum:

class PetEnum(Enum):
    dog: str = 'dog'
    cat: str = 'cat'
    iguana: str = 'iguana'

def make_sound_enum(pet: PetEnum) -> str:
    if pet == PetEnum.dog:
        return 'woof'
    elif pet == PetEnum.cat:
        return 'meow'
#    elif pet == PetEnum.iguana:
#        return 'confused silence'
    else:
        assert_never(pet)

and check again with --strict, I still get no complaints.

What are the versions of mypy and Python you are using?
mypy 0.660
Python 3.6.5
Do you see the same issue after installing mypy from Git master?

Yes, got the same behavior with mypy 0.680+dev.21c8a812c697baf7394eafe360188ededcec6d9c

What are the mypy flags you are using? (For example --strict-optional)
--strict

@Michael0x2a

The assert_never() signature proposed in the issue you reference is a bit different:

def assert_never(x: NoReturn) -> NoReturn:
    assert False, "Unhandled type: {}".format(type(x).__name__)

But with enums and literals it will probably always give you a false positive.

The exhaustive checks are not supported yet for enums and literals. But there are plans to add them, see also #5935. I am not sure about the timeline however, @Michael0x2a will you have time to work on this?

Generalize reachability checks to support enums

Note that #4223 gives an example where this can cause a false positive Missing return statement.

Generalize reachability checks to support enums (#7000)

FWIW a simple Enum example seems to work for me on mypy 0.740:

class MobileOperator(enum.Enum):
    ORANGE = "ORANGE"
    FREE = "FREE"
    EXPRESSO = "EXPRESSO"
    FOO = "FOO"

def _assert_never(x: NoReturn) -> NoReturn:
    assert False, "Unhandled type: {}".format(type(x).__name__)

def _mobile_operator_to_url(operator: MobileOperator) -> str:
    if operator is MobileOperator.ORANGE:
        return "/api/orange/"
    elif operator is MobileOperator.FREE:
        return "/api/free/izi"
    elif operator is MobileOperator.EXPRESSO:
        return "/api/expresso/yakalma"
    else:
        _assert_never(operator)

which gives the type error Argument 1 to "_assert_never" has incompatible type "Literal[MobileOperator.FOO]"; expected "NoReturn". Removing FOO from the enum fixes the error.

And on mypy 0.780 it seems Literal is working too:

def _assert_never(x: NoReturn) -> NoReturn:
    assert False, "Unhandled type: {}".format(type(x).__name__)

def f(x: Literal['a', 'b', 'c']) -> None:
    if x == 'a':
        pass
    elif x == 'b':
        pass
    else:
        _assert_never(x)

correctly errors:

error: Argument 1 to "_assert_never" has incompatible type "Literal['c']"; expected "NoReturn"
Found 1 error in 1 file (checked 1 source file)

Note that you have to use is to test enum equality and not ==

While the Python docs say that members can be compared by identity, the docs also mentioned that equality operators are defined.

I think as long as the Enum doesn't use a mixin, like class Foo(str, Enum): ... or is an IntEnum, then comparing via equality should work for refinement since only other enum members will be equal.

https://docs.python.org/3.8/library/enum.html#comparisons

I was looking into this more and it seems that not narrowing based on == is intended behavior to prevent false positives.

from enum import Enum
from typing import Literal

class Foo(Enum):
    a = "a"
    b = "b"
   
x: Foo
A: Literal[Foo.a]

if x == A:
    reveal_type(x) # note: Revealed type is 'Literal[main.Foo.a]'
else:
    reveal_type(x) # note: Revealed type is 'Literal[main.Foo.b]'

this doesn't:

if x == Foo.a:
    reveal_type(x) # note: Revealed type is 'main.Foo'
else:
    reveal_type(x) # note: Revealed type is 'main.Foo'

There are some test cases explaining the decision around narrowing:

mypy/test-data/unit/check-narrowing.test

Lines 783 to 802 in 4d5a1bc

    
           # See comments in testNarrowingEqualityRequiresExplicitStrLiteral and 
        
           # testNarrowingEqualityFlipFlop for more on why we can't narrow here. 
        
           x1: Foo 
        
           if x1 == Foo.A: 
        
               reveal_type(x1)  # N: Revealed type is '__main__.Foo' 
        
           else: 
        
               reveal_type(x1)  # N: Revealed type is '__main__.Foo' 
        
           x2: Foo 
        
           if x2 == A_final: 
        
               reveal_type(x2)  # N: Revealed type is '__main__.Foo' 
        
           else: 
        
               reveal_type(x2)  # N: Revealed type is '__main__.Foo' 
        
           # But we let this narrow since there's an explicit literal in the RHS. 
        
           x3: Foo 
        
           if x3 == A_literal: 
        
               reveal_type(x3)  # N: Revealed type is 'Literal[__main__.Foo.A]' 
        
           else: 
        
               reveal_type(x3)  # N: Revealed type is 'Literal[__main__.Foo.B]'

mypy/test-data/unit/check-narrowing.test

Lines 681 to 710 in 4d5a1bc

    
           def test1(switch: FlipFlopEnum) -> None: 
        
               # Naively, we might assume the 'assert' here would narrow the type to 
        
               # Literal[State.A]. However, doing this ends up breaking a fair number of real-world 
        
               # code (usually test cases) that looks similar to this function: e.g. checks 
        
               # to make sure a field was mutated to some particular value. 
        
               # 
        
               # And since mypy can't really reason about state mutation, we take a conservative 
        
               # approach and avoid narrowing anything here. 
        
               assert switch.state == State.A 
        
               reveal_type(switch.state)          # N: Revealed type is '__main__.State' 
        
               switch.mutate() 
        
               assert switch.state == State.B 
        
               reveal_type(switch.state)          # N: Revealed type is '__main__.State' 
        
           def test2(switch: FlipFlopEnum) -> None: 
        
               # So strictly speaking, we ought to do the same thing with 'is' comparisons 
        
               # for the same reasons as above. But in practice, not too many people seem to 
        
               # know that doing 'some_enum is MyEnum.Value' is idiomatic. So in practice, 
        
               # this is probably good enough for now. 
        
               assert switch.state is State.A 
        
               reveal_type(switch.state)          # N: Revealed type is 'Literal[__main__.State.A]' 
        
               switch.mutate() 
        
               assert switch.state is State.B     # E: Non-overlapping identity check (left operand type: "Literal[State.A]", right operand type: "Literal[State.B]") 
        
               reveal_type(switch.state)          # E: Statement is unreachable

And a related commit: 3dce3fd

Maybe narrowing enums via equality could be put behind another flag to prevent breaking existing code?

Here's a diff to change the narrowing, but it does require changing some tests:

mypy.diff

diff --git a/mypy/checker.py b/mypy/checker.py
index fce7e7d7..cf25bdb2 100644
--- a/mypy/checker.py
+++ b/mypy/checker.py
@@ -4083,7 +4083,7 @@ class TypeChecker(NodeVisitor[None], CheckerPluginInterface):
                         should_narrow_by_identity = True
                     else:
                         def is_exactly_literal_type(t: Type) -> bool:
-                            return isinstance(get_proper_type(t), LiteralType)
+                            return isinstance(coerce_to_literal(get_proper_type(t)), LiteralType)
 
                         def has_no_custom_eq_checks(t: Type) -> bool:
                             return (not custom_special_method(t, '__eq__', check_all=False)
@@ -4451,7 +4451,7 @@ class TypeChecker(NodeVisitor[None], CheckerPluginInterface):
             singleton_index = possible_target_indices[-1]
 
         enum_name = None
-        target = get_proper_type(target)
+        target = coerce_to_literal(get_proper_type(target))
         if isinstance(target, LiteralType) and target.is_enum_literal():
             enum_name = target.fallback.type.fullname
 
diff --git a/test-data/unit/check-enum.test b/test-data/unit/check-enum.test
index 0da3217b..de6c606e 100644
--- a/test-data/unit/check-enum.test
+++ b/test-data/unit/check-enum.test
@@ -855,6 +855,24 @@ else:
     reveal_type(z)  # No output: this branch is unreachable
 [builtins fixtures/bool.pyi]
 
+[case testEnumReachabilityChecksEquality]
+from enum import Enum
+from typing_extensions import Literal
+
+class Foo(Enum):
+    A = 1
+    B = 2
+    C = 3
+
+x: Foo
+
+if x == Foo.A:
+    reveal_type(x)  # N: Revealed type is 'Literal[__main__.Foo.A]'
+else:
+    reveal_type(x)  # N: Revealed type is 'Union[Literal[__main__.Foo.B], Literal[__main__.Foo.C]]'
+
+[builtins fixtures/bool.pyi]
+
 [case testEnumReachabilityNoNarrowingForUnionMessiness]
 from enum import Enum
 from typing_extensions import Literal
diff --git a/test-data/unit/check-narrowing.test b/test-data/unit/check-narrowing.test
index 8eb6f643..5eb356c6 100644
--- a/test-data/unit/check-narrowing.test
+++ b/test-data/unit/check-narrowing.test
@@ -688,12 +688,12 @@ def test1(switch: FlipFlopEnum) -> None:
     # approach and avoid narrowing anything here.
 
     assert switch.state == State.A
-    reveal_type(switch.state)          # N: Revealed type is '__main__.State'
+    reveal_type(switch.state)          # N: Revealed type is 'Literal[__main__.State.A]'
 
-    switch.mutate()
+    switch.mutate()                    
 
-    assert switch.state == State.B
-    reveal_type(switch.state)          # N: Revealed type is '__main__.State'
+    assert switch.state == State.B     # E: Non-overlapping equality check (left operand type: "Literal[State.A]", right operand type: "Literal[State.B]")
+    reveal_type(switch.state)          # E: Statement is unreachable
 
 def test2(switch: FlipFlopEnum) -> None:
     # So strictly speaking, we ought to do the same thing with 'is' comparisons
@@ -713,12 +713,12 @@ def test3(switch: FlipFlopStr) -> None:
     # This is the same thing as 'test1', except we try using str literals.
 
     assert switch.state == "state-1"
-    reveal_type(switch.state)          # N: Revealed type is 'builtins.str'
+    reveal_type(switch.state)          # N: Revealed type is 'Literal['state-1']'
 
     switch.mutate()
 
-    assert switch.state == "state-2"
-    reveal_type(switch.state)          # N: Revealed type is 'builtins.str'
+    assert switch.state == "state-2"   # E: Non-overlapping equality check (left operand type: "Literal['state-1']", right operand type: "Literal['state-2']")
+    reveal_type(switch.state)          # E: Statement is unreachable
 [builtins fixtures/primitives.pyi]
 
 [case testNarrowingEqualityRequiresExplicitStrLiteral]
@@ -733,13 +733,13 @@ A_literal: Literal["A"]
 # why more precise inference here is problematic.
 x_str: str
 if x_str == "A":
-    reveal_type(x_str)  # N: Revealed type is 'builtins.str'
+    reveal_type(x_str)  # N: Revealed type is 'Literal['A']'
 else:
     reveal_type(x_str)  # N: Revealed type is 'builtins.str'
 reveal_type(x_str)      # N: Revealed type is 'builtins.str'
 
 if x_str == A_final:
-    reveal_type(x_str)  # N: Revealed type is 'builtins.str'
+    reveal_type(x_str)  # N: Revealed type is 'Literal['A']'
 else:
     reveal_type(x_str)  # N: Revealed type is 'builtins.str'
 reveal_type(x_str)      # N: Revealed type is 'builtins.str'
@@ -784,15 +784,15 @@ A_literal: Literal[Foo.A]
 # testNarrowingEqualityFlipFlop for more on why we can't narrow here.
 x1: Foo
 if x1 == Foo.A:
-    reveal_type(x1)  # N: Revealed type is '__main__.Foo'
+    reveal_type(x1)  # N: Revealed type is 'Literal[__main__.Foo.A]'
 else:
-    reveal_type(x1)  # N: Revealed type is '__main__.Foo'
+    reveal_type(x1)  # N: Revealed type is 'Literal[__main__.Foo.B]'
 
 x2: Foo
 if x2 == A_final:
-    reveal_type(x2)  # N: Revealed type is '__main__.Foo'
+    reveal_type(x2)  # N: Revealed type is 'Literal[__main__.Foo.A]'
 else:
-    reveal_type(x2)  # N: Revealed type is '__main__.Foo'
+    reveal_type(x2)  # N: Revealed type is 'Literal[__main__.Foo.B]'
 
 # But we let this narrow since there's an explicit literal in the RHS.
 x3: Foo
diff --git a/test-data/unit/check-optional.test b/test-data/unit/check-optional.test
index 74a27093..6771bb38 100644
--- a/test-data/unit/check-optional.test
+++ b/test-data/unit/check-optional.test
@@ -487,7 +487,7 @@ foo([f])  # E: List item 0 has incompatible type "Callable[[], int]"; expected "
 from typing import Optional
 x = ''  # type: Optional[str]
 if x == '<string>':
-    reveal_type(x)  # N: Revealed type is 'builtins.str'
+    reveal_type(x)  # N: Revealed type is 'Literal['<string>']'
 else:
     reveal_type(x)  # N: Revealed type is 'Union[builtins.str, None]'
 if x is '<string>':
@@ -500,7 +500,7 @@ else:
 from typing import Union
 x = ''  # type: Union[str, int, None]
 if x == '<string>':
-    reveal_type(x)  # N: Revealed type is 'Union[builtins.str, builtins.int]'
+    reveal_type(x)  # N: Revealed type is 'Literal['<string>']'
 else:
     reveal_type(x)  # N: Revealed type is 'Union[builtins.str, builtins.int, None]'
 if x is '<string>':
@@ -526,7 +526,7 @@ else:
 from typing import Optional
 x = ''  # type: Optional[str]
 if x == 0:
-    reveal_type(x)  # N: Revealed type is 'Union[builtins.str, None]'
+    reveal_type(x)
 else:
     reveal_type(x)  # N: Revealed type is 'Union[builtins.str, None]'
 if x is 0:
@@ -554,8 +554,8 @@ from typing import Optional
 x: Optional[int]
 y: Optional[int]
 if x == y == 1:
-    reveal_type(x)  # N: Revealed type is 'builtins.int'
-    reveal_type(y)  # N: Revealed type is 'builtins.int'
+    reveal_type(x)  # N: Revealed type is 'Literal[1]'
+    reveal_type(y)  # N: Revealed type is 'Literal[1]'
 else:
     reveal_type(x)  # N: Revealed type is 'Union[builtins.int, None]'
     reveal_type(y)  # N: Revealed type is 'Union[builtins.int, None]'

ref(bot): use exhaustiveness checking via NoReturn type

ref(bot): use exhaustiveness checking via NoReturn type (#614)

Adds docs about exhaustive literal and enum checks (#10860)

This appears to work in the latest version of mypy, so I think this could be closed.

Is the lack of IntEnum narrowing/exhaustiveness in mypy well-defined behavior? I ask because Pyright/VS Code are able to narrow the signal.Handlers IntEnum, for example, based on integer equality comparisons:

i: signal.Handlers = signal.Handlers.SIG_IGN
if i == signal.Handlers.SIG_IGN:
  reveal_type(i)  # Literal[Handlers.SIG_IGN] in pyright, Handlers in mypy
elif i == signal.Handlers.SIG_DFL:
  reveal_type(i)  # Literal[Handlers.SIG_DFL] in pyright, Handlers in mypy
else:
  assert_never(i)  # fails in mypy

Interestingly I can compare the integers with is (which I think would run because a small range of cpython integers are backed by a single location in memory per integer IIRC?) and get mypy to pass, but generally this should be unsafe at runtime:

if i is signal.Handlers.SIG_IGN:
  reveal_type(i)
elif i is signal.Handlers.SIG_DFL:
  reveal_type(i)
else:
  assert_never(i)

Comparing enum values with is is safe if the value is actually an enum.

Cool, thanks Jelle! So this seems working as intended, then.

Everything in this issue works, and in #11521 I changed mypy to make == narrow enums in addition to is

Navigation Menu

Repository navigation

Exhaustive Matching on Enums / Literals #6366

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

	# See comments in testNarrowingEqualityRequiresExplicitStrLiteral and
	# testNarrowingEqualityFlipFlop for more on why we can't narrow here.
	x1: Foo
	if x1 == Foo.A:
	reveal_type(x1) # N: Revealed type is '__main__.Foo'
	else:
	reveal_type(x1) # N: Revealed type is '__main__.Foo'

	x2: Foo
	if x2 == A_final:
	reveal_type(x2) # N: Revealed type is '__main__.Foo'
	else:
	reveal_type(x2) # N: Revealed type is '__main__.Foo'

	# But we let this narrow since there's an explicit literal in the RHS.
	x3: Foo
	if x3 == A_literal:
	reveal_type(x3) # N: Revealed type is 'Literal[__main__.Foo.A]'
	else:
	reveal_type(x3) # N: Revealed type is 'Literal[__main__.Foo.B]'

	def test1(switch: FlipFlopEnum) -> None:
	# Naively, we might assume the 'assert' here would narrow the type to
	# Literal[State.A]. However, doing this ends up breaking a fair number of real-world
	# code (usually test cases) that looks similar to this function: e.g. checks
	# to make sure a field was mutated to some particular value.
	#
	# And since mypy can't really reason about state mutation, we take a conservative
	# approach and avoid narrowing anything here.

	assert switch.state == State.A
	reveal_type(switch.state) # N: Revealed type is '__main__.State'

	switch.mutate()

	assert switch.state == State.B
	reveal_type(switch.state) # N: Revealed type is '__main__.State'

	def test2(switch: FlipFlopEnum) -> None:
	# So strictly speaking, we ought to do the same thing with 'is' comparisons
	# for the same reasons as above. But in practice, not too many people seem to
	# know that doing 'some_enum is MyEnum.Value' is idiomatic. So in practice,
	# this is probably good enough for now.

	assert switch.state is State.A
	reveal_type(switch.state) # N: Revealed type is 'Literal[__main__.State.A]'

	switch.mutate()

	assert switch.state is State.B # E: Non-overlapping identity check (left operand type: "Literal[State.A]", right operand type: "Literal[State.B]")
	reveal_type(switch.state) # E: Statement is unreachable

Exhaustive Matching on Enums / Literals #6366

Description

Activity

ilevkivskyi commented on Feb 9, 2019

ilevkivskyi commented on Jul 6, 2019

davidshepherd7 commented on Dec 2, 2019

ezyang commented on Aug 5, 2020

ezyang commented on Aug 14, 2020

sbdchd commented on Dec 18, 2020

sbdchd commented on Jan 29, 2021

erictraut commented on Dec 31, 2021

bbatliner commented on Feb 17, 2022

bbatliner commented on Feb 17, 2022

JelleZijlstra commented on Feb 17, 2022

bbatliner commented on Feb 17, 2022

hauntsaninja commented on Apr 24, 2023

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions