False narrowing with repeated assignment inside if resulting in bad false negative

Bug Report

In a function, mypy fails to detect an incompatible assignment in a function, if that is after a __get_item__ access inside a branch. Weirdly, mypy correctly deduces the types, as shown with reveal_type but does not complain about the incorrect assignment anyways.

To Reproduce

I would have liked to give a playground link, but the MWE involves a pandas.DataFrame which seems not to work on the playground.

import pandas as pd

# fails as expected
some_condition: bool = "123".isnumeric()
list_of_dicts = [{"some": 1, "columns": 2, "will": 3, "go": 4, "here": 5}]
if some_condition > 0:
    output_df = pd.DataFrame(list_of_dicts)
    output_df = output_df[["some", "columns", "here"]]
else:
    output_df = None  # fails here


def fails_as_expected_1(some_condition: bool) -> pd.DataFrame:
    list_of_dicts = [{"some": 1, "columns": 2, "will": 3, "go": 4, "here": 5}]
    output_df = pd.DataFrame(list_of_dicts)
    output_df = output_df[["some", "columns", "here"]]
    output_df = None  # fails here
    other_df: pd.DataFrame = output_df
    return other_df


def fails_as_expected_2(some_condition: bool) -> pd.DataFrame:
    list_of_dicts = [{"some": 1, "columns": 2, "will": 3, "go": 4, "here": 5}]
    if some_condition > 0:
        output_df = pd.DataFrame(list_of_dicts)
    else:
        output_df = None
    other_df: pd.DataFrame = output_df  # fails here
    return other_df


def does_not_fail_but_should(some_condition: bool) -> pd.DataFrame:
    list_of_dicts = [{"some": 1, "columns": 2, "will": 3, "go": 4, "here": 5}]
    if some_condition > 0:
        output_df = pd.DataFrame(list_of_dicts)
        reveal_type(output_df)
        output_df = output_df[["some", "columns", "here"]]
        reveal_type(output_df)
    else:
        output_df = None
    other_df: pd.DataFrame = output_df
    return other_df

Expected Behavior
Each block and function should contain a mypy error regarding an incompatible assignment.

Actual Behavior

For the last function, no such error is produced:

mwe2.py:10: error: Incompatible types in assignment (expression has type "None", variable has type "DataFrame")  [assignment]
mwe2.py:17: error: Incompatible types in assignment (expression has type "None", variable has type "DataFrame")  [assignment]
mwe2.py:28: error: Incompatible types in assignment (expression has type "DataFrame | None", variable has type "DataFrame")  [assignment]
mwe2.py:36: note: Revealed type is "pandas.core.frame.DataFrame"
mwe2.py:38: note: Revealed type is "pandas.core.frame.DataFrame"
Found 3 errors in 1 file (checked 1 source file)

Your Environment

Mypy version used: mypy 1.14.1 (compiled: yes)
Mypy command-line flags: only the filename
Mypy configuration options from pyproject.toml:

[tool.mypy]
warn_unreachable = true
enable_error_code = [
    "possibly-undefined"
]
strict = true

Python version used: Python 3.12.8 from Debian Testing

Thanks! Pretty bad bug. Looks like it's just some issue if there's a double assignment (and specifically with None and without an explicit annotation, so likely related to partial types):

class A: ...

def does_not_fail_but_should(some_condition: bool) -> A:
    if some_condition > 0:
        var = A()
        reveal_type(var)
        # Commenting out this assignment makes mypy correctly error
        var = A()
        reveal_type(var)
    else:
        var = None

    reveal_type(var)
    other: A = var
    return other

I can reproduce the minified example with the same toolchain as above.

@hauntsaninja

@hauntsaninja While trying to reproduce this I encountered an issue with mypy where it fails to correctly identify the type of variable and ultimately raises an error at an undesired place

def gives_wrong_output(some_condition: bool) -> None:
    if some_condition:
        a = 3
    else:
        a = 's' # fails here
    reveal_type(a)
    b: int = a # should have failed here

main.py:5: error: Incompatible types in assignment (expression has type "str", variable has type "int")  [assignment]
main.py:6: note: Revealed type is "builtins.int"

@Prabhat-Thapa45

I've tried reproducing the original example and in the latest nightly (301c3b6) and it no longer reproduces. In the function does_not_fail_but_should there is a true positive error about assigning a value of type DataFrame | None to a variable expecting DataFrame.

original.py:43: error: Incompatible types in assignment (expression has type "DataFrame | None", variable has type "DataFrame")

The problem in the simplified example by @Prabhat-Thapa45 is still reproducible, though (mypy doesn't take into assignment a reachable assignment in the false branch).

However, with the experimental options --allow-redefinition-new --local-partial-types the false negative in the simplified example disappears and the behavior in the first example becomes better. The reassignment of output_df on line 12 becomes possible and in fails_as_expected_1 mypy reports assignment of a None value to other_df on line 20 instead of the reassigment of output_df on line 19.

some_condition: bool = "123".isnumeric()
list_of_dicts = [{"some": 1, "columns": 2, "will": 3, "go": 4, "here": 5}]
if some_condition > 0:
    output_df = pd.DataFrame(list_of_dicts)
    output_df = output_df[["some", "columns", "here"]]
else:
    output_df = None  # warning (false positive?) without --allow-redefinition-new --local-partial-type


def fails_as_expected_1(some_condition: bool) -> pd.DataFrame:
    list_of_dicts = [{"some": 1, "columns": 2, "will": 3, "go": 4, "here": 5}]
    output_df = pd.DataFrame(list_of_dicts)
    output_df = output_df[["some", "columns", "here"]]
    output_df = None  # warning here (false positive?) without --allow-redefinition-new --local-partial-type
    other_df: pd.DataFrame = output_df  # warning here with --allow-redefinition-new
    return other_df

Looks like this got fixed by #18538

Navigation Menu

Repository navigation

False narrowing with repeated assignment inside if resulting in bad false negative #18492

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

False narrowing with repeated assignment inside if resulting in bad false negative #18492

Description

Activity

hauntsaninja commented on Jan 20, 2025

MaxG87 commented on Jan 21, 2025

Prabhat-Thapa45 commented on Jan 31, 2025

east825 commented on May 19, 2025

hauntsaninja commented on May 19, 2025

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions