Skip to content

1.7 Crash on recursive type #16473

Closed
#16483
Closed
1.7 Crash on recursive type#16473
#16483
Labels
crashtopic-overlapOverlapping equality checktopic-recursive-types

Description

@NMertsch
NMertsch
opened on Nov 13, 2023

Crash Report

I have a recursive type NestedDict = dict[str, "NestedDict"]. Using it in a separate module causes a Segmentation fault with mypy --strict.
This worked with 1.6.1, but failed with 1.7 and also with master.

Traceback
Using the non-compiled 1.7 and master version results in this:

Traceback (most recent call last):
  File "/home/mertschn/.conda/envs/mypy-repro/bin/mypy", line 8, in <module>
    sys.exit(console_entry())
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/__main__.py", line 15, in console_entry
    main()
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/main.py", line 100, in main
    res, messages, blockers = run_build(sources, options, fscache, t0, stdout, stderr)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/main.py", line 182, in run_build
    res = build.build(sources, options, None, flush_errors, fscache, stdout, stderr)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/build.py", line 191, in build
    result = _build(
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/build.py", line 265, in _build
    graph = dispatch(sources, manager, stdout)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/build.py", line 2943, in dispatch
    process_graph(graph, manager)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/build.py", line 3341, in process_graph
    process_stale_scc(graph, scc, manager)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/build.py", line 3442, in process_stale_scc
    graph[id].type_check_first_pass()
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/build.py", line 2311, in type_check_first_pass
    self.type_checker().check_first_pass()
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/checker.py", line 481, in check_first_pass
    self.accept(d)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/checker.py", line 591, in accept
    stmt.accept(self)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/nodes.py", line 787, in accept
    return visitor.visit_func_def(self)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/checker.py", line 1000, in visit_func_def
    self._visit_func_def(defn)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/checker.py", line 1004, in _visit_func_def
    self.check_func_item(defn, name=defn.name)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/checker.py", line 1077, in check_func_item
    self.check_func_def(defn, typ, name, allow_empty)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/checker.py", line 1293, in check_func_def
    self.accept(item.body)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/checker.py", line 591, in accept
    stmt.accept(self)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/nodes.py", line 1223, in accept
    return visitor.visit_block(self)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/checker.py", line 2770, in visit_block
    self.accept(s)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/checker.py", line 591, in accept
    stmt.accept(self)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/nodes.py", line 1436, in accept
    return visitor.visit_assert_stmt(self)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/checker.py", line 4565, in visit_assert_stmt
    self.expr_checker.accept(s.expr)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/checkexpr.py", line 5716, in accept
    typ = node.accept(self)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/nodes.py", line 2076, in accept
    return visitor.visit_comparison_expr(self)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/checkexpr.py", line 3573, in visit_comparison_expr
    if self.dangerous_comparison(left_type, right_type):
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/checkexpr.py", line 3705, in dangerous_comparison
    ) or self.dangerous_comparison(left.args[1], right.args[1])
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/checkexpr.py", line 3705, in dangerous_comparison
    ) or self.dangerous_comparison(left.args[1], right.args[1])
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/checkexpr.py", line 3705, in dangerous_comparison
    ) or self.dangerous_comparison(left.args[1], right.args[1])
  [Previous line repeated 16340 more times]
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/checkexpr.py", line 3701, in dangerous_comparison
    left = map_instance_to_supertype(left, abstract_map)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/maptype.py", line 42, in map_instance_to_supertype
    return map_instance_to_supertypes(instance, superclass)[0]
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/maptype.py", line 54, in map_instance_to_supertypes
    a.extend(map_instance_to_direct_supertypes(t, sup))
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/maptype.py", line 96, in map_instance_to_direct_supertypes
    t = expand_type_by_instance(b, instance)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/expandtype.py", line 119, in expand_type_by_instance
    return expand_type(typ, variables)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/expandtype.py", line 71, in expand_type
    return typ.accept(ExpandTypeVisitor(env))
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/types.py", line 1435, in accept
    return visitor.visit_instance(self)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/expandtype.py", line 214, in visit_instance
    args = self.expand_types_with_unpack(list(t.args))
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/expandtype.py", line 396, in expand_types_with_unpack
    items.append(item.accept(self))
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/types.py", line 635, in accept
    return visitor.visit_type_var(self)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/expandtype.py", line 234, in visit_type_var
    return repl.copy_modified(last_known_value=None)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/types.py", line 1488, in copy_modified
    new = Instance(
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/types.py", line 1373, in __init__
    super().__init__(line, column)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/types.py", line 234, in __init__
    super().__init__(line, column)
RecursionError: maximum recursion depth exceeded while calling a Python object

To Reproduce

Code and instructions in this repo: https://github.com/NMertsch/mypy-repro.
tl;dr: I define a recursive type NestedDict = dict[str, "NestedDict"]. Using it in the same file is fine, but in my tests, it causes a crash.

Your Environment

  • Mypy version used: Installed from master, commit efa5dcb (reproducible with 1.7, works with 1.6.1)
  • Mypy command-line flags: --strict
  • Mypy configuration options from mypy.ini (and other config files): None
  • Python version used: 3.10.13
  • Operating system and version: Windows 10, Ubuntu 20.04.5

Activity

  1. NMertsch
    added
    crash
    on Nov 13, 2023
  2. AlexWaygood
    added
    topic-recursive-types
    topic-overlapOverlapping equality check
    on Nov 13, 2023

  3. AlexWaygood commented on Nov 13, 2023

    @AlexWaygood
    AlexWaygood
    on Nov 13, 2023
    Member

    Simpler repro: rather than running mypy --strict . (which enables many mypy options) on https://github.com/NMertsch/mypy-repro, just run mypy --strict-equality . (which enables only one mypy option) on the repo.

  4. AlexWaygood commented on Nov 13, 2023

    @AlexWaygood
    AlexWaygood
    on Nov 13, 2023
    Member
    a3af87bf252f0ed0c6e0f977ad4079418b37a70f is the first bad commit
commit a3af87bf252f0ed0c6e0f977ad4079418b37a70f
Author: Ivan Levkivskyi <levkivskyi@gmail.com>
Date:   Sat Oct 21 16:41:10 2023 +0100

    Narrow tuple types using len() (#16237)

    Fixes #1178
    Supersedes #10367

    This is includes implementation for fixed length tuples, homogeneous
    tuples, and variadic tuples (and combinations of those). Generally
    implementation is straightforward. Some notes:
    * Unfortunately, it is necessary to add a new attribute `min_len` to
    `TypeVarTupleType`, which is probably fine, as it doesn't have that many
    attributes so far.
    * Supporting more general use cases (like `>` comparisons for variadic
    tuples) can cause quick proliferation of unions. I added two mechanisms
    to counteract this: not applying the narrowing if the integer literal in
    comparison is itself large, and collapsing unions of tuples into a
    single tuple (if possible) after we are done with the narrowing. This
    looks a bit arbitrary, but I think it is important to have.
    * Main missing feature here is probably not inferring type information
    from indirect comparisons like `len(x) > foo() > 1`. Supporting this
    kind of things in full generality is cumbersome, and we may add cases
    that turn out to be important later.
    * Note I am quite careful with indexing "inside" a `TypeVarTuple`, it is
    not really needed now, but I wanted to make everything future proof, so
    that it will be easy to add support for upper bounds for
    `TypeVarTuple`s, like `Nums = TypeVarTuple("Nums", bound=tuple[float,
    ...])`.
    * I also fix couple existing inconsistencies with `Any` handling in type
    narrowing. It looks like they stem from the old incorrect logic that
    meet of `Any` and `X` should be `X`, while in fact it should be `Any`.
    These fixes are not strictly necessary, but otherwise there may be new
    false positives, because I introduce a bunch of additional type
    narrowing scenarios here.

    cc @hatal175, thanks for the test cases, and for your nice first attempt
    to implement this!
    Co-authored-by: Tal Hayon <talhayon1@gmail.com>

 mypy/binder.py                        |  83 +++++
 mypy/checker.py                       | 359 ++++++++++++++++++++-
 mypy/checkexpr.py                     |  53 +++-
 mypy/meet.py                          |   6 +-
 mypy/operators.py                     |  23 ++
 mypy/options.py                       |   3 +-
 mypy/subtypes.py                      |   2 +-
 mypy/test/testcheck.py                |   2 +-
 mypy/typeops.py                       |   2 +-
 mypy/types.py                         |  27 +-
 mypy_self_check.ini                   |   1 +
 test-data/unit/check-expressions.test |  13 +
 test-data/unit/check-namedtuple.test  |   2 +-
 test-data/unit/check-narrowing.test   | 576 ++++++++++++++++++++++++++++++++++
 test-data/unit/fixtures/len.pyi       |  39 +++
 test-data/unit/lib-stub/typing.pyi    |   1 +
 16 files changed, 1154 insertions(+), 38 deletions(-)
 create mode 100644 test-data/unit/fixtures/len.pyi

    Bisects to a3af87b (cc. @ilevkivskyi)

  5. ilevkivskyi
    closed this as completedin #16483on Nov 15, 2023
  6. ilevkivskyi
    added a commit that references this issue on Nov 15, 2023

    Fix crash on strict-equality with recursive types (#16483)

    0699dde
  7. JukkaL
    added a commit that references this issue on Nov 22, 2023

    Fix crash on strict-equality with recursive types (#16483)

    661adb7
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    crashtopic-overlapOverlapping equality checktopic-recursive-types

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions