Crash on forward references in pydantic and django plugins

Crash Report

Mypy was working fine on my django project until about a week ago, when I started getting this error. I'm not sure what caused it. My python code used to be in the root directory of my project and now I moved it to the backend/ directory, and the problem started around then, but I'm not sure whether that is related.

I am using pre-commit and have mypy set up in my .pre-commit-config.yaml. The error actually only happens when I run mypy as part of pre-commit. If I just run mypy . from the command line, it works fine.

Presently, whenever I run pre-commit, I get an INTERNAL ERROR (details below).

Traceback

/home/travis/Repos/bethpage/backend/bethpage/models.py:23: error: INTERNAL ERROR -- Please try using mypy master on GitHub:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 1.16.0+dev.23e2d0f8cbaa2f4874d8f6a6bee3756922c78407
Traceback (most recent call last):
  File "/home/travis/Repos/bethpage/.tox/py312/bin/mypy", line 8, in <module>
    sys.exit(console_entry())
  File "/home/travis/Repos/bethpage/.tox/py312/lib/python3.12/site-packages/mypy/__main__.py", line 15, in console_entry
    main()
  File "/home/travis/Repos/bethpage/.tox/py312/lib/python3.12/site-packages/mypy/main.py", line 119, in main
    res, messages, blockers = run_build(sources, options, fscache, t0, stdout, stderr)
  File "/home/travis/Repos/bethpage/.tox/py312/lib/python3.12/site-packages/mypy/main.py", line 203, in run_build
    res = build.build(sources, options, None, flush_errors, fscache, stdout, stderr)
  File "/home/travis/Repos/bethpage/.tox/py312/lib/python3.12/site-packages/mypy/build.py", line 191, in build
    result = _build(
  File "/home/travis/Repos/bethpage/.tox/py312/lib/python3.12/site-packages/mypy/build.py", line 267, in _build
    graph = dispatch(sources, manager, stdout)
  File "/home/travis/Repos/bethpage/.tox/py312/lib/python3.12/site-packages/mypy/build.py", line 2937, in dispatch
    process_graph(graph, manager)
  File "/home/travis/Repos/bethpage/.tox/py312/lib/python3.12/site-packages/mypy/build.py", line 3335, in process_graph
    process_stale_scc(graph, scc, manager)
  File "/home/travis/Repos/bethpage/.tox/py312/lib/python3.12/site-packages/mypy/build.py", line 3430, in process_stale_scc
    mypy.semanal_main.semantic_analysis_for_scc(graph, scc, manager.errors)
  File "/home/travis/Repos/bethpage/.tox/py312/lib/python3.12/site-packages/mypy/semanal_main.py", line 94, in semantic_analysis_for_scc
    process_functions(graph, scc, patches)
  File "/home/travis/Repos/bethpage/.tox/py312/lib/python3.12/site-packages/mypy/semanal_main.py", line 252, in process_functions
    process_top_level_function(
  File "/home/travis/Repos/bethpage/.tox/py312/lib/python3.12/site-packages/mypy/semanal_main.py", line 291, in process_top_level_function
    deferred, incomplete, progress = semantic_analyze_target(
  File "/home/travis/Repos/bethpage/.tox/py312/lib/python3.12/site-packages/mypy/semanal_main.py", line 351, in semantic_analyze_target
    analyzer.refresh_partial(
  File "/home/travis/Repos/bethpage/.tox/py312/lib/python3.12/site-packages/mypy/semanal.py", line 655, in refresh_partial
    self.accept(node)
  File "/home/travis/Repos/bethpage/.tox/py312/lib/python3.12/site-packages/mypy/semanal.py", line 7254, in accept
    node.accept(self)
  File "/home/travis/Repos/bethpage/.tox/py312/lib/python3.12/site-packages/mypy/nodes.py", line 811, in accept
    return visitor.visit_func_def(self)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/travis/Repos/bethpage/.tox/py312/lib/python3.12/site-packages/mypy/semanal.py", line 913, in visit_func_def
    self.analyze_func_def(defn)
  File "/home/travis/Repos/bethpage/.tox/py312/lib/python3.12/site-packages/mypy/semanal.py", line 958, in analyze_func_def
    self.defer(defn)
  File "/home/travis/Repos/bethpage/.tox/py312/lib/python3.12/site-packages/mypy/semanal.py", line 6900, in defer
    assert not self.final_iteration, "Must not defer during final iteration"
           ^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: Must not defer during final iteration
/home/travis/Repos/bethpage/backend/bethpage/models.py:23: : note: use --pdb to drop into pdb

To Reproduce

This crash happens any time I change any file in my backend/ directory and then run git add -A; pre-commit.

It does not happen if I just run cd backend; mypy .

If I use the --no-incremental flag or delete the cache directory, the problem still persists.

If I open up backend/bethpage/models.py and add a line that just says "pass" somewhere above line 23 (the line cited in the error), then run git add -A; pre-commit, it actually works fine and completes without errors. The next time I need to commit something, I'll get the error again, but I can just remove that "pass" line and once again, the error goes away.

Here is the beginning of that /backend/bethpage/models.py file:

import uuid
from typing import (
    Any,
    ClassVar,
    Dict,
    Generic,
    Iterable,
    List,
    Mapping,
    Optional,
    Self,
    Tuple,
    TypeVar,
)

from django.db import models, transaction
from django.db.models.fields.reverse_related import ForeignObjectRel
from django.utils import timezone

T = TypeVar("T", bound="BaseModel")


class BaseQuerySet(models.QuerySet[T], Generic[T]):

    def bulk_create(self, objs: Iterable[T], *args: Any, **kwargs: Any) -> List[T]:
        for obj in objs:
            obj.full_clean()
        return super().bulk_create(objs, *args, **kwargs)
...etc...

Your Environment

Mypy version used: I have tried versions of mypy from 1.0.0 to 1.14.1 and have also tried using the master branch on github, always with the same error.
Python version used: 3.12
Operating system: Arch Linux

.pre-commit-config.yaml

repos:
  - repo: local
    hooks:
      - id: mypy
        name: mypy
        entry: mypy
        language: system
        types: [python]
        args:
          - --config-file=backend/mypy.ini
          - --show-traceback
...etc...

mypy.ini

[mypy]
python_version = 3.12
mypy_path = backend
plugins =
  mypy_django_plugin.main,
  mypy_drf_plugin.main
strict = True
show_error_codes = True
disallow_untyped_calls = True
disallow_untyped_defs = True
disallow_incomplete_defs = True
disallow_any_unimported = True
check_untyped_defs = True
no_implicit_optional = True
warn_unused_ignores = True
warn_return_any = True
warn_unreachable = True

[mypy.plugins.django-stubs]
django_settings_module = bethpage.settings

What happens if you disable the plugins?

And does pre-commit run -a fail?

pre-commit run -a does not work. I get the same error I have in the description above.

Disabling the plugins does work, and mypy no longer returns an error. Specifically, disabling the mypy_django_plugin.main causes it to work. I can keep the mypy_drf_plugin.main plugin and there are no issues.

I updated django-stubs and django-stubs-ext to the latest version but the problem persists. Of course, I do kind of need the django-stubs plugin for my project to work with mypy.

@sobolevn

If you run mypy <list of files> --config-file=backend/mypy.ini --show-traceback that should also crash, I think. Could you check?

cc @sobolevn

Actually no. It still works when running mypy directly.

(py312) [travis@bananas bethpage]$ git status
On branch main
Your branch is up to date with 'origin/main'.

Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
        modified:   backend/bethpage/settings.py

(py312) [travis@bananas bethpage]$ git add -A
(py312) [travis@bananas bethpage]$ pre-commit
mypy.....................................................................Failed
- hook id: mypy
- exit code: 2

/home/travis/Repos/bethpage/backend/bethpage/models.py:24: error: INTERNAL ERROR -- Please try using mypy master on GitHub:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
If this issue continues with mypy master, please report a bug at https://github.com/python/mypy/issues
version: 1.14.1
/home/travis/Repos/bethpage/backend/bethpage/models.py:24: : note: please use --show-traceback to print a traceback when reporting a bug

black....................................................................Passed
isort....................................................................Passed
flake8...................................................................Passed
codespell................................................................Passed
ESLint...............................................(no files to check)Skipped
Prettier.............................................(no files to check)Skipped
trim trailing whitespace.................................................Passed
fix end of files.........................................................Passed

At this point I made another change in backend/bethpage/settings.py to make sure to bypass any cache created from the prior step, then continued:

(py312) [travis@bananas bethpage]$ git add -A
(py312) [travis@bananas bethpage]$ mypy backend/ --config-file=backend/mypy.ini --show-traceback
Success: no issues found in 80 source files

Sorry just to be explicit, I mean passing mypy a list of .py files. IIRC the way pre-commit works is it does that, so I assume that's the cause of the crash. Such as, find -name "*.py" | xargs mypy --config-file=backend/mypy.ini --show-traceback. (maybe I'm wrong about how pre-commit works)

Does it happen without django-stubs enabled?

@A5rocks

@A5rocks The issue still does not happen when passing specific files to mypy. The problem does happen with pre-commit, but then I run directly with the list of files and get:

(py312) [travis@bananas bethpage]$ find backend/ -name "*.py" | xargs mypy --config-file=backend/mypy.ini --show-traceback
Success: no issues found in 80 source files

So it is still limited only to pre-commit as far as I can tell.

@sobolevn When I remove django-stubs, the error does not happen, even with pre-commit. It does seem to be tied specifically to pre-commit and specifically to django-stubs. For clarity, earlier post where I checked that: #18579 (comment)

This is indeed likely specific to django-stubs, but has nothing to do with pre-commit.

Full script:

cd "$(mktemp -d)"
uv venv --python 3.12 && . ./.venv/bin/activate
uv pip install mypy django django-stubs

mkdir -p backend/bethpage
touch backend/bethpage/__init__.py

cat <<EOF > backend/bethpage/models.py
from typing import Generic, TypeVar

from django.db import models

T = TypeVar("T", bound="BaseModel")

class BaseQuerySet(models.QuerySet[T], Generic[T]): pass
EOF

cat <<EOF > backend/bethpage/settings.py
from pathlib import Path

BASE_DIR = Path(__file__).resolve().parent.parent
INSTALLED_APPS = []
EOF

cat <<EOF > backend/mypy.ini
[mypy]
python_version = 3.12
mypy_path = backend
plugins =
  mypy_django_plugin.main
strict = True

[mypy.plugins.django-stubs]
django_settings_module = bethpage.settings
EOF

mypy backend --config-file backend/mypy.ini --show-traceback

Out:

backend/bethpage/settings.py:4: error: Need type annotation for "INSTALLED_APPS" (hint: "INSTALLED_APPS: list[<type>] = ...")  [var-annotated]
backend/bethpage/models.py:5: error: Name "BaseModel" is not defined  [name-defined]
backend/bethpage/models.py:7: error: INTERNAL ERROR -- Please try using mypy master on GitHub:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 1.14.1
Traceback (most recent call last):
  File "mypy/semanal.py", line 7286, in accept
  File "mypy/nodes.py", line 827, in accept
  File "mypy/semanal.py", line 907, in visit_func_def
  File "mypy/semanal.py", line 952, in analyze_func_def
  File "mypy/semanal.py", line 6940, in defer
AssertionError: Must not defer during final iteration
backend/bethpage/models.py:7: : note: use --pdb to drop into pdb

The core of the problem is in unresolved quoted name in typevar bound: replacing BaseModel with something defined or removing the surrounding quotes prevents the crash.

PRs to django-stubs are welcome :)

Related, possibly a duplicate of the same underlying cause (similar issue, and a workaround and the django plugin are mentioned in its discussion): #15927

I was looking at a bunch of issues today, and it looks like the same traceback affects both django and pydantic plugins. I am going to close some likely duplicates. Please use this issue to track the investigations and progress.

X-posting a repro for the pydantic plugin

from typing import *
from pydantic import BaseModel

AnyBaseFoo = TypeVar("AnyBaseFoo", bound="BaseFoo")

class BaseFoo:
    pass

class Foo(BaseModel, Generic[AnyBaseFoo]):
    pass

For both plugins the error points to a class definition line and happens in analyze_func_def. My working hypothesis is that both plugins generate some methods while some types still contain PlaceholderType (due to forward references not yet processed), and don't update the auto-generated methods on subsequent passes.

Most likely this can be fixed from within plugins. But I am thinking maybe we should find some way to communicate to plugin authors that auto-generated methods should be updated if they have any placeholders in signature. Possibly we can detect methods marked as generated and if they have a placeholder add a detailed error (not crash) linking to some URL explaining mypy multi-pass architecture, and erase the placeholder to prevent crashes in other places.

@sobolevn

Few updates here:

It looks like the django-stubs repro above Crash on forward references in pydantic and django plugins #18579 (comment) doesn't crash anymore. Probably because they use has_placeholder() and defer if needed. One thing I noticed is that they only call has_placeholder() on type variables, a more prudent approach would be to call it on all types that would appear in the generated signature. cc @sobolevn FYI
The pydantic example above still crashes, I will make a PR to pydantic to fix it later today or tomorrow. Note they vendor a (slightly modified) copy of add_method() to generate methods in their plugin.
In general, this doesn't look like something that we can fix/prevent on the mypy side. I am going to post a PSA in Announcement issue for plugin API changes #6617 explaining best practices for plugin authors w.r.t. multi-pass semantic analysis.

It looks like all known concrete repros are now fixed in the corresponding plugins, and there really not much we can do in general. I propose to close this. Feel free to re-open if there are other scenarios where this happens.

Navigation Menu

Crash on forward references in pydantic and django plugins #18579

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Crash on forward references in pydantic and django plugins #18579

Description

Activity

A5rocks commented on Feb 1, 2025

TravisHardman commented on Feb 3, 2025

A5rocks commented on Feb 3, 2025

TravisHardman commented on Feb 3, 2025

A5rocks commented on Feb 3, 2025

sobolevn commented on Feb 3, 2025

TravisHardman commented on Feb 3, 2025

sterliakov commented on Feb 4, 2025

sterliakov commented on Feb 4, 2025

sobolevn commented on Feb 4, 2025

wyattscarpenter commented on Sep 21, 2025

ilevkivskyi commented on Jan 29, 2026

ilevkivskyi commented on Jan 29, 2026

ilevkivskyi commented on Jan 29, 2026

ilevkivskyi commented on Oct 19, 2026

ilevkivskyi commented on Oct 19, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions