Repository navigation
internal server error - dmypy daemon crashed #18396
Description
Bug Report
./src/intellibricks/models/supervised/engines.py: error: INTERNAL ERROR -- Please try using mypy master on GitHub:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 1.14.1
Daemon crashed!
Traceback (most recent call last):
File "mypy/dmypy_server.py", line 236, in serve
File "mypy/dmypy_server.py", line 285, in run_command
File "mypy/dmypy_server.py", line 353, in cmd_run
File "mypy/dmypy_server.py", line 432, in check
File "mypy/dmypy_server.py", line 700, in fine_grained_increment_follow_imports
File "mypy/server/update.py", line 285, in update
File "mypy/server/update.py", line 881, in propagate_changes_using_dependencies
File "mypy/server/update.py", line 1010, in reprocess_nodes
File "mypy/semanal_main.py", line 137, in semantic_analysis_for_targets
File "mypy/semanal_main.py", line 220, in process_top_levels
File "mypy/semanal_main.py", line 351, in semantic_analyze_target
File "mypy/semanal.py", line 646, in refresh_partial
File "mypy/semanal.py", line 657, in refresh_top_level
File "mypy/semanal.py", line 7288, in accept
File "mypy/errors.py", line 1288, in report_internal_error
File "mypy/semanal.py", line 7286, in accept
File "mypy/nodes.py", line 471, in accept
File "mypy/semanal.py", line 2958, in visit_import_from
File "mypy/semanal.py", line 3054, in report_missing_module_attribute
File "mypy/semanal.py", line 7267, in fail
File "mypy/errors.py", line 463, in report
File "mypy/errors.py", line 529, in add_error_info
File "mypy/errors.py", line 466, in _add_error_info
AssertionError
[35] Error running mypy in /Users/arthurbrenno/Documents/Dev/Public/ctx/intellibricks: the mypy daemon crashed. This is probably a bug in mypy itself, see Output panel for details. The daemon will be restarted automatically.
To Reproduce
uv python install 3.13.1
uv python pin 3.13.1
uv venv
source .venv/bin/activate
uv add mypy --dev
dmypy startx: str = 2
y: int = "3"
# ERRORYour Environment
- Mypy version used: 1.14.1
- Mypy command-line flags: uv run dmypy start
- Mypy configuration options from
mypy.ini(and other config files): - Python version used: 3.13.1
Activity
A5rocks commentedon Dec 31, 2024
Are you sure that's the right file? The traceback says that it crashed visiting an import from:
File "mypy/semanal.py", line 2958, in visit_import_from.hauntsaninja commentedon Dec 31, 2024
Yeah, the traceback suggests
./src/intellibricks/models/supervised/engines.pyarthurbrenno commentedon Dec 31, 2024
Hi. supervised/engines.py does not exist anymore. I deleted it. Maybe I clear mypy cache?
arthurbrenno commentedon Dec 31, 2024
I'm sure! Also, I'm using the mypy extension, from Matan Gover. That might not be your problem, right? I should tell the extension author. The error I told you happens automatically. I didn't run any mypy command. The extension is running mypy daemon in the background. I tested now in a file called
schema.py.Just replicated the error again now:
Document saved: /Users/arthurbrenno/Documents/Dev/Public/ctx/intellibricks/src/intellibricks/llms/schema.py
[62] Check folder: /Users/arthurbrenno/Documents/Dev/Public/ctx/intellibricks
[62] Received Python path from Python extension: /Users/arthurbrenno/Documents/Dev/Public/ctx/intellibricks/.venv/bin/python
[62] Running dmypy in folder /Users/arthurbrenno/Documents/Dev/Public/ctx/intellibricks
/Users/arthurbrenno/Documents/Dev/Public/ctx/intellibricks/.venv/bin/dmypy --status-file '/Users/arthurbrenno/Library/Application Support/Code/User/workspaceStorage/8a9c88162dc1465d3ab80eb67813bccb/matangover.mypy/dmypy-8472a25bb76665ac15378ffb435fc2ea45288337-1975.json' run --log-file '/Users/arthurbrenno/Library/Application Support/Code/User/workspaceStorage/8a9c88162dc1465d3ab80eb67813bccb/matangover.mypy/dmypy-8472a25bb76665ac15378ffb435fc2ea45288337.log' -- --python-executable /Users/arthurbrenno/Documents/Dev/Public/ctx/intellibricks/.venv/bin/python . --show-error-end --no-error-summary --no-pretty --no-color-output
[62] stderr:
Daemon crashed!
Traceback (most recent call last):
File "mypy/dmypy_server.py", line 236, in serve
File "mypy/dmypy_server.py", line 285, in run_command
File "mypy/dmypy_server.py", line 353, in cmd_run
File "mypy/dmypy_server.py", line 432, in check
File "mypy/dmypy_server.py", line 700, in fine_grained_increment_follow_imports
File "mypy/server/update.py", line 285, in update
File "mypy/server/update.py", line 881, in propagate_changes_using_dependencies
File "mypy/server/update.py", line 1025, in reprocess_nodes
File "mypy/checker.py", line 535, in check_second_pass
File "mypy/checker.py", line 540, in check_partial
File "mypy/checker.py", line 555, in check_top_level
File "mypy/nodes.py", line 1364, in accept
File "mypy/checker.py", line 3001, in visit_assignment_stmt
File "mypy/checker.py", line 3179, in check_assignment
File "mypy/checker.py", line 4379, in check_simple_assignment
File "mypy/checker.py", line 7007, in check_subtype
File "mypy/checker.py", line 7350, in fail
File "mypy/messages.py", line 289, in fail
File "mypy/messages.py", line 264, in report
File "mypy/errors.py", line 463, in report
File "mypy/errors.py", line 529, in add_error_info
File "mypy/errors.py", line 466, in _add_error_info
AssertionError
[62] Error running mypy in /Users/arthurbrenno/Documents/Dev/Public/ctx/intellibricks: the mypy daemon crashed. This is probably a bug in mypy itself, see Output panel for details. The daemon will be restarted automatically.
Idk where to look honestly
A5rocks commentedon Dec 31, 2024
The traceback is different now (and is plausible on the repro you gave), so that's good! I'm not sure why it's happening since I'm not familiar with mypyd :/
miky-rola commentedon Jan 2, 2025
Hi, I wanted to know whether this bug has been fixed.
A5rocks commentedon Jan 2, 2025
It is unlikely this bug has been fixed. You can always install mypy via
pip install git+https://github.com/python/mypyto check.arthurbrenno commentedon Jan 2, 2025
Just reproduced the error again, more details below:
sanmai-NL commentedon Jan 14, 2025
I have the same exception. It could be induced by ungraceful shutdown of the daemon.
gerner commentedon Jan 14, 2025
I ran into the same issue and posted my problem to #15486. @hauntsaninja suggested it's actually this bug. I did a bisect and determined that I don't get the crash in 1.13.0 but I do get the crash in 1.14.0
The behavior I see is that from a fresh
dmypy restartI can rundmypy runtwice successfully and then I get a crash with the following stack trace on the third run. This is 100% reproduceable on this code base, you'll have to introduce a syntax/type error of some sort. I try not to check those in :)https://github.com/gerner/stellarpunk/tree/87748840624727e521b7641bd947b17f71ca061d
hauntsaninja commentedon Jan 19, 2025
Hmm, I couldn't trivially repro on 8774884 with numpy 2.1. I introduced a type error in
src/stellarpunk/intel.py. Could you share what error you introduced and when, and what third party package versions you're using? (Alternatively if you have a few lines of bash that set up a new environment in which it repros, that would be amazing!)lovettchris commentedon Jan 29, 2025
I'm getting this more and more on Windows lately with this assertion stack trace:
gerner commentedon Feb 4, 2025
@hauntsaninja sorry for the delay! I just did a clean clone of my repo and indeed I can't reproduce the error in that clean clone.
The kind of type error I'm introducing doesn't seem to matter on my main clone. I'm trying, for example, something like this:
Notice the missing annotation for
**kwargsDoes mypy keep some state around that survives a
dmypy restart?11 remaining items
sterliakov commentedon Apr 9, 2025
Following the investigation by @svalentin (thanks!), a few more observations:
Crash reproduces on master
First two runs aren't equivalent (
Found 61 errors in 10 files (checked 90 source files)andFound 60 errors in 9 files (checked 90 source files), resp.). The error that gets lost isThis is likely irrelevant - with minimal deps (see below) both runs produce the same output.
Reverting #976d105 fixes the problem, producing 60 errors for all subsequent runs without crashing. The "offending" (and the only affected by
is_stdlib_fileaddition) module isxml.etree.The only needed dependency is
imageio(impliesnumpy).testscan be excluded fromfiles(srcis enough), andnumpyplugin can be disabled. If I excludetests, crash happens during the third run, otherwise - during the second run.It's slow as hell, but at least happens consistently (. I wasn't able to narrow the scope further. The following crashed 5/5 during the second run:
sterliakov commentedon Apr 9, 2025
Hm, I might be onto something - can someone more familiar with dmypy updates take a look at my observations?
fine_grained_increment_follow_importsbuildsseenset. If my understanding is correct, any module should depend on its parent, so the finalseenshould contain a ladder from root to every reached submodule. It... isn't in some cases, resulting innumpy.testing._privateandxml.etreeconsidered deleted and processed as such.With the following patch:
I get the following log:
Long scrolling!
Here's the list from "xxx" unwrapped and sorted:
Full list
PIL
PIL.ExifTags
PIL.GifImagePlugin
PIL.GimpGradientFile
PIL.GimpPaletteFile
PIL.Image
PIL.ImageChops
PIL.ImageCms
PIL.ImageColor
PIL.ImageFile
PIL.ImageFilter
PIL.ImageMath
PIL.ImageMode
PIL.ImageOps
PIL.ImagePalette
PIL.ImageQt
PIL.ImageSequence
PIL.PaletteFile
PIL.TiffImagePlugin
PIL.TiffTags
PIL._binary
PIL._deprecate
PIL._imaging
PIL._imagingcms
PIL._imagingmath
PIL._typing
PIL._util
PIL._version
future
_ast
_asyncio
_bisect
_blake2
_bz2
_codecs
_collections_abc
_compression
_contextvars
_ctypes
_curses
_decimal
_frozen_importlib
_frozen_importlib_external
_hashlib
_heapq
_io
_lsprof
_operator
_pickle
_queue
_sitebuiltins
_socket
_ssl
_struct
_thread
_typeshed
_typeshed.importlib
_warnings
_weakref
_weakrefset
abc
argparse
array
ast
asyncio
asyncio.base_events
asyncio.coroutines
asyncio.events
asyncio.exceptions
asyncio.futures
asyncio.locks
asyncio.mixins
asyncio.protocols
asyncio.queues
asyncio.runners
asyncio.selector_events
asyncio.streams
asyncio.subprocess
asyncio.taskgroups
asyncio.tasks
asyncio.threads
asyncio.timeouts
asyncio.transports
asyncio.unix_events
atexit
bdb
bisect
builtins
bz2
cProfile
cmd
codecs
collections
collections.abc
concurrent
concurrent.futures
concurrent.futures._base
concurrent.futures.process
concurrent.futures.thread
contextlib
contextvars
copy
copyreg
ctypes
ctypes._endian
curses
curses.ascii
curses.textpad
dataclasses
datetime
decimal
dis
email
email._policybase
email.charset
email.contentmanager
email.errors
email.header
email.message
email.policy
enum
fcntl
fractions
functools
gc
genericpath
glob
gzip
hashlib
heapq
imageio
imageio.config
imageio.config.extensions
imageio.config.plugins
imageio.core
imageio.core.fetching
imageio.core.findlib
imageio.core.format
imageio.core.imopen
imageio.core.legacy_plugin_wrapper
imageio.core.request
imageio.core.util
imageio.core.v3_plugin_api
imageio.plugins
imageio.plugins.opencv
imageio.plugins.pillow
imageio.plugins.pyav
imageio.plugins.rawpy
imageio.plugins.tifffile_v3
imageio.typing
imageio.v2
imageio.v3
importlib
importlib._abc
importlib._bootstrap
importlib._bootstrap_external
importlib.abc
importlib.machinery
importlib.metadata
importlib.metadata._meta
importlib.readers
importlib.resources
importlib.resources._common
importlib.resources._functional
importlib.resources.abc
inspect
io
itertools
json
json.decoder
json.encoder
logging
math
mmap
multiprocessing
multiprocessing.connection
multiprocessing.context
multiprocessing.managers
multiprocessing.pool
multiprocessing.popen_fork
multiprocessing.popen_forkserver
multiprocessing.popen_spawn_posix
multiprocessing.popen_spawn_win32
multiprocessing.process
multiprocessing.queues
multiprocessing.reduction
multiprocessing.shared_memory
multiprocessing.sharedctypes
multiprocessing.spawn
multiprocessing.synchronize
multiprocessing.util
numbers
numpy
numpy.config
numpy._array_api_info
numpy._core
numpy._core._asarray
numpy._core._internal
numpy._core._type_aliases
numpy._core._ufunc_config
numpy._core.arrayprint
numpy._core.defchararray
numpy._core.einsumfunc
numpy._core.fromnumeric
numpy._core.function_base
numpy._core.multiarray
numpy._core.numeric
numpy._core.numerictypes
numpy._core.records
numpy._core.shape_base
numpy._core.strings
numpy._expired_attrs_2_0
numpy._globals
numpy._pytesttester
numpy._typing
numpy._typing._add_docstring
numpy._typing._array_like
numpy._typing._callable
numpy._typing._char_codes
numpy._typing._dtype_like
numpy._typing._extended_precision
numpy._typing._nbit
numpy._typing._nbit_base
numpy._typing._nested_sequence
numpy._typing._scalars
numpy._typing._shape
numpy._typing._ufunc
numpy._utils
numpy._utils._convertions
numpy.char
numpy.core
numpy.ctypeslib
numpy.dtypes
numpy.exceptions
numpy.f2py
numpy.fft
numpy.fft._helper
numpy.fft._pocketfft
numpy.lib
numpy.lib._array_utils_impl
numpy.lib._arraypad_impl
numpy.lib._arraysetops_impl
numpy.lib._arrayterator_impl
numpy.lib._datasource
numpy.lib._function_base_impl
numpy.lib._histograms_impl
numpy.lib._index_tricks_impl
numpy.lib._nanfunctions_impl
numpy.lib._npyio_impl
numpy.lib._polynomial_impl
numpy.lib._scimath_impl
numpy.lib._shape_base_impl
numpy.lib._stride_tricks_impl
numpy.lib._twodim_base_impl
numpy.lib._type_check_impl
numpy.lib._ufunclike_impl
numpy.lib._utils_impl
numpy.lib._version
numpy.lib.array_utils
numpy.lib.format
numpy.lib.introspect
numpy.lib.mixins
numpy.lib.npyio
numpy.lib.scimath
numpy.lib.stride_tricks
numpy.linalg
numpy.linalg._linalg
numpy.ma
numpy.ma.core
numpy.ma.extras
numpy.ma.mrecords
numpy.matlib
numpy.matrixlib
numpy.matrixlib.defmatrix
numpy.polynomial
numpy.polynomial._polybase
numpy.polynomial._polytypes
numpy.polynomial.chebyshev
numpy.polynomial.hermite
numpy.polynomial.hermite_e
numpy.polynomial.laguerre
numpy.polynomial.legendre
numpy.polynomial.polynomial
numpy.polynomial.polyutils
numpy.random
numpy.random._generator
numpy.random._mt19937
numpy.random._pcg64
numpy.random._philox
numpy.random._sfc64
numpy.random.bit_generator
numpy.random.mtrand
numpy.rec
numpy.strings
numpy.testing
numpy.testing._private.utils
numpy.testing.overrides
numpy.typing
numpy.version
opcode
operator
os
os.path
pathlib
pdb
pickle
posixpath
profile
pstats
pydoc
pyexpat
pyexpat.errors
pyexpat.model
queue
re
reprlib
resource
selectors
shutil
signal
socket
sre_compile
sre_constants
sre_parse
ssl
stellarpunk
stellarpunk.agenda
stellarpunk.config
stellarpunk.core
stellarpunk.core.base
stellarpunk.core.character
stellarpunk.core.combat
stellarpunk.core.entity_order_watch
stellarpunk.core.gamestate
stellarpunk.core.order
stellarpunk.core.production_chain
stellarpunk.core.sector
stellarpunk.core.sector_entity
stellarpunk.core.ship
stellarpunk.data
stellarpunk.dialog
stellarpunk.econ
stellarpunk.econ_sim
stellarpunk.effects
stellarpunk.events
stellarpunk.events.core
stellarpunk.events.events
stellarpunk.generate
stellarpunk.generate.core
stellarpunk.generate.markov
stellarpunk.goap_toy
stellarpunk.hist_extract
stellarpunk.intel
stellarpunk.interface
stellarpunk.interface.audio
stellarpunk.interface.character
stellarpunk.interface.command_input
stellarpunk.interface.comms
stellarpunk.interface.manager
stellarpunk.interface.pilot
stellarpunk.interface.presenter
stellarpunk.interface.sector
stellarpunk.interface.starfield
stellarpunk.interface.startup
stellarpunk.interface.station
stellarpunk.interface.ui_events
stellarpunk.interface.ui_util
stellarpunk.interface.universe
stellarpunk.narrative
stellarpunk.narrative.director
stellarpunk.narrative.rule_parser
stellarpunk.orders
stellarpunk.orders.collision
stellarpunk.orders.core
stellarpunk.orders.movement
stellarpunk.orders.steering
stellarpunk.predicates
stellarpunk.sensors
stellarpunk.serialization
stellarpunk.serialization.agenda
stellarpunk.serialization.character
stellarpunk.serialization.combat
stellarpunk.serialization.econ
stellarpunk.serialization.effect
stellarpunk.serialization.events
stellarpunk.serialization.gamestate
stellarpunk.serialization.intel
stellarpunk.serialization.movement
stellarpunk.serialization.order
stellarpunk.serialization.order_core
stellarpunk.serialization.save_game
stellarpunk.serialization.sector
stellarpunk.serialization.sector_entity
stellarpunk.serialization.sensors
stellarpunk.serialization.serialize_econ_sim
stellarpunk.serialization.util
stellarpunk.sim
stellarpunk.spunk_image
stellarpunk.task_schedule
stellarpunk.util
string
struct
subprocess
sys
sys._monitoring
tarfile
tempfile
termios
textwrap
threading
time
types
typing
typing_extensions
unittest
unittest._log
unittest.async_case
unittest.case
unittest.loader
unittest.main
unittest.result
unittest.runner
unittest.signals
unittest.suite
uuid
warnings
weakref
xml
xml.etree.ElementTree
xml.parsers
xml.parsers.expat
zipfile
zipfile._path
zipimport
zlib
This feels wrong - if we have
xml.etree.ElementTree, how comexml.etreeis not there?numpy.testing._privateandxml.etreeare the only such modules; they are added byself.find_reachable_changed_modules.Honestly, I'm done for today, I'd appreciate if someone who knows this part better could confirm whether I'm on the right track.
sterliakov commentedon Apr 10, 2025
Well, the following (I don't know whether this is a reasonable thing to do or the patch should be at some lower level!) fixes the symptoms.
It also fixes the "vanishing error" problem I mentioned earlier, so the dependency omission is indeed the common cause for both of them?
@ilevkivskyi sorry to bother you, but it's a p-high issue affecting many users, and you should be familiar with this
beastdaemon... Does this change make sense at all?This change also somehow fixes #17652 (I'm able to reproduce that one consistently on 1.15.0 and it's gone with this patch applied)
ilevkivskyi commentedon Apr 10, 2025
FWIW something very wrong happens here. IIUC there were no changes between
dmypy runs, so daemon should have literally just fetched old error messages and printed them. I will try to figure out what is going on, but it may take a while.sterliakov commentedon Apr 10, 2025
That happens on the third and subsequent runs, while the second run is still slow (for me it's like 80s, 20s, <0.5s with non-compiled mypy). With the patch applied, it goes exactly as you say: all runs after the first complete immediately (
roundtrip_time : 0.047).So either this is the right thing to do or dependency is missed somewhere else.
sterliakov commentedon Apr 10, 2025
Wait, but it is the right thing to do. When we traverse dependencies in
find_reachable_changed_modules, we do want to include all ancestors as well - at least because we did so previously, and that's the reason they are on the graph. The same happens in another place where we ensure that all deps are loaded:mypy/mypy/server/update.py
Lines 472 to 497 in a4e79ea
I'm opening a PR with this change. I don't yet understand how to write a self-contained unit test, but at least understand why my suggestion actually helps. Now some ancestors are unreachable when we only consider explicit dependencies, and so dmypy treats those modules as deleted. Modules are still there, the state becomes inconsistent.
Traverse module ancestors when traversing reachable graph nodes durin…
DavideCanton commentedon May 16, 2025
I see that this issue is fixed in the main branch, is there a plan for releasing this fix? Mypy in vscode is quite unusable at the moment, since it is quite slow when the daemon is turned off.
A5rocks commentedon May 16, 2025
I assume this will be in mypy 1.16
svalentin commentedon May 16, 2025
Yes, this will be in 1.16. We are almost done with the release. Follow the release issue here. It will most likely be out right after PyCon (ends May 22)
DavideCanton commentedon May 16, 2025
Ah ok, I thought that since that's a big issue for mypy users in IDE it would have been released in a 1.15.1, but I guess I'll wait for 1.16 and use a built wheel of 1.16.dev in the meanwhile.
Thanks!
svalentin commentedon May 16, 2025
I recommend the wheel for 9e45dad. It should address another dmypy bug.