Mypy report missing type stub in third-party library when uninstalling a library #17049

Bug Report

Mypy reports a missing type stub library from a third-party module in some case.

In some case I will get error such as

venv/lib/python3.11/site-packages/opensearchpy/connection/base.py:38: error: Library stubs not installed for "simplejson"  [import-untyped]
venv/lib/python3.11/site-packages/opensearchpy/serializer.py:32: error: Library stubs not installed for "simplejson"  [import-untyped]
venv/lib/python3.11/site-packages/opensearchpy/serializer.py:32: note: Hint: "python3 -m pip install types-simplejson"
venv/lib/python3.11/site-packages/opensearchpy/serializer.py:32: note: (or run "mypy --install-types" to install all missing stub packages)
venv/lib/python3.11/site-packages/opensearchpy/serializer.py:32: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports

Note that this error is coming from site-packages.

(A clear and concise description of what the bug is.)

To Reproduce

echo "import opensearchpy" > foo.py
pip install opensearch-py
mypy foo.py
pip install types-simplejson
mypy foo.py
pip uninstall types-simplejson
mypy foo.py # Fail with the error above!

Expected Behavior

I would expect the mypy following the uninstall to give the same result as the first mypy call before the install.

Deleting the cache in between command does not result in an error.

Actual Behavior

It seems like the mypy cache change the behavior and expect the types library to be present.

Your Environment

$ mypy --version
mypy 1.9.0 (compiled: yes)
$ python --version
Python 3.11.3

No mypy.ini set.

While this bug seems to be niche, it's making our CI flaky when we share cache between jobs that might not have the same dependency set. For example, someone adding a new dependency in a branch that runs a job sharing the mypy cache will make all the project other branches fail until all code rebase on this commit to include the new dependency.

Other people seems to have similar issues here: pantsbuild/pants#18519