diff --git a/docs/source/additional_features.rst b/docs/source/additional_features.rst index 4ac46132e829..13f9b4cdd160 100644 --- a/docs/source/additional_features.rst +++ b/docs/source/additional_features.rst @@ -1,13 +1,7 @@ Additional features ------------------- -Several mypy features are not currently covered by this tutorial, -including the following: - -- inheritance between generic classes -- compatibility and subtyping of generic types, including covariance of generic types -- ``super()`` - +This section discusses various features outside core mypy features. .. _attrs_package: @@ -177,6 +171,30 @@ commit id of the merge base produced by the git command above. The script will decompress the data so that mypy will start with a fresh ``.mypy_cache``. Finally, the script runs mypy normally. And that's all! +Caching with mypy daemon +======================== + +You can also use remote caching with the :ref:`mypy daemon `. +The remote cache will significantly speed up the the first ``dmypy check`` +run after starting or restarting the daemon. + +The mypy daemon requires extra fine-grained dependency data in +the cache files which aren't included by default. To use caching with +the mypy daemon, use the ``--cache-fine-grained`` option in your CI +build:: + + $ mypy --cache-fine-grained + +This flag adds extra information for the daemon to the cache. In +order to use this extra information, you will also need to use the +``--use-fine-grained-cache`` option with ``dymypy start`` or +``dmypy restart``. Example:: + + $ dmypy start -- --use-fine-grained-cache + +Now your first ``dmypy check`` run should be much faster, as it can use +cache information to avoid processing the whole program. + Refinements =========== @@ -187,6 +205,11 @@ at least if your codebase is hundreds of thousands of lines or more: from a previous run, there's no need to download the cache data and it's better to instead reuse the existing local cache data. +* If you use the mypy daemon, you may want to restart the daemon each time + after the merge base or local branch has changed to avoid processing a + potentially large number of changes in an incremental build, as this can + be much slower than downloading cache data and restarting the daemon. + * If the current local branch is based on a very recent master commit, the remote cache data may not yet be available for that commit, as there will necessarily be some latency to build the cache files. It diff --git a/docs/source/index.rst b/docs/source/index.rst index 582c1c4ee1b8..3556e23d34c3 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -29,6 +29,7 @@ Mypy is a static type checker for Python. additional_features command_line config_file + mypy_daemon python36 installed_packages faq diff --git a/docs/source/mypy_daemon.rst b/docs/source/mypy_daemon.rst new file mode 100644 index 000000000000..b315804b1ed0 --- /dev/null +++ b/docs/source/mypy_daemon.rst @@ -0,0 +1,99 @@ +.. _mypy_daemon: + +Mypy daemon (mypy server) +========================= + +Instead of running mypy as a command-line tool, you can also run it as +a long-running daemon (server) process and use a command-line client to +send type-checking requests to the server. This way mypy can perform type +checking much faster, since program state cached from previous runs is kept +in memory and doesn't have to be read from the file system on each run. +The server also uses finer-grained dependency tracking to reduce the amount +of work that needs to be done. + +If you have a large codebase to check, running mypy using the mypy +daemon can be *10 or more times faster* than the regular command-line +``mypy`` tool, especially if your workflow involves running mypy +repeatedly after small edits -- which is often a good idea, as this way +you'll find errors sooner. + +.. note:: + + The mypy daemon is experimental. In particular, the command-line + interface may change in future mypy releases. + +.. note:: + + The mypy daemon currently supports macOS and Linux only. + +.. note:: + + Each mypy daemon process supports one user and one set of source files, + and it can only process one type checking request at a time. You can + run multiple mypy daemon processes to type check multiple repositories. + +Basic usage +*********** + +The client utility ``dmypy`` is used to control the mypy daemon. +Use ``dmypy start -- `` to start the daemon. You can use almost +arbitrary mypy flags after ``--``. The daemon will always run on the +current host. Example:: + + dmypy start -- --follow-imports=skip + +.. note:: + You'll need to use either the ``--follow-imports=skip`` or the + ``--follow-imports=error`` option with dmypy because the current + implementation can't follow imports. + See :ref:`follow-imports` for details on how these work. + You can also define these using a + :ref:`configuration file `. + +The daemon will not type check anything when it's started. +Use ``dmypy check `` to check some files (or directories):: + + dmypy check prog.py pkg1/ pkg2/ + +You need to provide all files or directories you want to type check +(other than stubs) as arguments. This is a result of the +``--follow-imports`` restriction mentioned above. + +The initial run will process all the code and may take a while to +finish, but subsequent runs will be quick, especially if you've only +changed a few files. You can use :ref:`remote caching ` +to speed up the initial run. The speedup can be significant if +you have a large codebase. + +Additional features +******************* + +You have precise control over the lifetime of the daemon process: + +* ``dymypy stop`` stops the daemon. + +* ``dmypy restart -- `` restarts the daemon. The flags are the same + as with ``dmypy start``. This is equivalent to a stop command followed + by a start. + +* Use ``dmypy start --timeout SECONDS -- `` (or + ``dmypy restart --timeout SECONDS -- ``) to automatically + shut down the daemon after inactivity. By default, the daemon runs + until it's explicitly stopped. + +Use ``dmypy --help`` for help on additional commands and command-line +options not discussed here, and ``dmypy --help`` for help on +command-specific options. + +Limitations +*********** + +* Changes related to protocol classes are not reliably propagated. + +* You have to use either the ``--follow-imports=skip`` or + the ``--follow-imports=error`` option because of an implementation + limitation. This can be defined + through the command line or through a + :ref:`configuration file `. + +* Windows is not supported.