This repository was archived by the owner on Apr 14, 2022. It is now read-only.
Repository navigation
This repository was archived by the owner on Apr 14, 2022. It is now read-only.
Recursive or looping symlinks can cause language server to repeatedly analyze files #181
Description
Original issue can be found here along with the details microsoft/vscode-python#2613
I commented previously on #2297 but I found that symlinks were still causing problems after I grabbed a recent build today.
Environment data
- VS Code version: 1.27.2
- Extension version (available under the Extensions sidebar): 2018.9.0-alpha, retrieved at Sep 18, 10:30AM PDT.
- OS and version: Mac OS 10.13.6
- Python version (& distribution if applicable, e.g. Anaconda): 2.7.14
- Type of virtual environment used (N/A | venv | virtualenv | conda | ...): N/A
- Relevant/affected Python packages and their versions: N/A
Actual behavior
The Python language server gets caught in a super deep nest of recursive symlinks. The repro case given completes quickly (there's nothing to analyze) but still demonstrates the problem, I think.
If there is actual code to parse/analyze it pretty much goes on forever. "Analyzing workspace, #### items remaining" goes down and then quickly shoots up, and this repeats over and over.
I can workaround this by adding the offending symlinks/folder structures to a VSCode exclusion list.
Expected behavior
The Python language server should be able to handle these structures in some reasonable way.
Steps to reproduce:
- Create a folder structure like this
/code
/test
/some
test.py
/folder
/symlink -> ../../some
__init__.py (empty)
lib.py (empty)
- test.py only needs to contain
import folder.lib. - Enable trace output for the language server and watch the logs, it gets progressively deeper and deeper into a repeated
folder/symlink/folder/symlink/folder structure.
Logs
Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)
Initializing for /usr/local/opt/python/bin/python2.7
Loading files from /Users/username/code/test/some
Parsing document file:///Users/username/code/test/some/test.py
Parsing document file:///Users/username/code/test/some/folder/lib.py
Parsing document file:///Users/username/code/test/some/folder/__init__.py
Parse complete for file:///Users/username/code/test/some/test.py at version -1
Parse complete for file:///Users/username/code/test/some/folder/__init__.py at version -1
Analysis queued for file:///Users/username/code/test/some/test.py
Analysis queued for file:///Users/username/code/test/some/folder/__init__.py
Parsing document file:///Users/username/code/test/some/folder/symlink/test.py
Parsing document file:///Users/username/code/test/some/folder/symlink/folder/lib.py
Parse complete for file:///Users/username/code/test/some/folder/symlink/test.py at version -1
Analysis queued for file:///Users/username/code/test/some/folder/symlink/test.py
Parse complete for file:///Users/username/code/test/some/folder/symlink/folder/lib.py at version -1
Analysis queued for file:///Users/username/code/test/some/folder/symlink/folder/lib.py
Parse complete for file:///Users/username/code/test/some/folder/lib.py at version -1
Analysis queued for file:///Users/username/code/test/some/folder/lib.py
Parsing document file:///Users/username/code/test/some/folder/symlink/folder/__init__.py
Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/test.py
Parse complete for file:///Users/username/code/test/some/folder/symlink/folder/__init__.py at version -1
Analysis queued for file:///Users/username/code/test/some/folder/symlink/folder/__init__.py
Parse complete for file:///Users/username/code/test/some/folder/symlink/folder/symlink/test.py at version -1
Analysis queued for file:///Users/username/code/test/some/folder/symlink/folder/symlink/test.py
Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/lib.py
Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/__init__.py
Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/symlink/test.py
Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/symlink/folder/lib.py
Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/symlink/folder/__init__.py
Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/symlink/folder/symlink/test.py
Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/lib.py
Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/__init__.py
Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/test.py
Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/lib.py
Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/__init__.py
Parse complete for file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/lib.py at version -1
Analysis queued for file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/lib.py
<more>
Waiting for parsing to complete
Parsing complete. Waiting for analysis entries to enqueue
Enqueue complete. Waiting for analysis to complete
Received new analysis for file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/test.py
Output from Console under the Developer Tools panel (toggle Developer Tools on under Help)
Symlinks to a common folder (e.g. two projects which have a symlink to the same folder) also get re-analyzed but that's usually okay as analysis is pretty fast in general.
Activity
MikhailArkhipov commentedon Oct 3, 2018
Yeah, I might have solution
MikhailArkhipov commentedon Feb 8, 2019
Most probably no longer applies, so for verification with #432
AlexanderSher commentedon Feb 8, 2019
If we create correct PathResolverSnapshot, there should be no problem.
jakebailey commentedon Mar 4, 2019
Tested using this specific example, the new LS does not loop infinitely. There are some other symlink oddities (long import hints, loss of import completion once a symlink is imported), but I'll file separate issues for those.
jakebailey commentedon Mar 4, 2019
Reopening, there may be some other looping issues.
astrozot commentedon Apr 2, 2019
I actually still experiment this issue with the latest VS Code version (1.32.3).
Battleman commentedon Apr 4, 2019
From my side, I seem to have problems in looping too (characteristic of the "Analyzing in background, X items left"). But in my side, it's not the file/directory/workspace structure, but more the virtual environment. With a simple file (no import, one function), and using a conda env from another project I got absolutely no problem.
But when I import certain packages (I noticed sklearn poses problems), LS tries to analyses this and get in trouble. But for example, numpy (also "complex" package) poses no problem at all.
STRANGER YET, when going with a blank conda env, with only numpy and sklearn, the same import leads to no problem at all. So it's linked to packages in environment, but there also seems to be a third party factor.
Problematic case:
Environment
[edit] and extension version : 2019.3.6352 [/edit]
Description of the problem
As many others, with certain workspaces, the "Analyzing in background, X items left" appears, grows to a certain point, sometimes decreases and grows again, until being stuck indefinitely. Sucks up CPU and memory usage.
"Simple file" in question:
Content of the problematic conda env (sorry it's not minimal, I didn't find yet a minimal case)
Extract of the python output
SO, some files are queued, while some are completed in a matter of ms.
I guess that is the problem, some recursivity in specific packages. Maybe two (or more) of the packages in the env are referencing each other, creating a loop. Point is, with a conda env having only numpy and sklearn, this issue does not appear.
Sorry if I mis-followed the guidelines...
jakebailey commentedon Apr 4, 2019
Please file a new issue; this one is closed and is probably not related.
Regarding "only numpy and sklearn", there's a lot behind those libraries which may be affected by things relating to my explanation in #832 (comment).