Skip to content
This repository was archived by the owner on Apr 14, 2022. It is now read-only.
This repository was archived by the owner on Apr 14, 2022. It is now read-only.

Analysis of peewee with the new LS never finishes #193

Closed
Closed
Analysis of peewee with the new LS never finishes#193
Assignees
MikhailArkhipov
Labels
bugSomething isn't workingfeature: symbols
Milestone
Oct 2018.1

Description

@MikhailArkhipov
Mikhail Arkhipov (MikhailArkhipov)
opened on Oct 4, 2018

microsoft/vscode-python#2682

"Check out peewee. When the new LS tries to analyse the main file, peewee.py, it uses >100% of CPU and the RAM usage increases continuously (over 40GB at some point). This doesn't seem to complete, even after leaving it open and running for an hour. Things like the Go to symbol list never get rendered."

Activity

  1. Mikhail Arkhipov (MikhailArkhipov)
    added
    bugSomething isn't working
    on Oct 4, 2018
  2. Mikhail Arkhipov (MikhailArkhipov)
    self-assigned this
    on Oct 4, 2018

  3. MikhailArkhipov commented on Oct 4, 2018

    @MikhailArkhipov
    Mikhail Arkhipov (MikhailArkhipov)
    on Oct 4, 2018
    Author

    Looks like #173

    Assertion Failed
Hash code changed when merging AnalysisValues

   at Microsoft.PythonTools.Analysis.AnalysisSetDetails.AnalysisHashSet.AddOne(BucketSet& buckets, AnalysisValue key, Int32 hc, IEqualityComparer`1 comparer) in F:\PLS\src\Analysis\Engine\Impl\AnalysisHashSet.cs:line 425
   at Microsoft.PythonTools.Analysis.AnalysisSetDetails.AnalysisHashSet.AddOne(BucketSet& buckets, AnalysisValue key, IEqualityComparer`1 comparer) in F:\PLS\src\Analysis\Engine\Impl\AnalysisHashSet.cs:line 329
   at Microsoft.PythonTools.Analysis.AnalysisSetDetails.AnalysisHashSet.AddFromEnumerator(BucketSet& buckets, IEnumerator`1 items, IEqualityComparer`1 comparer) in F:\PLS\src\Analysis\Engine\Impl\AnalysisHashSet.cs:line 240
   at Microsoft.PythonTools.Analysis.AnalysisSetDetails.AnalysisHashSet.AddFromEnumerator(IEnumerator`1 items) in F:\PLS\src\Analysis\Engine\Impl\AnalysisHashSet.cs:line 232
   at Microsoft.PythonTools.Analysis.AnalysisSetDetails.AnalysisHashSet..ctor(IEnumerable`1 enumerable, IEqualityComparer`1 comparer) in F:\PLS\src\Analysis\Engine\Impl\AnalysisHashSet.cs:line 85
   at Microsoft.PythonTools.Analysis.AnalysisSet.AsUnion(IAnalysisSet set, UnionComparer comparer, Boolean& wasChanged) in F:\PLS\src\Analysis\Engine\Impl\AnalysisSet.cs:line 288
   at Microsoft.PythonTools.Analysis.AnalysisSet.AsUnion(IAnalysisSet set, Int32 strength, Boolean& wasChanged) in F:\PLS\src\Analysis\Engine\Impl\AnalysisSet.cs:line 247
   at Microsoft.PythonTools.Analysis.Analyzer.TypedDependencyInfo.MakeUnion(Int32 strength) in F:\PLS\src\Analysis\Engine\Impl\Analyzer\DependencyInfo.cs:line 130
   at Microsoft.PythonTools.Analysis.TypedDef`1.MakeUnion(Int32 strength) in F:\PLS\src\Analysis\Engine\Impl\VariableDef.cs:line 415
   at Microsoft.PythonTools.Analysis.TypedDef`1.MakeUnionStronger() in F:\PLS\src\Analysis\Engine\Impl\VariableDef.cs:line 406
   at Microsoft.PythonTools.Analysis.TypedDef`1.MakeUnionStrongerIfMoreThan(Int32 typeCount, IAnalysisSet extraTypes) in F:\PLS\src\Analysis\Engine\Impl\VariableDef.cs:line 398
   at Microsoft.PythonTools.Analysis.Analyzer.FunctionScope.UpdateParameters(FunctionAnalysisUnit unit, ArgumentSet others, Boolean enqueue, FunctionScope scopeWithDefaultParameters, Boolean usePlaceholders) in F:\PLS\src\Analysis\Engine\Impl\Analyzer\FunctionScope.cs:line 198
   at Microsoft.PythonTools.Analysis.Analyzer.FunctionAnalysisUnit.UpdateParameters(ArgumentSet callArgs, Boolean enqueue) in F:\PLS\src\Analysis\Engine\Impl\Analyzer\FunctionAnalysisUnit.cs:line 64
   at Microsoft.PythonTools.Analysis.Values.FunctionInfo.DoCall(Node node, AnalysisUnit callingUnit, FunctionAnalysisUnit calledUnit, ArgumentSet callArgs) in F:\PLS\src\Analysis\Engine\Impl\Values\FunctionInfo.cs:line 140
   at Microsoft.PythonTools.Analysis.Values.FunctionInfo.Call(Node node, AnalysisUnit unit, IAnalysisSet[] args, NameExpression[] keywordArgNames) in F:\PLS\src\Analysis\Engine\Impl\Values\FunctionInfo.cs:line 96
   at Microsoft.PythonTools.Analysis.Values.ClassInfo.AddCall(Node node, NameExpression[] keywordArgNames, AnalysisUnit unit, IAnalysisSet[] args) in F:\PLS\src\Analysis\Engine\Impl\Values\ClassInfo.cs:line 66
   at Microsoft.PythonTools.Analysis.Values.ClassInfo.Call(Node node, AnalysisUnit unit, IAnalysisSet[] args, NameExpression[] keywordArgNames) in F:\PLS\src\Analysis\Engine\Impl\Values\ClassInfo.cs:line 56
   at Microsoft.PythonTools.Analysis.Analyzer.ExpressionEvaluator.EvaluateCall(ExpressionEvaluator ee, Node node) in F:\PLS\src\Analysis\Engine\Impl\Analyzer\ExpressionEvaluator.cs:line 382
   at Microsoft.PythonTools.Analysis.Analyzer.ExpressionEvaluator.EvaluateWorker(Node node) in F:\PLS\src\Analysis\Engine\Impl\Analyzer\ExpressionEvaluator.cs:line 214
   at Microsoft.PythonTools.Analysis.Analyzer.ExpressionEvaluator.Evaluate(Expression node) in F:\PLS\src\Analysis\Engine\Impl\Analyzer\ExpressionEvaluator.cs:line 54
   at Microsoft.PythonTools.Analysis.Analyzer.DDG.Walk(ReturnStatement node) in F:\PLS\src\Analysis\Engine\Impl\Analyzer\DDG.cs:line 578
   at Microsoft.PythonTools.Parsing.Ast.ReturnStatement.Walk(PythonWalker walker) in F:\PLS\src\Analysis\Engine\Impl\Parsing\Ast\ReturnStatement.cs:line 32
   at Microsoft.PythonTools.Analysis.Analyzer.DDG.Walk(SuiteStatement node) in F:\PLS\src\Analysis\Engine\Impl\Analyzer\DDG.cs:line 662
   at Microsoft.PythonTools.Parsing.Ast.SuiteStatement.Walk(PythonWalker walker) in F:\PLS\src\Analysis\Engine\Impl\Parsing\Ast\SuiteStatement.cs:line 34
   at Microsoft.PythonTools.Analysis.Analyzer.FunctionAnalysisUnit.AnalyzeWorker(DDG ddg, CancellationToken cancel) in F:\PLS\src\Analysis\Engine\Impl\Analyzer\FunctionAnalysisUnit.cs:line 89
   at Microsoft.PythonTools.Analysis.AnalysisUnit.Analyze(DDG ddg, CancellationToken cancel) in F:\PLS\src\Analysis\Engine\Impl\AnalysisUnit.cs:line 139
   at Microsoft.PythonTools.Analysis.Analyzer.DDG.Analyze(Deque`1 queue, CancellationToken cancel, Action`1 reportQueueSize, Int32 reportQueueInterval) in F:\PLS\src\Analysis\Engine\Impl\Analyzer\DDG.cs:line 75
   at Microsoft.PythonTools.Analysis.PythonAnalyzer.AnalyzeQueuedEntries(CancellationToken cancel) in F:\PLS\src\Analysis\Engine\Impl\PythonAnalyzer.cs:line 925
   at Microsoft.PythonTools.Intellisense.AnalysisQueue.GroupAnalysis.Analyze(CancellationToken cancel) in F:\PLS\src\Analysis\Engine\Impl\Intellisense\AnalysisQueue.cs:line 224
   at Microsoft.PythonTools.Intellisense.AnalysisQueue.HandleAnalyzable(IAnalyzable item, AnalysisPriority priority, CancellationToken cancellationToken) in F:\PLS\src\Analysis\Engine\Impl\Intellisense\AnalysisQueue.cs:line 148
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
   at Microsoft.PythonTools.Intellisense.AnalysisQueue.HandleAnalyzable(IAnalyzable item, AnalysisPriority priority, CancellationToken cancellationToken)
   at Microsoft.PythonTools.Intellisense.AnalysisQueue.<>c__DisplayClass28_0.<Enqueue>b__0(CancellationToken ct) in F:\PLS\src\Analysis\Engine\Impl\Intellisense\AnalysisQueue.cs:line 132
   at Microsoft.PythonTools.Intellisense.AnalysisQueue.ConsumerLoop() in F:\PLS\src\Analysis\Engine\Impl\Intellisense\AnalysisQueue.cs:line 60
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.MoveNext()
   at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(IAsyncStateMachineBox box, Boolean allowInlining)
   at System.Threading.Tasks.Task.RunContinuations(Object continuationObject)
   at System.Threading.Tasks.Task`1.TrySetResult(TResult result)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.SetExistingTaskResult(TResult result)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.SetResult(TResult result)
   at Microsoft.PythonTools.Intellisense.AnalysisQueue.ConsumeAsync() in F:\PLS\src\Analysis\Engine\Impl\Intellisense\AnalysisQueue.cs:line 85
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.MoveNext()
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
  4. Mikhail Arkhipov (MikhailArkhipov)
    added this to the Oct 2018.1 milestone on Oct 5, 2018
  5. Mikhail Arkhipov (MikhailArkhipov)
    added
    feature: symbols
    on Oct 5, 2018

  6. MikhailArkhipov commented on Oct 5, 2018

    @MikhailArkhipov
    Mikhail Arkhipov (MikhailArkhipov)
    on Oct 5, 2018
    Author

    #201

  7. Mikhail Arkhipov (MikhailArkhipov)
    closed this as completedon Oct 5, 2018

  8. stessaris commented on Oct 17, 2018

    @stessaris
    stessaris
    on Oct 17, 2018

    I still have the RAM problem on OSX with Version 2018.9.0

  9. MikhailArkhipov commented on Oct 17, 2018

    @MikhailArkhipov
    Mikhail Arkhipov (MikhailArkhipov)
    on Oct 17, 2018
    Author

    stessaris - how big is the RAM consumption? Runaway or large but stable? What is the project size? Is it better with Jedi?

  10. stessaris commented on Oct 17, 2018

    @stessaris
    stessaris
    on Oct 17, 2018

    Mikhail Arkhipov (@MikhailArkhipov) - over 5 GB (enough to bring my 8GB iMac to its knees with the rest of applications running), I got the feeling that is not growing a lot after that but I'll do some tests. The project is 20 files for a total of 6500 lines of code. No issues with Jedi

  11. MikhailArkhipov commented on Oct 17, 2018

    @MikhailArkhipov
    Mikhail Arkhipov (MikhailArkhipov)
    on Oct 17, 2018
    Author

    stessaris - This bug is specifically about peewee, could you open another specific to your project please.

  12. GinRyan commented on Nov 10, 2018

    @GinRyan
    XellossRyan (GinRyan)
    on Nov 10, 2018

    It's still happening in language-server.0.1.60 (Platform: on Windows 10 1803, VSCode 1.28.2, Python 3.6.4)

    default

    There are just 5 python scripts in my project.

    After I killed this process, it launched again and auto increment to 4GB even 10GB .Auto increment step by 10MB per sec.

    I just import numpy and matplotlib in this project.

  13. jakebailey commented on Nov 10, 2018

    @jakebailey
    Jake Bailey (jakebailey)
    on Nov 10, 2018
    Member

    XellossRyan (@GinRyan) As previously mentioned, this issue is specifically about peewee. If this is occurring for another project, I'd appreciate it if you opened a new issue (especially if you can share the project that causes the high memory usage so we can profile it).

  14. MikhailArkhipov commented on Mar 6, 2019

    @MikhailArkhipov
    Mikhail Arkhipov (MikhailArkhipov)
    on Mar 6, 2019
    Author

    0.2.16 is now available on beta channel.

    "python.analysis.downloadChannel": "beta"

  15. maxn0d3x commented on Apr 27, 2019

    @maxn0d3x
    NiNJAD3vel0per (maxn0d3x)
    on Apr 27, 2019

    Mikhail Arkhipov (@MikhailArkhipov)
    What is this ? ""python.analysis.downloadChannel": "beta""
    Why we should put this odds line in setting.json without explanation ?
    My Mac is always +300% whenever turning of vscode with Python. This is not good !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingfeature: symbols

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions