Skip to content

Memory Leak disposing oldProgramΒ #58137

Description

@cplepage

πŸ”Ž Search Terms

memory leak

πŸ•— Version & Regression Information

  • This is a crash

⏯ Playground Link

No response

πŸ’» Code

// Your code here

πŸ™ Actual behavior

In safari (Webkit/JavaScriptCore). Old Program never gets GCed because of reference on it.

Memory build up really quickly.

πŸ™‚ Expected behavior

Unreferenced and gc

Additional information about the issue

No response

Activity

  1. cplepage commented on Apr 9, 2024

    @cplepage
    ContributorAuthor

    Safari web inspector, running TS language service to getDiagnostics on file update.

    without fix:
    Screenshot 2024-04-09 at 3 25 25β€―PM

    with fix:
    Screenshot 2024-04-09 at 3 27 02β€―PM

  2. RyanCavanaugh commented on Apr 11, 2024

    @RyanCavanaugh
    Member

    Accepting PRs to fix memory leaks, but we obviously can't really investigate something without a repro

  3. cplepage commented on Apr 17, 2024

    @cplepage
    ContributorAuthor

    Ryan Cavanaugh (@RyanCavanaugh) definitely, I will build a little something to make this issue reproducible and figure out what could go on. But I will need some time. Please allow me 1-2 week and I will come back here with a clear workable playground.

  4. tmm1 commented on Jan 19, 2025

    @tmm1

    Friendly ping on repro

  5. cplepage commented on Jan 21, 2025

    @cplepage
    ContributorAuthor

    Yes sorry about going radio silent here.

    Since discovering this issue and digging deep enough to believe the problem lays deep inside WebKit/JavaScriptCore, I simply patched it with my dumb fix https://github.com/fullstackedorg/editor/blob/main/build.ts#L9

    To reproduce the issue, it needs a full in-browser setup of loading tons of files (node_modules) and requesting the language host repeatedly solely in-browser. I haven't found any browser IDE or code-editor which does that and is more than a playground which implies only a few files.

    If you have any example of fully in-browser code-editor/IDE that allows to load packages from say npm of added by .tgz, I would be glad to check if the issue is reproducible in any of those.

    The project I'm working on, FullStacked, relies on that. For the last couple months, I've been working heavily on reaching a production ready stability, so I haven't put much thoughts on this issue. But you could try spinning up the project and simply compare in Safari the behaviour if you comment or not the code block previously mentioned here.

    There is no up-to-date contribution or how to build from source guide, so if you dive into that path, you'll probably need to figure out and modify some stuff. Although, I'm always available for any questions on the repo!

    I will hopefully invest some time in documentation and said guides shortly. As soon as I can send a straightforward way to reproduce, I will do that here for sure. Also, send me links of app/IDE/code-editor that runs in-browser with the possibility of loading complete libraries with dependencies. I'll help as much as possible on the issue.

  6. tmm1 commented on Jan 21, 2025

    @tmm1

    Very interesting, thanks for the details.

    There is another way to run tsserver with JSC, and in-fact I have observed similar memory growth issues in that environment- much more so than the equivalent code running in V8.

    In VSCode:

    Image

  7. cplepage commented on Jan 21, 2025

    @cplepage
    ContributorAuthor

    Yes!! Aman Karmani (@tmm1) You are one clever guy!

    The issue seems to be the same. The oldProgram being retained only in JSC.

    Here's the repo to test : https://github.com/cplepage/ts-leak.git

    It simply getSemanticDiagnostics every 100ms on a basic MUI tsx file. The test runs for 60s while logging the memory usage.

    To run:

    bun install
    
    bun index.js
    

    run with fix:

    bun index.js --fix
    

    run with node

    node index.js
    
    node index.js --fix
    

    Here are the results on my end

    no fix with fix
    bun Memory after 1m: 1.05 GB
    [max: 1.05 GB, min: 130 MB]
    Memory after 1m: 126 MB
    [max: 130 MB, min: 124 MB]
    node Memory after 1m: 383 MB
    [max: 528 MB, min: 161 MB]
    Memory after 1m: 311 MB
    [max: 528 MB, min: 167 MB]
    % bun --version 
    1.1.45
    
    % node --version
    v20.14.0
    
    % system_profiler SPHardwareDataType
    Hardware:
    
        Hardware Overview:
    
          Model Name: MacBook Pro
          Model Identifier: MacBookPro15,4
          Processor Name: Quad-Core Intel Core i5
          Processor Speed: 1.4 GHz
          Number of Processors: 1
          Total Number of Cores: 4
          L2 Cache (per Core): 256 KB
          L3 Cache: 6 MB
          Hyper-Threading Technology: Enabled
          Memory: 16 GB
          System Firmware Version: 2069.40.2.0.0 (iBridge: 22.16.11072.0.0,0)
          OS Loader Version: 582~2132
          Activation Lock Status: Enabled
    
  8. tmm1 commented on Jan 21, 2025

    @tmm1

    Wonderful we have a repro! What a dramatic difference on the bun side.

    Ryan Cavanaugh (@RyanCavanaugh) wdyt

  9. tmm1 commented on Jan 21, 2025

    @tmm1

    My results:

    no fix fix
    bun 1.1.45 Memory after 1m: 1.7 GB [max: 1.7 GB, min: 148 MB] Memory after 1m: 127 MB [max: 131 MB, min: 127 MB]
    node 18.17.1 Memory after 1m: 445 MB [max: 536 MB, min: 194 MB] Memory after 1m: 304 MB [max: 534 MB, min: 190 MB]
    node 20.18.1 Memory after 1m: 518 MB [max: 518 MB, min: 193 MB] Memory after 1m: 511 MB [max: 524 MB, min: 193 MB]
  10. maschwenk commented on Jan 23, 2025

    @maschwenk
    Contributor

    That repro is very interesting. We have a similar problem where specifically on an app that also depends on

        "@emotion/react": "^11.14.0",
        "@emotion/styled": "^11.14.0",
    

    Can consume 9-10 GB in < a minute. I don't know if it'd technically be called a leak because it does not grow indefinitely though.

    I've tried applying this fix and does not seem to help, so maybe unrelated, but I'm sort of curious if the choice of mui/@emotion was on purpose? I've also noticed that even after all files are closed in VScode, memory will be pinned to that 9-10GB number indefinitely (which does seem a bit leaky?). This https://github.com/cplepage/ts-leak repro has inspired me to maybe make my own.

  11. jakebailey commented on Jan 23, 2025

    @jakebailey
    Member

    I've also noticed that even after all files are closed in VScode, memory will be pinned to that 9-10GB number indefinitely (which does seem a bit leaky?).

    It's doing that intentionally (the keeping in memory part, not the 10GB part), since you may reopen a file. If you close all your files, then open a file, it's not so good to have that be a complete restart.

    That being said, I don't know how you can possibly see 9-10GB of memory usage; we have a hard cap when run via VS Code of 4GB, but maybe you have instructed VS Code to use the "real" node instead with different max space flags, but then surely that's a different bug to this JSC problem.

    This https://github.com/cplepage/ts-leak repro has inspired me to maybe make my own.

    More repros are of course, good.

  12. jakebailey commented on Jan 23, 2025

    @jakebailey
    Member

    FWIW I still do not think #58138 itself is the right fix, just from an implementation perspective; it's possible there's a similar change that would work fine. But I really do wonder if this is something JSC should be fixing given V8 does not have this problem.

    Note that in #58137 (comment), Node's (V8's) min/max memory usage is the same before and after the change (I would not focus on a specific measurement since that really depends on when the last GC run ran among other things).

  13. maschwenk commented on Jan 23, 2025

    @maschwenk
    Contributor

    It's doing that intentionally (the keeping in memory part, not the 10GB part), since you may reopen a file. If you close all your files, then open a file, it's not so good to have that be a complete restart.

    ❀ makes sense. though I wonder, for folks working on very large monorepos 300+ packages, 30+ apps, would it be possible to consider submitting a patch that allows tsserver to dispose of projects fully after some configurable window? i.e. infinity by default but 5 minutes by configuration. one way we had conceived of keeping our editors stable was by modifying the "max open" settings so that folks couldn't accidentally have 50+ tabs open, and as they moved between parts of the codebase the memory would idly drain out.

    but maybe you have instructed VS Code to use the "real" node instead with different max space flags, but then surely that's a different bug to this JSC problem.

    yep exactly. which is a bummer because we don't get the pointer compression wins either.

    totally understand that this is far afield from original question. I can file a seperate issue for the "dispose of old Projects after n minutes" idea. though still a bit curious if emotion was picked for a specific reason in this repro example.

  14. jakebailey commented on Jan 23, 2025

    @jakebailey
    Member

    I'm looking at the bun thing, but it sounds like your issue is likely unrelated unless you're applying #58138 and finding a benefit.

    mui / emotion are famously some of the slowest types out there, so it doesn't surprise me too much; in your case, I would try and verify that your packages use the same tsconfig options to maximize reuse, and also consider using some of the "make tsserver load less with downsides" options:

  15. jakebailey commented on Jan 23, 2025

    @jakebailey
    Member

    I really don't like the fix, but I cannot find anything that works. #61034 is my version of #51838 and some additional modifications which make me feel less scared to accept the hack. Let me know if it still works. It should, and the repro at least let me test it (thank you for making one).

  16. jakebailey commented on Jan 23, 2025

    @jakebailey
    Member

    Actually, I found the actual problem, #61034 now contains a fix for that.

  17. tmm1 commented on Jan 23, 2025

    @tmm1

    Note that in #58137 (comment), Node's (V8's) min/max memory usage is the same before and after the change (I would not focus on a specific measurement since that really depends on when the last GC run ran among other things).

    Good point, if you run with node --expose-gc and add explicit global.gc() calls before the measurements, the usage is completely flat (just like bunjs is after the patch).

  18. jakebailey commented on Jan 23, 2025

    @jakebailey
    Member

    Yep, that's exactly what I did locally when testing.

  19. tmm1 commented on Jan 24, 2025

    @tmm1

    repro has inspired me to maybe make my own.

    Would be great if we can find a repro and make a new issue.

    I agree the current behavior wrt to memory usage on large typescript projects is far from ideal.

    we have a hard cap when run via VS Code of 4GB

    Is this referring to the default max-old-space limit, or the pointer compression limit?

    It's doing that intentionally (the keeping in memory part, not the 10GB part), since you may reopen a file. If you close all your files, then open a file, it's not so good to have that be a complete restart.

    Has there been any consideration to using some kind of LRU, instead of an unbounded map here:

    readonly filenameToScriptInfo: Map<Path, ScriptInfo> = new Map();

    I can observe this growing as I navigate into files and dependencies, and sticking around longer after I care about those files. Eventually memory will hit the defined limits, which results in a very poor experience as GC and cpu usage increase until an eventual OOM.

    Here for instance 500mb is retained by this map, and there is 1.6G worth of SourceFileObject

    Image
  20. jakebailey commented on Jan 24, 2025

    @jakebailey
    Member

    Is this referring to the default max-old-space limit, or the pointer compression limit?

    Pointer compression limit.

    Has there been any consideration to using some kind of LRU, instead of an unbounded map here:

    I would suggest making a new issue; that seems unrelated to the leak from this thread. In your screenshot, there are only 4000 entries or so; that is the number of files in the VS Code project; it's not that large.

  21. locked as resolved and limited conversation to collaborators on Oct 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Help WantedYou can do thisPossible ImprovementThe current behavior isn't wrong, but it's possible to see that it might be better in some cases

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions