Skip to content

Large number of C++ objects unclaimed in node v0.12.7  #2813

Description

@gireeshpunathil

Node version: v0.12.7

Platform tested: Linux IA32

  1. Here is a standalone test case which reproduces the issue: https://github.com/gireeshpunathil/nodev0.12.7tlsmemoryleak
  2. Over time, Quite many objects of type WriteWrap, TLSWrap, SecureContext, TCP etc. pile up in the native heap, and the RSS growth drives node out of memory.
  3. Sample snapshots of the Object containment chain below. The leak is not apparent in the heapdump, as neither the comparison view nor the retention chain show up the anything, as these objects have lost their association with any JS objects.
  4. This issue seem related to Possible memory leak in TLS(Wrap?) #1075 but as the code base is different here, a fix does not look straightforward.
  5. We migrated from v0.10 to v0.12 due to an outstanding memory leak in the former. It would be great if this can be fixed in v0.12.7

image

image

image

Activity

  1. mscdex commented on Sep 11, 2015

    @mscdex
    Contributor

    Have you compared with node v4?

  2. added
    tlsIssues and PRs related to the tls subsystem.
    memoryIssues and PRs related to Node.js memory management or memory footprint.
    on Sep 11, 2015
  3. gireeshpunathil commented on Sep 11, 2015

    @gireeshpunathil
    MemberAuthor

    @mscdex - thank you. I would like to test in v4, but: i) 'npm install heapdump' fails with that version of node, ii) I also see couple of outstanding memory leak issue in v4, iii) as I said earlier, we came from v0.10 to resolve some of the issues out there, so would like to work with v0.12.7, and hence this request.

    bash-4.1$ npm install heapdump

    heapdump@0.3.7 install /home/node/node_modules/heapdump
    node-gyp rebuild

    make: Entering directory /home/node/node_modules/heapdump/build' make: Warning: Fileaddon.target.mk' has modification time 53 s in the future
    CXX(target) Release/obj.target/addon/src/heapdump.o
    In file included from /home/.node-gyp/4.0.0/include/node/node.h:42,
    from ../src/heapdump.cc:15:
    /home/.node-gyp/4.0.0/include/node/v8.h:336: error: expected unqualified-id before âusingâ

  4. targos commented on Sep 11, 2015

    @targos
    Member

    @gireeshpunathil it is weird. I can install heapdump without error on v4. Maybe try to remove the node_modules directory and reinstall ?

  5. bnoordhuis commented on Sep 11, 2015

    @bnoordhuis
    Member

    @gireeshpunathil You need a C++11-capable compiler, i.e., g++ 4.8 or clang++ 3.4 or newer.

  6. gireeshpunathil commented on Sep 11, 2015

    @gireeshpunathil
    MemberAuthor

    @bnoordhuis thanks - I guessed something similar from the compiler error which warns about the C++ syntax.
    I will try to fix this. Meanwhile, if the root cause of the leak in 0.12 can be detected, that would be great. Thanks once again!

  7. ChALkeR commented on Sep 11, 2015

    @ChALkeR
    Member

    Over time, Quite many objects of type WriteWrap, TLSWrap, SecureContext, TCP etc. pile up in the native heap, and the RSS growth drives node out of memory.

    #1522, #1529, #1580 — fixed in v1.8.2.

  8. ChALkeR commented on Sep 13, 2015

    @ChALkeR
    Member

    @indutny Should this be backported to 0.12 branch or not?
    If not, I guess we can close this issue.

  9. indutny commented on Sep 13, 2015

    @indutny
    Member

    @ChALkeR I think no.

  10. ChALkeR commented on Sep 18, 2015

    @ChALkeR
    Member

    Closing this. Feel free to reopen if this is going to be backported to 0.12 branch or if upgrading to 4.x doesn't fix things.

  11. base698 commented on Sep 21, 2015

    @base698

    We just upgraded to node 0.12.7. Our memory usage jumped from a baseline of about 350MB per node to 512MB per node--right at Heroku's max. It's fairly stable, just higher than normal. There are several issues mentioning a memory leak in later versions. What version is now safe? Should we jump to io.js 1.8.2 because the later memory leak is fixed?

  12. indutny commented on Sep 21, 2015

    @indutny
    Member

    @base698 I would go to 4.1.0 node.js

  13. base698 commented on Sep 21, 2015

    @base698

    Is the amount of memory used higher? Found this stackoverflow.com post about the issue we are experiencing: http://stackoverflow.com/questions/29914837/node-js-0-12-x-memory-usage Said to use --max-old-space-size=512

  14. indutny commented on Sep 21, 2015

    @indutny
    Member

    @base698 I think the memory amount is pretty low now. cc @rvagg

  15. base698 commented on Sep 22, 2015

    @base698

    We can't upgrade because of this issue effecting numerous packages: brianmcd/contextify#180 Seems @rvagg is referenced there as well.

  16. indutny commented on Sep 22, 2015

    @indutny
    Member

    Oh, sorry! I didn't read full thread. Thought, it was a rival of the old discussion.

  17. bnoordhuis commented on Sep 22, 2015

    @bnoordhuis
    Member

    @base698 I don't know what you do with contextify but you may be able to drop the dependency when you switch to v4.x, the built-in vm module basically works the same as contextify now.

  18. base698 commented on Sep 22, 2015

    @base698

    It's other modules we depend on--jsdom being one.
    On Sep 22, 2015 3:19 AM, "Ben Noordhuis" notifications@github.com wrote:

    @base698 https://github.com/base698 I don't know what you do with
    contextify but you may be able to drop the dependency when you switch to
    v4.x, the built-in vm module basically works the same as contextify now.

    —
    Reply to this email directly or view it on GitHub
    #2813 (comment).

  19. bnoordhuis commented on Sep 22, 2015

    @bnoordhuis
    Member

    What version of jsdom are you using? The v4.x releases only work with io.js v3.x and node.js v4.x.

  20. base698 commented on Sep 22, 2015

    @base698

    Tried 3.5.0, 4.0.0 on you recommendation just now, and 6.5.0 was the
    previous try.

    npm install jsdom # fails with contextify C++ error mentioned here:
    brianmcd/contextify#180 (comment)

    On Tue, Sep 22, 2015 at 7:25 AM, Ben Noordhuis notifications@github.com
    wrote:

    What version of jsdom are you using? The v4.x releases only work with
    io.js v3.x and node.js v4.x.

    —
    Reply to this email directly or view it on GitHub
    #2813 (comment).

  21. base698 commented on Sep 22, 2015

    @base698

    I did a small load test in our testing environment using node 12.7 and 4.1.0 for those interested. It does appear to have way better memory usage in 4.1.0.

    At the start of the graph where the vertical black bar is node 4.1.0 was installed. I did a big load test and it looks to have GC'd and behaved fine. Memory flat lined at a pretty low value. We deployed node 0.12.7 at the second vertical black bar and mem usage immediately went high. Doing a small load test put the mem usage even higher than it got with node 4.1.0 during a large load test.

    screen shot 2015-09-22 at 1 02 40 pm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    memoryIssues and PRs related to Node.js memory management or memory footprint.tlsIssues and PRs related to the tls subsystem.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions