Skip to content

Should Buffer.concat really return the source buffer when length == 1? #1891

Description

@ronkorving

Buffer.concat returns a new Buffer instance that contains all given buffers combined. However, if the array of buffers that is passed has length 1, the original buffer from that array is returned as-is. Not a copy. That means that manipulation of Buffer.concat's output has no effect on the source buffers if they had length > 1, yet alters the source buffer when length == 1.

This seems very unpredictable (Array.prototype.concat for example always returns a copy) and is bound to at least confuse people, and at worst cause weird bugs.

Original discussion: #1825 (comment)

Activity

  1. added
    bufferIssues and PRs related to the buffer subsystem.
    on Jun 4, 2015
  2. trevnorris commented on Jun 4, 2015

    @trevnorris
    Contributor

    I do agree that returning the original buffer if length is 1 does seem slightly unintuitive, should mention that the way it currently works is documented:

    If the list has exactly one item, then the first item of the list is returned.

  3. dcousens commented on Jun 4, 2015

    @dcousens

    Most people end up doing this optimization themselves.

    That is, you see code all the time handling:

    if (arrayOfBuffers.length === 1) return arrayBuffers[0]
    // ...

    The most annoying part of the current behaviour is that if you really want a Buffer to be copied every time, you would have to handle it with new Buffer(Buffer.concat(arrayOfBuffers)) unless you directly check for it.

  4. thefourtheye commented on Jun 4, 2015

    @thefourtheye
    Contributor

    @trevnorris is there any specific reason for this behaviour?

  5. mscdex commented on Jun 4, 2015

    @mscdex
    Contributor

    @thefourtheye Optimization to avoid creating a new Buffer and doing a copy and making more work for the GC?

  6. trevnorris commented on Jun 4, 2015

    @trevnorris
    Contributor

    @thefourtheye legacy? it's been that way since before I seriously started working on node.

  7. benjamingr commented on Jun 4, 2015

    @benjamingr
    Member

    +1 on fixing it, it sounds like a good idea

  8. thefourtheye commented on Jun 4, 2015

    @thefourtheye
    Contributor

    I think that, if possible, we should fix this, so that people don't have to worry about this special case when using concat.

  9. monsanto commented on Jun 10, 2015

    @monsanto
    Contributor

    I agree that this should be fixed. Reminds me of the "Zalgo" problem with callbacks. Either the callback should always be called asynchronously, or always be called synchronously. Likewise, the object returned by Buffer.concat should always be safe to mutate, or never safe to mutate. (where safe means "you won't be surprised by aliasing")

  10. trevnorris commented on Jun 10, 2015

    @trevnorris
    Contributor

    Done in f4f16bf.

  11. benjamingr commented on Jun 10, 2015

    @benjamingr
    Member

    @trevnorris you're a beast 👍 so many small but useful fixes.

  12. trevnorris commented on Jun 10, 2015

    @trevnorris
    Contributor

    @benjamingr thanks. that patch was all @thefourtheye. I just ported it to the "next" branch. :)

  13. benjamingr commented on Jun 10, 2015

    @benjamingr
    Member

    Well @thefourtheye already got credit for being awesome yesterday in front of a thousand people - I guess saying thanks again wouldn't hurt! thanks!

  14. thefourtheye commented on Jun 10, 2015

    @thefourtheye
    Contributor

    @trevnorris Thanks :-) @benjamingr Oh, I am the one who has to thank you, for being such an inspiration. Thanks a lot man :-)

  15. ronkorving commented on Jun 11, 2015

    @ronkorving
    ContributorAuthor

    Thanks guys!

  16. reqshark commented on Jun 11, 2015

    @reqshark

    woah, that's how it's done! good work!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bufferIssues and PRs related to the buffer subsystem.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions