Repository navigation
Suggestion: stream.read(buffer, offset, length, callback) #3403
Description
I am not sure if this is possible with the design of libuv and Node, but it would be great to be able to do:
stream.read(buffer, offset, length, callback)
buffer is the buffer that the data will be written to.
offset is the offset in the buffer to start writing at.
length is an integer specifying the number of bytes to read.
Especially for net and tls sockets.
I am not sure if Node itself currently manages and re-uses the underlying buffers for net sockets? But if not, then this would make it possible to do static memory allocation in a server. In the past, streams did not support pull mode at all, now they support pull mode, but the user still has no control over the allocation of the underlying buffers, so this creates more allocations and more work for the GC.
I am working on a client-side program at the moment and am finding that V8's GC is just too lazy. Things work and eventually get collected but memory usage climbs and climbs. I would like to have more control and rather statically allocate buffers and re-use them. This would also have a positive side-effect of controlling queue sizes and workloads in various parts of the program more explicitly.
Activity
Fishrock123 commentedon Oct 20, 2015
cc @nodejs/streams
chrisdickinson commentedon Oct 20, 2015
Something like this would be cool. WHATWG currently has "getByobReader", which implements something similar. I would advise we watch how that pans out closely to see if there are any rough edges we should know about before implementing such a thing ourselves.
bnoordhuis commentedon Oct 20, 2015
Apropos nothing:
Request-driven reading is not the model that libuv currently uses but it can be modeled reasonably efficiently by selectively calling
uv_read_start()anduv_read_stop().That's what libuv's socks5-proxy example does in fact and it's the most natural model for most applications. The plan is to implement it natively in libuv v2.x.
Trott commentedon May 26, 2016
Anything new to add to this issue in the past 7 months?
jhamhader commentedon May 26, 2016
Looking into it.
Need to think on how this API makes sense in the case of
objectModeorstringMode(encoding set) on the readable stream.vkurchatkin commentedon May 26, 2016
#6923
Trott commentedon Jul 7, 2017
This issue has been inactive for sufficiently long that it seems like perhaps it should be closed. Feel free to re-open (or leave a comment requesting that it be re-opened) if you disagree. I'm just tidying up and not acting on a super-strong opinion or anything like that.
jorangreef commentedon Jul 21, 2017
This issue is blocked on libuv v2.x and nodejs/node-eps#27 which may take some time.
Can we re-open?
We definitely need a way to do more static allocation in Node specifically when working with TCP.
refack commentedon Nov 11, 2018
Put into https://github.com/nodejs/node/projects/13 backlog