Repository navigation
Investigate compressing TS Server socket communication #43156
Description
Activity
Matt Bierner (@mjbvz) let's discuss more next Wednesday.
Sounds good. Just a few quick notes from an initial investigation :
-
For small messages, the savings by compressing the message body using
deflateare minimal. For example going from 76 bytes to 68 bytes. However for large messages, the savings are much better, with a quick example I pulled up going from14529to1884(7x size reduction) -
From my (limited) understanding, if we used
gzipfor compression we could avoid using thecontent-lengthon wire header entirely since gzip already includes its own header. This would simplify parsing on the client side, although I haven't found good node examples of reading multiple gzip compressed messages from a stream
Dirk Bäumer (@dbaeumer) Does the LSP use compression? We would like to align with the LSP if possible here
-
I wonder if there's an intermediate approach where only the
bodyproperty of the response is compressed? That might make it easier for the server to decide per-message whether compression is worthwhile (assuming the client has opted in via user-preference).Reacted by Andrew BranchMatt Bierner (@mjbvz) JSONRPC does support compression and even a different encoding (for example MessagePack). But this currently needs to be setup on the client and server side when the connection is created. The LSP spec does currently not have support to upgrade a connection from an uncompressed to a compressed state.
To clarify: compression and encoding is pluggable. So you can choose whatever you want as long as client and server do the same :-)
I wonder if there's an intermediate approach where only the body property of the response is compressed?
As far as I know. We may have to apply base64 encode. but that will increase (about) 1/3 size.
I thought we may just add some header (As
Content-Length) likeContent-Encoding?Wenlu Wang (@Kingwl) I looked into that approach in LSP / JSONRPC and it makes things complicated with notification. In addition
Content-Encodingheader value in the response would require aAccept-Encodingheader in the request. Otherwise the server doesn't know which content encoding the client understands.For LSP I decided to propose the following route: during initialization handshake the client & server agree on a specific encoding. After the handshake the connection will be upgraded to that encoding or left at the default if no agreement could be reached.
would require a Accept-Encoding header in the request.
Seems we are repeating http protocol. :XD.
Another problem is TypeScript does not have external dependency (If i'm correct). So will we add something (for gzip) dependency, or just implement yet another compressor?
Problem
On desktop, TS Server uses a json based protocol. This protocol up being quite verbose for larger documents, especially for requests such as
navTreewhich return results from the entire fileJust opening
checker.tsfor example results in individual requests and responses that are a few MB in size.Proposal
We've seen up to a 10x reduction of on the wire data by enabling websocket per-message compression for VS Code's internal communications. We aren't using web sockets here, but I believe we should be able to fairly easily compress individual messages on the server side and decompress them on the editor side. Most importantly, we could enable this without any changes to the json protocol itself.
For backwards compatibility, this would have to be an opt-in feature