You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Python] Create Python examples of HTTP GET Arrow client and server using HTTP compression #40601
For example, the client sends a request with header Accept-Encoding: gzip, deflate, br, zstd and the server sends a response with header Content-Encoding: gzip and with the response body gzip-compressed which the HTTP client library automatically decompresses.
@felipecrv let's please make sure we test HTTP compression using the zstd codec. That way we will have a same-codec comparison between HTTP compression and Arrow IPC buffer compression.
We should also test other HTTP compression codecs, since zstd is not as common for HTTP.
(Besides zstd, the only other codec that IPC buffer compression supports is lz4. But lz4 is rarely or never supported for HTTP compression. So zstd is our only choice for same-codec comparisons.)
I'm doing all of them actually. Server can acts according to the Accept-Encoding header and negotiate the chosen compression. But I will put a recommendation for zstd for sure.
Activity
felipecrv commentedon Mar 17, 2024
watching
ianmcook commentedon Jun 29, 2024
@felipecrv I suggested using Python here for simplicity, but it could be in some other language if needed
ianmcook commentedon Sep 4, 2024
@felipecrv let's please make sure we test HTTP compression using the zstd codec. That way we will have a same-codec comparison between HTTP compression and Arrow IPC buffer compression.
We should also test other HTTP compression codecs, since zstd is not as common for HTTP.
(Besides zstd, the only other codec that IPC buffer compression supports is lz4. But lz4 is rarely or never supported for HTTP compression. So zstd is our only choice for same-codec comparisons.)
felipecrv commentedon Sep 5, 2024
I'm doing all of them actually. Server can acts according to the
Accept-Encodingheader and negotiate the chosen compression. But I will put a recommendation forzstdfor sure.felipecrv commentedon Sep 6, 2024
apache/arrow-experiments#35