You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Following on from #2677 we now support streaming responses from object storage
Describe the solution you'd like
The underlying challenge is to take arbitrary Stream<Bytes> and convert it into a Stream<Bytes> where each stream element contains complete rows, as delimited by a newline character. Once we have this DelimitedStream, it is trivial to feed each of these byte chunks individually into the corresponding decoder.
Activity
[-]Streaming CSV/JSON Read[/-][+]Streaming CSV/JSON Object Store Read[/+]Add streaming JSON and CSV (apache#2935)
Add streaming JSON and CSV reading, `NewlineDelimitedStream' (#2935) (#…