Repository navigation
aiter_lines() doesn't return full lines that span multiple chunks #1033
Description
https://gist.github.com/scr-oath/aa76d200222a0409d09a0d6feb1a13e2 shows an example setup using cherry.py as server that just outputs two lines - the json is big enough to be sent in two chunks; httpx aiter_lines() gets confused and sends data from the middle of the json line - seems to skip the starting part - which was most likely sent in a chunk without a newline
test-httpx.py
import asyncio
import json
import httpx
class TestHttpx:
def __init__(self):
pass
async def __call__(self):
http_client = httpx.AsyncClient()
async with http_client.stream(method="GET", url='http://localhost:8080/lines') as response:
is_message = True
async for line in response.aiter_lines():
is_message = not is_message
if is_message:
message = json.loads(line)
print(message)
def main():
test_httpx = TestHttpx()
asyncio.run(test_httpx())
if __name__ == '__main__':
main()
Activity
scr-oath commentedon Jun 24, 2020
It seems like the LineDecoder needs to remember partial work when called again so that it can prepend the new text with the previous line.
scr-oath commentedon Jun 24, 2020
Here's a really small test case:
Fixes encode#1033 - Ensure that text that spans 3 invocations before …
Fixes #1033 - Ensure that text that spans 3 invocations before newlin…
Fixes for LineDecoding (#1075)