Repository navigation
Connection reuse in HTTP/2 on concurrent requests #80
Description
While doing some tests with HTTP/2 and trace logging I noticed that when using concurrent requests connections are being created until hitting max_connections, even though (I believe) only one should be created and new streams should be used within the same connection.
Here's a quick example:
import asyncio
import httpx
async def main(url, n):
async with httpx.AsyncClient(
pool_limits=httpx.PoolLimits(soft_limit=2, hard_limit=5),
timeout=httpx.Timeout(5.0),
http2=True,
) as client:
await asyncio.gather(*[request(client, url, i) for i in range(n)])
async def request(client, url, i):
response = await client.get(url)
assert response.status_code == 200
assert response.http_version == "HTTP/2"
if __name__ == "__main__":
asyncio.run(main("https://example.org", 10))With some logging tweaks prints:
DEBUG [2020-05-10 10:36:32] httpcore._async.connection_pool - get_connection_from_pool=(b'https', b'example.org', 443)
DEBUG [2020-05-10 10:36:32] httpcore._async.connection_pool - get_connection_from_pool=(b'https', b'example.org', 443)
DEBUG [2020-05-10 10:36:32] httpcore._async.connection_pool - get_connection_from_pool=(b'https', b'example.org', 443)
DEBUG [2020-05-10 10:36:32] httpcore._async.connection_pool - get_connection_from_pool=(b'https', b'example.org', 443)
DEBUG [2020-05-10 10:36:32] httpcore._async.connection_pool - get_connection_from_pool=(b'https', b'example.org', 443)
DEBUG [2020-05-10 10:36:32] httpcore._async.connection_pool - get_connection_from_pool=(b'https', b'example.org', 443)
DEBUG [2020-05-10 10:36:32] httpcore._async.connection_pool - get_connection_from_pool=(b'https', b'example.org', 443)
DEBUG [2020-05-10 10:36:32] httpcore._async.connection_pool - get_connection_from_pool=(b'https', b'example.org', 443)
DEBUG [2020-05-10 10:36:32] httpcore._async.connection_pool - get_connection_from_pool=(b'https', b'example.org', 443)
DEBUG [2020-05-10 10:36:32] httpcore._async.connection_pool - get_connection_from_pool=(b'https', b'example.org', 443)
DEBUG [2020-05-10 10:36:32] httpcore._async.connection_pool - created connection=AsyncHTTPConnection origin=(b'https', b'example.org', 443) http2=True state=0 pool_size=1
DEBUG [2020-05-10 10:36:32] httpcore._async.connection_pool - created connection=AsyncHTTPConnection origin=(b'https', b'example.org', 443) http2=True state=0 pool_size=2
DEBUG [2020-05-10 10:36:32] httpcore._async.connection_pool - created connection=AsyncHTTPConnection origin=(b'https', b'example.org', 443) http2=True state=0 pool_size=3
DEBUG [2020-05-10 10:36:32] httpcore._async.connection_pool - created connection=AsyncHTTPConnection origin=(b'https', b'example.org', 443) http2=True state=0 pool_size=4
DEBUG [2020-05-10 10:36:32] httpcore._async.connection_pool - created connection=AsyncHTTPConnection origin=(b'https', b'example.org', 443) http2=True state=0 pool_size=5
DEBUG [2020-05-10 10:36:32] httpcore._async.connection_pool - created connection=AsyncHTTPConnection origin=(b'https', b'example.org', 443) http2=True state=0 pool_size=4
DEBUG [2020-05-10 10:36:32] httpcore._async.connection_pool - created connection=AsyncHTTPConnection origin=(b'https', b'example.org', 443) http2=True state=0 pool_size=5
DEBUG [2020-05-10 10:36:32] httpcore._async.connection_pool - created connection=AsyncHTTPConnection origin=(b'https', b'example.org', 443) http2=True state=0 pool_size=4
DEBUG [2020-05-10 10:36:32] httpcore._async.connection_pool - created connection=AsyncHTTPConnection origin=(b'https', b'example.org', 443) http2=True state=0 pool_size=5
DEBUG [2020-05-10 10:36:32] httpcore._async.connection_pool - created connection=AsyncHTTPConnection origin=(b'https', b'example.org', 443) http2=True state=0 pool_size=5
Tracing the flow in the code I found that:
- The first coro will attempt to fetch a connection for an origin
- It will return None so one is created
- It will attempt to add to the pool
- But the
asyncio.wait_foracquire will not return immediately and will trigger an event loop switch - The next coroutine will come in and try to pick a connection from the pool
- Will not find one and will create it and so on
And sure enough, removing the wait_for the output is:
DEBUG [2020-05-10 10:43:35] httpcore._async.connection_pool - get_connection_from_pool=(b'https', b'example.org', 443)
DEBUG [2020-05-10 10:43:35] httpcore._async.connection_pool - created connection=AsyncHTTPConnection origin=(b'https', b'example.org', 443) http2=True state=0 pool_size=1
DEBUG [2020-05-10 10:43:35] httpcore._async.connection_pool - get_connection_from_pool=(b'https', b'example.org', 443)
DEBUG [2020-05-10 10:43:35] httpcore._async.connection_pool - reuse connection=AsyncHTTPConnection origin=(b'https', b'example.org', 443) http2=True state=0 pool_size=1
DEBUG [2020-05-10 10:43:35] httpcore._async.connection_pool - get_connection_from_pool=(b'https', b'example.org', 443)
DEBUG [2020-05-10 10:43:35] httpcore._async.connection_pool - reuse connection=AsyncHTTPConnection origin=(b'https', b'example.org', 443) http2=True state=0 pool_size=1
DEBUG [2020-05-10 10:43:35] httpcore._async.connection_pool - get_connection_from_pool=(b'https', b'example.org', 443)
DEBUG [2020-05-10 10:43:35] httpcore._async.connection_pool - reuse connection=AsyncHTTPConnection origin=(b'https', b'example.org', 443) http2=True state=0 pool_size=1
DEBUG [2020-05-10 10:43:35] httpcore._async.connection_pool - get_connection_from_pool=(b'https', b'example.org', 443)
DEBUG [2020-05-10 10:43:35] httpcore._async.connection_pool - reuse connection=AsyncHTTPConnection origin=(b'https', b'example.org', 443) http2=True state=0 pool_size=1
DEBUG [2020-05-10 10:43:35] httpcore._async.connection_pool - get_connection_from_pool=(b'https', b'example.org', 443)
DEBUG [2020-05-10 10:43:35] httpcore._async.connection_pool - reuse connection=AsyncHTTPConnection origin=(b'https', b'example.org', 443) http2=True state=0 pool_size=1
DEBUG [2020-05-10 10:43:35] httpcore._async.connection_pool - get_connection_from_pool=(b'https', b'example.org', 443)
DEBUG [2020-05-10 10:43:35] httpcore._async.connection_pool - reuse connection=AsyncHTTPConnection origin=(b'https', b'example.org', 443) http2=True state=0 pool_size=1
DEBUG [2020-05-10 10:43:35] httpcore._async.connection_pool - get_connection_from_pool=(b'https', b'example.org', 443)
DEBUG [2020-05-10 10:43:35] httpcore._async.connection_pool - reuse connection=AsyncHTTPConnection origin=(b'https', b'example.org', 443) http2=True state=0 pool_size=1
DEBUG [2020-05-10 10:43:35] httpcore._async.connection_pool - get_connection_from_pool=(b'https', b'example.org', 443)
DEBUG [2020-05-10 10:43:35] httpcore._async.connection_pool - reuse connection=AsyncHTTPConnection origin=(b'https', b'example.org', 443) http2=True state=0 pool_size=1
DEBUG [2020-05-10 10:43:35] httpcore._async.connection_pool - get_connection_from_pool=(b'https', b'example.org', 443)
DEBUG [2020-05-10 10:43:35] httpcore._async.connection_pool - reuse connection=AsyncHTTPConnection origin=(b'https', b'example.org', 443) http2=True state=0 pool_size=1
Note a single connection is created and the following coroutines reuse it.
Obviously just removing the wait_for is not the right thing to do since we want to keep the pooling timeout, but we probably need to think of a different way to achieve it to maximize throughput in HTTP/2.
Activity