Skip to content

HTTP/2 connections are always active within ConnectionPool #289

Closed

Description

@sethmlarson

It's trivial to dead-lock a ConnectionPool due to HTTP/2 connections being persistent and never cleaned up even when not servicing a request due to always returning False for is_closed.

>>> import httpx
>>> client = httpx.Client(http_versions=["HTTP/2"], pool_limits=httpx.PoolLimits(hard_limit=1))
>>> r = client.request("GET", "https://google.com")
19:20:50.480 - httpx.dispatch.connection_pool - acquire_connection origin=Origin(scheme='https' host='google.com' port=443)
19:20:50.480 - httpx.dispatch.connection_pool - new_connection connection=HTTPConnection(origin=Origin(scheme='https' host='google.com' port=443))
19:20:50.494 - httpx.dispatch.connection - start_connect host='google.com' port=443 timeout=TimeoutConfig(timeout=5.0)
19:20:50.559 - httpx.dispatch.connection - connected http_version='HTTP/2'
19:20:50.560 - httpx.dispatch.http2 - send_headers stream_id=1 method='GET' target='/' headers=[(b':method', b'GET'), (b':authority', b'google.com'), (b':scheme', b'https'), (b':path', b'/'), (b'user-agent', b'python-httpx/0.7.1'), (b'accept', b'*/*'), (b'accept-encoding', b'gzip, deflate, br'), (b'connection', b'keep-alive')]
19:20:50.562 - httpx.dispatch.http2 - end_stream stream_id=1
19:20:50.584 - httpx.dispatch.http2 - receive_event stream_id=0 event=<RemoteSettingsChanged changed_settings:{ChangedSetting(setting=SettingCodes.MAX_CONCURRENT_STREAMS, original_value=None, new_value=100), ChangedSetting(setting=SettingCodes.INITIAL_WINDOW_SIZE, original_value=65535, new_value=1048576), ChangedSetting(setting=SettingCodes.MAX_HEADER_LIST_SIZE, original_value=None, new_value=16384)}>
19:20:50.585 - httpx.dispatch.http2 - receive_event stream_id=0 event=<WindowUpdated stream_id:0, delta:983041>
19:20:50.585 - httpx.dispatch.http2 - receive_event stream_id=0 event=<SettingsAcknowledged changed_settings:{}>
19:20:50.618 - httpx.dispatch.http2 - receive_event stream_id=1 event=<ResponseReceived stream_id:1, headers:[(b':status', b'301'), (b'location', b'https://www.google.com/'), (b'content-type', b'text/html; charset=UTF-8'), (b'date', b'Wed, 28 Aug 2019 00:20:50 GMT'), (b'expires', b'Fri, 27 Sep 2019 00:20:50 GMT'), (b'cache-control', b'public, max-age=2592000'), (b'server', b'gws'), (b'content-length', b'220'), (b'x-xss-protection', b'0'), (b'x-frame-options', b'SAMEORIGIN'), (b'alt-svc', b'quic=":443"; ma=2592000; v="46,43,39"')]>
19:20:50.618 - httpx.dispatch.http2 - receive_event stream_id=1 event=<DataReceived stream_id:1, flow_controlled_length:220, data:3c48544d4c3e3c484541443e3c6d657461206874>
19:20:50.618 - httpx.dispatch.http2 - receive_event stream_id=1 event=<DataReceived stream_id:1, flow_controlled_length:0, data:>
19:20:50.619 - httpx.dispatch.http2 - receive_event stream_id=1 event=<StreamEnded stream_id:1>
19:20:50.619 - httpx.dispatch.http2 - receive_event stream_id=0 event=<PingReceived ping_data:0000000000000000>
19:20:50.622 - httpx.dispatch.connection_pool - release_connection connection=HTTPConnection(origin=Origin(scheme='https' host='google.com' port=443))
19:20:50.623 - httpx.dispatch.connection_pool - acquire_connection origin=Origin(scheme='https' host='www.google.com' port=443)
# ... deadlock

Activity

  1. added
    bugSomething isn't working
    http/2Issues and PRs related to HTTP/2
    on Aug 28, 2019
  2. florimondmanca commented on Aug 29, 2019

    @florimondmanca
    Contributor

    Just a question here, probably more related to how HTTP/2 works: how come the client is trying to acquire another connection in the first place, although it seems we have received the response? (Or maybe we've just sent the request data?)

  3. sethmlarson commented on Aug 29, 2019

    @sethmlarson
    ContributorAuthor

    Different origin: google.com vs www.google.com

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workinghttp/2Issues and PRs related to HTTP/2

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions