ssl.SSLError: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:852)

Hi guys by any chance any of you have seen this issue when using httpx + trio, if so how did you manage to fix it?
This is happening when starting a request to an IP on port 80 that then redirects to 443.

Traceback (most recent call last):
  File "/home/project/lib/python3.6/site-packages/trio/_ssl.py", line 466, in _retry
    ret = fn(*args)
  File "/usr/lib/python3.6/ssl.py", line 689, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:852)

Not seems that, no.
Does it reproduce on asyncio too?
Does it reproduce if connecting to https directly, rather than the redirect you mention?
Is the URL public, so that we could reproduce the issue?

@freis

@freis Is it the entire traceback? The more details we get on what happened exactly, the easier it is to debug. :-)

Also, you can run your program with trace mode on and report back the results here (there will be many details, but that's okay):

$ HTTPX_LOG_LEVEL=trace python program.py

Also, it might just be that the error you're encountering is just what it says: the server is trying to use an SSL/TLS version we don't support.

These are the options set on the SSLContext by HTTPX:

context = ssl.SSLContext(ssl.PROTOCOL_TLS)
context.options |= ssl.OP_NO_SSLv2
context.options |= ssl.OP_NO_SSLv3
context.options |= ssl.OP_NO_TLSv1
context.options |= ssl.OP_NO_TLSv1_1
context.options |= ssl.OP_NO_COMPRESSION
context.set_ciphers(DEFAULT_CIPHERS)

Related: #155

@tomchristie

Hello all thanks for your responses.

@tomchristie didn't test it on asyncio, also this doesn't happen when connecting to https directly.

@florimondmanca here is the log trace:
When connecting with port 80

TRACE [2019-12-18 11:26:41] httpx.dispatch.connection_pool - acquire_connection origin=Origin(scheme='https' host='109.169.53.101' port=80)
TRACE [2019-12-18 11:26:41] httpx.dispatch.connection_pool - new_connection connection=HTTPConnection(origin=Origin(scheme='https' host='109.169.53.101' port=80))
TRACE [2019-12-18 11:26:41] httpx.config - load_ssl_context verify=False cert=None trust_env=True http2=True
TRACE [2019-12-18 11:26:41] httpx.dispatch.connection - start_connect tcp host='109.169.53.101' port=80 timeout=Timeout(timeout=40)
TRACE [2019-12-18 11:26:41] httpx.dispatch.connection_pool - acquire_connection origin=Origin(scheme='http' host='109.169.53.101' port=80)
TRACE [2019-12-18 11:26:41] httpx.dispatch.connection_pool - new_connection connection=HTTPConnection(origin=Origin(scheme='http' host='109.169.53.101' port=80))
TRACE [2019-12-18 11:26:41] httpx.dispatch.connection - start_connect tcp host='109.169.53.101' port=80 timeout=Timeout(timeout=40)
TRACE [2019-12-18 11:26:41] httpx.dispatch.connection - connected http_version='HTTP/1.1'
TRACE [2019-12-18 11:26:41] httpx.dispatch.http11 - send_headers method='GET' target='/' headers=Headers({'host': '109.169.53.101:80', 'accept': '*/*', 'accept-encoding': 'gzip, deflate', 'connection': 'keep-alive', 'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36'})
TRACE [2019-12-18 11:26:41] httpx.dispatch.http11 - receive_event event=NEED_DATA
TRACE [2019-12-18 11:26:41] httpx.dispatch.http11 - receive_event event=Response(status_code=301, headers=[(b'date', b'Wed, 18 Dec 2019 11:26:41 GMT'), (b'server', b'Apache/2.4.29'), (b'location', b'https://109.169.53.101:80/'), (b'content-length', b'234'), (b'keep-alive', b'timeout=5, max=100'), (b'connection', b'Keep-Alive'), (b'content-type', b'text/html; charset=iso-8859-1')], http_version=b'1.1', reason=b'Moved Permanently')
DEBUG [2019-12-18 11:26:41] httpx.client - HTTP Request: GET http://109.169.53.101:80 "HTTP/1.1 301 Moved Permanently"
TRACE [2019-12-18 11:26:41] httpx.dispatch.http11 - receive_event event=Data(<234 bytes>)
TRACE [2019-12-18 11:26:41] httpx.dispatch.http11 - receive_event event=EndOfMessage(headers=[])
TRACE [2019-12-18 11:26:41] httpx.dispatch.http11 - response_closed our_state=DONE their_state=DONE
TRACE [2019-12-18 11:26:41] httpx.dispatch.connection_pool - release_connection connection=HTTPConnection(origin=Origin(scheme='http' host='109.169.53.101' port=80))
TRACE [2019-12-18 11:26:41] httpx.dispatch.connection_pool - acquire_connection origin=Origin(scheme='https' host='109.169.53.101' port=80)
TRACE [2019-12-18 11:26:41] httpx.dispatch.connection_pool - new_connection connection=HTTPConnection(origin=Origin(scheme='https' host='109.169.53.101' port=80))
TRACE [2019-12-18 11:26:41] httpx.config - load_ssl_context verify=False cert=None trust_env=True http2=True
TRACE [2019-12-18 11:26:41] httpx.dispatch.connection - start_connect tcp host='109.169.53.101' port=80 timeout=Timeout(timeout=40)
TRACE [2019-12-18 11:26:41] httpx.dispatch.connection - close_connection
TRACE [2019-12-18 11:26:41] httpx.dispatch.http11 - send_event event=ConnectionClosed()

When connecting to port 443

TRACE [2019-12-18 11:29:04] httpx.dispatch.connection_pool - acquire_connection origin=Origin(scheme='https' host='109.169.53.101' port=443)
TRACE [2019-12-18 11:29:04] httpx.dispatch.connection_pool - new_connection connection=HTTPConnection(origin=Origin(scheme='https' host='109.169.53.101' port=443))
TRACE [2019-12-18 11:29:04] httpx.config - load_ssl_context verify=False cert=None trust_env=True http2=True
TRACE [2019-12-18 11:29:04] httpx.dispatch.connection - start_connect tcp host='109.169.53.101' port=443 timeout=Timeout(timeout=40)
TRACE [2019-12-18 11:29:04] httpx.dispatch.connection - connected http_version='HTTP/2'
TRACE [2019-12-18 11:29:04] httpx.dispatch.http2 - send_headers stream_id=1 method='GET' target='/' headers=[(b':method', b'GET'), (b':authority', b'109.169.53.101:443'), (b':scheme', b'https'), (b':path', b'/'), (b'accept', b'*/*'), (b'accept-encoding', b'gzip, deflate'), (b'connection', b'keep-alive'), (b'user-agent', b'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36')]
TRACE [2019-12-18 11:29:04] httpx.dispatch.http2 - send_body stream_id=1
TRACE [2019-12-18 11:29:04] httpx.dispatch.http2 - receive_event stream_id=0 event=<RemoteSettingsChanged changed_settings:{ChangedSetting(setting=SettingCodes.MAX_CONCURRENT_STREAMS, original_value=None, new_value=100)}>
TRACE [2019-12-18 11:29:04] httpx.dispatch.http2 - receive_event stream_id=0 event=<WindowUpdated stream_id:0, delta:2147418112>
TRACE [2019-12-18 11:29:04] httpx.dispatch.http2 - receive_event stream_id=0 event=<SettingsAcknowledged changed_settings:{}>
TRACE [2019-12-18 11:29:04] httpx.dispatch.http2 - receive_event stream_id=1 event=<ResponseReceived stream_id:1, headers:[(b':status', b'301'), (b'date', b'Wed, 18 Dec 2019 11:29:04 GMT'), (b'server', b'Apache/2.4.29'), (b'location', b'https://www.109.169.53.101:443/'), (b'content-length', b'239'), (b'content-type', b'text/html; charset=iso-8859-1')]>
TRACE [2019-12-18 11:29:04] httpx.dispatch.http2 - receive_event stream_id=1 event=<DataReceived stream_id:1, flow_controlled_length:239, data:3c21444f43545950452048544d4c205055424c49>
TRACE [2019-12-18 11:29:04] httpx.dispatch.http2 - receive_event stream_id=1 event=<StreamEnded stream_id:1>
DEBUG [2019-12-18 11:29:04] httpx.client - HTTP Request: GET https://109.169.53.101:443 "HTTP/2 301 Moved Permanently"
TRACE [2019-12-18 11:29:04] httpx.dispatch.connection_pool - release_connection connection=HTTPConnection(origin=Origin(scheme='https' host='109.169.53.101' port=443))
TRACE [2019-12-18 11:29:04] httpx.dispatch.connection_pool - acquire_connection origin=Origin(scheme='https' host='www.109.169.53.101' port=443)
TRACE [2019-12-18 11:29:04] httpx.dispatch.connection_pool - new_connection connection=HTTPConnection(origin=Origin(scheme='https' host='www.109.169.53.101' port=443))
TRACE [2019-12-18 11:29:04] httpx.config - load_ssl_context verify=False cert=None trust_env=True http2=True
TRACE [2019-12-18 11:29:04] httpx.dispatch.connection - start_connect tcp host='www.109.169.53.101' port=443 timeout=Timeout(timeout=40)
TRACE [2019-12-18 11:29:04] httpx.dispatch.connection_pool - acquire_connection origin=Origin(scheme='http' host='109.169.53.101' port=443)
TRACE [2019-12-18 11:29:04] httpx.dispatch.connection_pool - new_connection connection=HTTPConnection(origin=Origin(scheme='http' host='109.169.53.101' port=443))
TRACE [2019-12-18 11:29:04] httpx.dispatch.connection - start_connect tcp host='109.169.53.101' port=443 timeout=Timeout(timeout=40)
TRACE [2019-12-18 11:29:04] httpx.dispatch.connection - connected http_version='HTTP/1.1'
TRACE [2019-12-18 11:29:04] httpx.dispatch.http11 - send_headers method='GET' target='/' headers=Headers({'host': '109.169.53.101:443', 'accept': '*/*', 'accept-encoding': 'gzip, deflate', 'connection': 'keep-alive', 'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36'})
TRACE [2019-12-18 11:29:04] httpx.dispatch.http11 - receive_event event=NEED_DATA
TRACE [2019-12-18 11:29:04] httpx.dispatch.http11 - receive_event event=Response(status_code=400, headers=[(b'date', b'Wed, 18 Dec 2019 11:29:04 GMT'), (b'server', b'Apache/2.4.29'), (b'content-length', b'362'), (b'connection', b'close'), (b'content-type', b'text/html; charset=iso-8859-1')], http_version=b'1.1', reason=b'Bad Request')
DEBUG [2019-12-18 11:29:04] httpx.client - HTTP Request: GET http://109.169.53.101:443 "HTTP/1.1 400 Bad Request"
TRACE [2019-12-18 11:29:04] httpx.dispatch.http11 - receive_event event=Data(<362 bytes>)
TRACE [2019-12-18 11:29:04] httpx.dispatch.http11 - receive_event event=EndOfMessage(headers=[])
TRACE [2019-12-18 11:29:04] httpx.dispatch.http11 - response_closed our_state=MUST_CLOSE their_state=MUST_CLOSE
TRACE [2019-12-18 11:29:04] httpx.dispatch.http11 - send_event event=ConnectionClosed()
TRACE [2019-12-18 11:29:04] httpx.dispatch.connection_pool - release_connection connection=HTTPConnection(origin=Origin(scheme='http' host='109.169.53.101' port=443))
HTTP/1.1
TRACE [2019-12-18 11:29:04] httpx.dispatch.connection - close_connection

On the connection with 443 he decided to use http2 at the beggining because the server also has it, but it can be done without it also.

Trying to access https://109.169.53.101/ in the browser shows an insecure connection page (on iOS Safari) — are certificates up to date there? Besides, inspecting the certificate in my browser shows « Version: 3 » — I assume that means SSLv3? If so, then the error is probably due to HTTPX only supporting non-deprecated certificates (TLS v1.2 and up). (And it might be that iOS Safari uses the same rule, ie dropped support for SSLv3.)

At this point I’d suggest verifying the SSL/TLS version used by the server certificates.

Shoudn't verify=False bypass any certificates issues? I inspected the IP with testssl.sh and SSLv3 is not being served, only TLSv1.2 and H2

 SSLv2      not offered (OK)
 SSLv3      not offered (OK)
 TLS 1      not offered
 TLS 1.1    not offered
 TLS 1.2    offered (OK)
 TLS 1.3    not offered and downgraded to a weaker protocol
 NPN/SPDY   not offered
 ALPN/HTTP2 h2, http/1.1 (offered)

And these are the ciphers ofered:

 PFS is offered (OK)          ECDHE-RSA-AES256-GCM-SHA384 ECDHE-RSA-AES256-SHA384 ECDHE-RSA-AES256-SHA DHE-RSA-AES256-GCM-SHA384 ECDHE-RSA-CHACHA20-POLY1305 DHE-RSA-CHACHA20-POLY1305 DHE-RSA-AES256-CCM8 
                              DHE-RSA-AES256-CCM DHE-RSA-AES256-SHA256 DHE-RSA-AES256-SHA ECDHE-RSA-CAMELLIA256-SHA384 DHE-RSA-CAMELLIA256-SHA256 DHE-RSA-CAMELLIA256-SHA DHE-RSA-ARIA256-GCM-SHA384 
                              ECDHE-ARIA256-GCM-SHA384 ECDHE-RSA-AES128-GCM-SHA256 ECDHE-RSA-AES128-SHA256 ECDHE-RSA-AES128-SHA DHE-RSA-AES128-GCM-SHA256 DHE-RSA-AES128-CCM8 DHE-RSA-AES128-CCM 
                              DHE-RSA-AES128-SHA256 DHE-RSA-AES128-SHA ECDHE-RSA-CAMELLIA128-SHA256 DHE-RSA-CAMELLIA128-SHA256 DHE-RSA-CAMELLIA128-SHA DHE-RSA-ARIA128-GCM-SHA256 ECDHE-ARIA128-GCM-SHA256

The error doesn't exists if I simple try to connect directly to 443, only. So I would think that would be a redirect problem? Not passing context or something?

Also in what way would I be able to change the context in order to support SSLv2 and SSLv3 by default? I don't see any mention on how to do it in the documentation.

@freis

@freis I was able to reproduce the issue using these sample commands (in an IPython shell/asyncio):

>>> import httpx
>>> url = 'https://109.169.53.101:80'
>>> r = await httpx.get(url, verify=False)

Full traceback + TRACE logs.

TRACE [2019-12-18 13:27:50] httpx.dispatch.connection_pool - acquire_connection origin=Origin(scheme='https' host='109.169.53.101' port=80)
TRACE [2019-12-18 13:27:50] httpx.dispatch.connection_pool - new_connection connection=HTTPConnection(origin=Origin(scheme='https' host='109.169.53.101' port=80))
TRACE [2019-12-18 13:27:50] httpx.config - load_ssl_context verify=False cert=None trust_env=True http2=False
TRACE [2019-12-18 13:27:50] httpx.dispatch.connection - start_connect tcp host='109.169.53.101' port=80 timeout=Timeout(timeout=5.0)
---------------------------------------------------------------------------
SSLError                                  Traceback (most recent call last)
<ipython-input-7-d8fec8261382> in <module>
----> 1 r = await httpx.get(url, verify=False)

~/.pyenv/versions/3.8.0/lib/python3.8/site-packages/httpx/api.py in get(url, params, headers, cookies, stream, auth, allow_redirects, cert, verify, timeout, trust_env)
    160     this function, as `GET` requests should not include a request body.
    161     """
--> 162     return await request(
    163         "GET",
    164         url,

~/.pyenv/versions/3.8.0/lib/python3.8/site-packages/httpx/api.py in request(method, url, params, data, files, json, headers, cookies, auth, timeout, allow_redirects, verify, cert, stream, trust_env)
     85         cert=cert, verify=verify, timeout=timeout, trust_env=trust_env,
     86     ) as client:
---> 87         return await client.request(
     88             method=method,
     89             url=url,

~/.pyenv/versions/3.8.0/lib/python3.8/site-packages/httpx/client.py in request(self, method, url, data, files, json, params, headers, cookies, stream, auth, allow_redirects, cert, verify, timeout, trust_env)
    258             cookies=cookies,
    259         )
--> 260         response = await self.send(
    261             request,
    262             stream=stream,

~/.pyenv/versions/3.8.0/lib/python3.8/site-packages/httpx/client.py in send(self, request, stream, auth, allow_redirects, verify, cert, timeout, trust_env)
    403         if not isinstance(auth, Middleware):
    404             request = self.authenticate(request, trust_env, auth)
--> 405             response = await self.send_handling_redirects(
    406                 request,
    407                 verify=verify,

~/.pyenv/versions/3.8.0/lib/python3.8/site-packages/httpx/client.py in send_handling_redirects(self, request, timeout, verify, cert, allow_redirects, history)
    467                 raise RedirectLoop()
    468
--> 469             response = await self.send_single_request(
    470                 request, verify=verify, cert=cert, timeout=timeout
    471             )

~/.pyenv/versions/3.8.0/lib/python3.8/site-packages/httpx/client.py in send_single_request(self, request, timeout, verify, cert)
    596         try:
    597             with ElapsedTimer() as timer:
--> 598                 response = await dispatcher.send(
    599                     request, verify=verify, cert=cert, timeout=timeout
    600                 )

~/.pyenv/versions/3.8.0/lib/python3.8/site-packages/httpx/dispatch/connection_pool.py in send(self, request, verify, cert, timeout)
    155             self.active_connections.remove(connection)
    156             self.max_connections.release()
--> 157             raise exc
    158
    159         return response

~/.pyenv/versions/3.8.0/lib/python3.8/site-packages/httpx/dispatch/connection_pool.py in send(self, request, verify, cert, timeout)
    149         )
    150         try:
--> 151             response = await connection.send(
    152                 request, verify=verify, cert=cert, timeout=timeout
    153             )

~/.pyenv/versions/3.8.0/lib/python3.8/site-packages/httpx/dispatch/connection.py in send(self, request, verify, cert, timeout)
     51
     52         if self.open_connection is None:
---> 53             await self.connect(verify=verify, cert=cert, timeout=timeout)
     54
     55         assert self.open_connection is not None

~/.pyenv/versions/3.8.0/lib/python3.8/site-packages/httpx/dispatch/connection.py in connect(self, timeout, verify, cert)
     76                 f"start_connect tcp host={host!r} port={port!r} timeout={timeout!r}"
     77             )
---> 78             stream = await self.backend.open_tcp_stream(
     79                 host, port, ssl_context, timeout
     80             )

~/.pyenv/versions/3.8.0/lib/python3.8/site-packages/httpx/concurrency/auto.py in open_tcp_stream(self, hostname, port, ssl_context, timeout)
     31         timeout: Timeout,
     32     ) -> BaseSocketStream:
---> 33         return await self.backend.open_tcp_stream(hostname, port, ssl_context, timeout)
     34
     35     async def open_uds_stream(

~/.pyenv/versions/3.8.0/lib/python3.8/site-packages/httpx/concurrency/asyncio.py in open_tcp_stream(self, hostname, port, ssl_context, timeout)
    212     ) -> SocketStream:
    213         try:
--> 214             stream_reader, stream_writer = await asyncio.wait_for(  # type: ignore
    215                 asyncio.open_connection(hostname, port, ssl=ssl_context),
    216                 timeout.connect_timeout,

~/.pyenv/versions/3.8.0/lib/python3.8/asyncio/tasks.py in wait_for(fut, timeout, loop)
    481
    482         if fut.done():
--> 483             return fut.result()
    484         else:
    485             fut.remove_done_callback(cb)

~/.pyenv/versions/3.8.0/lib/python3.8/asyncio/streams.py in open_connection(host, port, loop, limit, **kwds)
     50     reader = StreamReader(limit=limit, loop=loop)
     51     protocol = StreamReaderProtocol(reader, loop=loop)
---> 52     transport, _ = await loop.create_connection(
     53         lambda: protocol, host, port, **kwds)
     54     writer = StreamWriter(transport, protocol, reader, loop)

~/.pyenv/versions/3.8.0/lib/python3.8/asyncio/base_events.py in create_connection(self, protocol_factory, host, port, ssl, family, proto, flags, sock, local_addr, server_hostname, ssl_handshake_timeout, happy_eyeballs_delay, interleave)
   1040                     f'A Stream Socket was expected, got {sock!r}')
   1041
-> 1042         transport, protocol = await self._create_connection_transport(
   1043             sock, protocol_factory, ssl, server_hostname,
   1044             ssl_handshake_timeout=ssl_handshake_timeout)

~/.pyenv/versions/3.8.0/lib/python3.8/asyncio/base_events.py in _create_connection_transport(self, sock, protocol_factory, ssl, server_hostname, server_side, ssl_handshake_timeout)
   1070
   1071         try:
-> 1072             await waiter
   1073         except:
   1074             transport.close()

~/.pyenv/versions/3.8.0/lib/python3.8/asyncio/sslproto.py in data_received(self, data)
    527
    528         try:
--> 529             ssldata, appdata = self._sslpipe.feed_ssldata(data)
    530         except (SystemExit, KeyboardInterrupt):
    531             raise

~/.pyenv/versions/3.8.0/lib/python3.8/asyncio/sslproto.py in feed_ssldata(self, data, only_handshake)
    187             if self._state == _DO_HANDSHAKE:
    188                 # Call do_handshake() until it doesn't raise anymore.
--> 189                 self._sslobj.do_handshake()
    190                 self._state = _WRAPPED
    191                 if self._handshake_cb:

~/.pyenv/versions/3.8.0/lib/python3.8/ssl.py in do_handshake(self)
    942     def do_handshake(self):
    943         """Start the SSL/TLS handshake."""
--> 944         self._sslobj.do_handshake()
    945
    946     def unwrap(self):

SSLError: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1108)

(You'll notice that it fails on the first attempt to connect — doesn't get a chance to get a response and redirect.)

If I omit :80 (or pass :443), I get a different error: gaierror: [Errno 8] nodename nor servname provided, or not known.

Full traceback and TRACE logs.

TRACE [2019-12-18 13:32:25] httpx.dispatch.connection_pool - acquire_connection origin=Origin(scheme='https' host='109.169.53.101' port=443)
TRACE [2019-12-18 13:32:25] httpx.dispatch.connection_pool - new_connection connection=HTTPConnection(origin=Origin(scheme='https' host='109.169.53.101' port=443))
TRACE [2019-12-18 13:32:25] httpx.config - load_ssl_context verify=False cert=None trust_env=True http2=False
TRACE [2019-12-18 13:32:25] httpx.dispatch.connection - start_connect tcp host='109.169.53.101' port=443 timeout=Timeout(timeout=5.0)
TRACE [2019-12-18 13:32:25] httpx.dispatch.connection - connected http_version='HTTP/1.1'
TRACE [2019-12-18 13:32:25] httpx.dispatch.http11 - send_headers method='GET' target='/' headers=Headers({'host': '109.169.53.101', 'user-agent': 'python-httpx/0.9.4', 'accept': '*/*', 'accept-encoding': 'gzip, deflate', 'connection': 'keep-alive'})
TRACE [2019-12-18 13:32:25] httpx.dispatch.http11 - receive_event event=NEED_DATA
TRACE [2019-12-18 13:32:25] httpx.dispatch.http11 - receive_event event=Response(status_code=301, headers=[(b'date', b'Wed, 18 Dec 2019 12:32:25 GMT'), (b'server', b'Apache/2.4.29'), (b'location', b'https://www.109.169.53.101/'), (b'content-length', b'235'), (b'keep-alive', b'timeout=5, max=100'), (b'connection', b'Keep-Alive'), (b'content-type', b'text/html; charset=iso-8859-1')], http_version=b'1.1', reason=b'Moved Permanently')
DEBUG [2019-12-18 13:32:25] httpx.client - HTTP Request: GET https://109.169.53.101 "HTTP/1.1 301 Moved Permanently"
TRACE [2019-12-18 13:32:25] httpx.dispatch.http11 - receive_event event=Data(<235 bytes>)
TRACE [2019-12-18 13:32:25] httpx.dispatch.http11 - receive_event event=EndOfMessage(headers=[])
TRACE [2019-12-18 13:32:25] httpx.dispatch.http11 - response_closed our_state=DONE their_state=DONE
TRACE [2019-12-18 13:32:25] httpx.dispatch.connection_pool - release_connection connection=HTTPConnection(origin=Origin(scheme='https' host='109.169.53.101' port=443))
TRACE [2019-12-18 13:32:25] httpx.dispatch.connection_pool - acquire_connection origin=Origin(scheme='https' host='www.109.169.53.101' port=443)
TRACE [2019-12-18 13:32:25] httpx.dispatch.connection_pool - new_connection connection=HTTPConnection(origin=Origin(scheme='https' host='www.109.169.53.101' port=443))
TRACE [2019-12-18 13:32:25] httpx.config - load_ssl_context verify=False cert=None trust_env=True http2=False
TRACE [2019-12-18 13:32:25] httpx.dispatch.connection - start_connect tcp host='www.109.169.53.101' port=443 timeout=Timeout(timeout=5.0)
TRACE [2019-12-18 13:32:25] httpx.dispatch.connection - close_connection
TRACE [2019-12-18 13:32:25] httpx.dispatch.http11 - send_event event=ConnectionClosed()
---------------------------------------------------------------------------
gaierror                                  Traceback (most recent call last)
<ipython-input-11-d8fec8261382> in <module>
----> 1 r = await httpx.get(url, verify=False)

~/.pyenv/versions/3.8.0/lib/python3.8/site-packages/httpx/api.py in get(url, params, headers, cookies, stream, auth, allow_redirects, cert, verify, timeout, trust_env)
    160     this function, as `GET` requests should not include a request body.
    161     """
--> 162     return await request(
    163         "GET",
    164         url,

~/.pyenv/versions/3.8.0/lib/python3.8/site-packages/httpx/api.py in request(method, url, params, data, files, json, headers, cookies, auth, timeout, allow_redirects, verify, cert, stream, trust_env)
     85         cert=cert, verify=verify, timeout=timeout, trust_env=trust_env,
     86     ) as client:
---> 87         return await client.request(
     88             method=method,
     89             url=url,

~/.pyenv/versions/3.8.0/lib/python3.8/site-packages/httpx/client.py in request(self, method, url, data, files, json, params, headers, cookies, stream, auth, allow_redirects, cert, verify, timeout, trust_env)
    258             cookies=cookies,
    259         )
--> 260         response = await self.send(
    261             request,
    262             stream=stream,

~/.pyenv/versions/3.8.0/lib/python3.8/site-packages/httpx/client.py in send(self, request, stream, auth, allow_redirects, verify, cert, timeout, trust_env)
    403         if not isinstance(auth, Middleware):
    404             request = self.authenticate(request, trust_env, auth)
--> 405             response = await self.send_handling_redirects(
    406                 request,
    407                 verify=verify,

~/.pyenv/versions/3.8.0/lib/python3.8/site-packages/httpx/client.py in send_handling_redirects(self, request, timeout, verify, cert, allow_redirects, history)
    467                 raise RedirectLoop()
    468
--> 469             response = await self.send_single_request(
    470                 request, verify=verify, cert=cert, timeout=timeout
    471             )

~/.pyenv/versions/3.8.0/lib/python3.8/site-packages/httpx/client.py in send_single_request(self, request, timeout, verify, cert)
    596         try:
    597             with ElapsedTimer() as timer:
--> 598                 response = await dispatcher.send(
    599                     request, verify=verify, cert=cert, timeout=timeout
    600                 )

~/.pyenv/versions/3.8.0/lib/python3.8/site-packages/httpx/dispatch/connection_pool.py in send(self, request, verify, cert, timeout)
    155             self.active_connections.remove(connection)
    156             self.max_connections.release()
--> 157             raise exc
    158
    159         return response

~/.pyenv/versions/3.8.0/lib/python3.8/site-packages/httpx/dispatch/connection_pool.py in send(self, request, verify, cert, timeout)
    149         )
    150         try:
--> 151             response = await connection.send(
    152                 request, verify=verify, cert=cert, timeout=timeout
    153             )

~/.pyenv/versions/3.8.0/lib/python3.8/site-packages/httpx/dispatch/connection.py in send(self, request, verify, cert, timeout)
     51
     52         if self.open_connection is None:
---> 53             await self.connect(verify=verify, cert=cert, timeout=timeout)
     54
     55         assert self.open_connection is not None

~/.pyenv/versions/3.8.0/lib/python3.8/site-packages/httpx/dispatch/connection.py in connect(self, timeout, verify, cert)
     76                 f"start_connect tcp host={host!r} port={port!r} timeout={timeout!r}"
     77             )
---> 78             stream = await self.backend.open_tcp_stream(
     79                 host, port, ssl_context, timeout
     80             )

~/.pyenv/versions/3.8.0/lib/python3.8/site-packages/httpx/concurrency/auto.py in open_tcp_stream(self, hostname, port, ssl_context, timeout)
     31         timeout: Timeout,
     32     ) -> BaseSocketStream:
---> 33         return await self.backend.open_tcp_stream(hostname, port, ssl_context, timeout)
     34
     35     async def open_uds_stream(

~/.pyenv/versions/3.8.0/lib/python3.8/site-packages/httpx/concurrency/asyncio.py in open_tcp_stream(self, hostname, port, ssl_context, timeout)
    212     ) -> SocketStream:
    213         try:
--> 214             stream_reader, stream_writer = await asyncio.wait_for(  # type: ignore
    215                 asyncio.open_connection(hostname, port, ssl=ssl_context),
    216                 timeout.connect_timeout,

~/.pyenv/versions/3.8.0/lib/python3.8/asyncio/tasks.py in wait_for(fut, timeout, loop)
    481
    482         if fut.done():
--> 483             return fut.result()
    484         else:
    485             fut.remove_done_callback(cb)

~/.pyenv/versions/3.8.0/lib/python3.8/asyncio/streams.py in open_connection(host, port, loop, limit, **kwds)
     50     reader = StreamReader(limit=limit, loop=loop)
     51     protocol = StreamReaderProtocol(reader, loop=loop)
---> 52     transport, _ = await loop.create_connection(
     53         lambda: protocol, host, port, **kwds)
     54     writer = StreamWriter(transport, protocol, reader, loop)

~/.pyenv/versions/3.8.0/lib/python3.8/asyncio/base_events.py in create_connection(self, protocol_factory, host, port, ssl, family, proto, flags, sock, local_addr, server_hostname, ssl_handshake_timeout, happy_eyeballs_delay, interleave)
    976                     'host/port and sock can not be specified at the same time')
    977
--> 978             infos = await self._ensure_resolved(
    979                 (host, port), family=family,
    980                 type=socket.SOCK_STREAM, proto=proto, flags=flags, loop=self)

~/.pyenv/versions/3.8.0/lib/python3.8/asyncio/base_events.py in _ensure_resolved(self, address, family, type, proto, flags, loop)
   1348             return [info]
   1349         else:
-> 1350             return await loop.getaddrinfo(host, port, family=family, type=type,
   1351                                           proto=proto, flags=flags)
   1352

~/.pyenv/versions/3.8.0/lib/python3.8/asyncio/base_events.py in getaddrinfo(self, host, port, family, type, proto, flags)
    815             getaddr_func = socket.getaddrinfo
    816
--> 817         return await self.run_in_executor(
    818             None, getaddr_func, host, port, family, type, proto, flags)
    819

~/.pyenv/versions/3.8.0/lib/python3.8/concurrent/futures/thread.py in run(self)
     55
     56         try:
---> 57             result = self.fn(*self.args, **self.kwargs)
     58         except BaseException as exc:
     59             self.future.set_exception(exc)

~/.pyenv/versions/3.8.0/lib/python3.8/socket.py in getaddrinfo(host, port, family, type, proto, flags)
    912     # and socket type values to enum constants.
    913     addrlist = []
--> 914     for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
    915         af, socktype, proto, canonname, sa = res
    916         addrlist.append((_intenum_converter(af, AddressFamily),

gaierror: [Errno 8] nodename nor servname provided, or not known

I assume that's because said IP is located on an internal network I can't access here?

My best guess is — are you supposed to make an HTTPS request on port 80 anyway? Aren't those supposed to be issued on port 443? Can't you switch to making the request on port 443 in the first place, in order to not hit the HTTP:80 => HTTPS:443 redirection?

This leads me to think — it might be possible that HTTPX does not honor changing the port when it receives a new redirection. We'd need to check that.

it might be possible that HTTPX does not honor changing the port when it receives a new redirection

So, this line in the TRACE logs seems to indicate that HTTPX is issuing the post-redirect HTTPS request on port 80:

TRACE [2019-12-18 11:26:41] httpx.dispatch.connection_pool - new_connection connection=HTTPConnection(origin=Origin(scheme='https' host='109.169.53.101' port=80))

And the reason why that is is because the server included :80 in the Location, i.e. https://109.169.53.101:80/, as can be seen in this log line:

TRACE [2019-12-18 11:26:41] httpx.dispatch.http11 - receive_event event=Response(status_code=301, headers=[(b'date', b'Wed, 18 Dec 2019 11:26:41 GMT'), (b'server', b'Apache/2.4.29'), (b'location', b'https://109.169.53.101:80/'), (b'content-length', b'234'), (b'keep-alive', b'timeout=5, max=100'), (b'connection', b'Keep-Alive'), (b'content-type', b'text/html; charset=iso-8859-1')], http_version=b'1.1', reason=b'Moved Permanently')

So it seems to me that HTTPX is doing what it's told, and it's an issue with the server — it shouldn't include :80 in the HTTPS redirect URL.

As far as I see via curl when the redirect is issued the port isn't there. By any chance httpx or even h11 is forcing the chosen port into the redirect?

$ curl -vLk "http://109.169.53.101:80"
* Rebuilt URL to: http://109.169.53.101:80/
*   Trying 109.169.53.101...
* TCP_NODELAY set
* Connected to 109.169.53.101 (109.169.53.101) port 80 (#0)
> GET / HTTP/1.1
> Host: 109.169.53.101
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 301 Moved Permanently
< Date: Wed, 18 Dec 2019 14:56:01 GMT
< Server: Apache/2.4.29
< Location: https://109.169.53.101/
< Content-Length: 231
< Content-Type: text/html; charset=iso-8859-1
<
* Ignoring the response-body
* Connection #0 to host 109.169.53.101 left intact
* Issue another request to this URL: 'https://109.169.53.101/'
*   Trying 109.169.53.101...
* TCP_NODELAY set
* Connected to 109.169.53.101 (109.169.53.101) port 443 (#1)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=cozybot.com
*  start date: Oct 20 02:46:07 2019 GMT
*  expire date: Jan 18 02:46:07 2020 GMT
*  issuer: C=US; O=Let's Encrypt; CN=Let's Encrypt Authority X3
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x55c581148580)
> GET / HTTP/2
> Host: 109.169.53.101
> User-Agent: curl/7.58.0
> Accept: */*
>
* Connection state changed (MAX_CONCURRENT_STREAMS updated)!
< HTTP/2 301
< date: Wed, 18 Dec 2019 14:56:01 GMT
< server: Apache/2.4.29
< location: https://www.109.169.53.101/
< content-length: 235
< content-type: text/html; charset=iso-8859-1
<
* Ignoring the response-body
* Connection #1 to host 109.169.53.101 left intact
* Issue another request to this URL: 'https://www.109.169.53.101/'
* Could not resolve host: www.109.169.53.101
* Closing connection 2
curl: (6) Could not resolve host: www.109.169.53.101

Also you can find this issue if you try to connect to facebook.com on port 80.
It seems that everytime there is a Location something is forcing the add of port 80 when it shoudn't.

Also if it helps what I'm doing is:

    async def worker(self, target, args, client, limit):
        async with limit:
            host = target['host']
            port = int(target['port'])
            url  = host + ':' + str(port)

            try:
                response = await client.get('https://' + url)
            except:
                try:
                    response = await client.get('http://' + url)
                except Exception as err:
                    return

    async with httpx.Client(headers=headers, http2=False, verify=False, timeout=40) as client:
            async with trio.open_nursery() as nursery:
                for target in targets:
                    nursery.start_soon(self.worker, target, args, client, limit)

It looks like the issue is that httpx is including the :80 in the outgoing Host: header on the first request, and then the server is mirroring that back when it tries to redirect to https. So technically httpx is doing nothing wrong and the server is buggy, but httpx is doing something unusual that tickles the bug. Instead, httpx should leave out the port on outgoing host headers iff it's the default port.

It's wierd saying that the server is buggy if the same happens when trying the same with facebook.com, right?

I've test it with curl and it is what you are saying is true, it happens when the host has the port there:

$ curl -vL --header 'Host: facebook.com:80' "http://facebook.com:80"
* Rebuilt URL to: http://facebook.com:80/
*   Trying 31.13.83.36...
* TCP_NODELAY set
* Connected to facebook.com (31.13.83.36) port 80 (#0)
> GET / HTTP/1.1
> Host: facebook.com:80
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 302 Found
< Location: https://facebook.com:80/
< Content-Type: text/html; charset="utf-8"
< X-FB-Debug: kPOS+1weEHxq7fynoo6EIdIl+LeFDijjwBq8O+fPoswnqAanj1aLnl29lvZ1EY6xmvUbJh1K98Xl4vYSvlZ79Q==
< Date: Wed, 18 Dec 2019 22:29:12 GMT
< Alt-Svc: h3-24=":443"; ma=3600
< Connection: keep-alive
< Content-Length: 0
<
* Connection #0 to host facebook.com left intact
* Issue another request to this URL: 'https://facebook.com:80/'
* Hostname facebook.com was found in DNS cache
*   Trying 31.13.83.36...
* TCP_NODELAY set
* Connected to facebook.com (31.13.83.36) port 80 (#1)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* error:1408F10B:SSL routines:ssl3_get_record:wrong version number
* stopped the pause stream!
* Closing connection 1
curl: (35) error:1408F10B:SSL routines:ssl3_get_record:wrong version number

Navigation Menu

Repository navigation

ssl.SSLError: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:852) #646

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

ssl.SSLError: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:852) #646

Description

Activity

lovelydinosaur commented on Dec 18, 2019

florimondmanca commented on Dec 18, 2019

florimondmanca commented on Dec 18, 2019

freis commented on Dec 18, 2019

florimondmanca commented on Dec 18, 2019

freis commented on Dec 18, 2019

florimondmanca commented on Dec 18, 2019

florimondmanca commented on Dec 18, 2019

freis commented on Dec 18, 2019

freis commented on Dec 18, 2019

njsmith commented on Dec 18, 2019

freis commented on Dec 18, 2019

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions