Repository navigation
AttributeError 'NoneType' object has no attribute 'fileno'. #239
Description
We somewhat discussed the issue at https://gitter.im/encode/community .
TL;DR when connection is closed/reset httpcore assumes socket is still available for transport and crashes with
AttributeError 'NoneType' object has no attribute 'fileno'.
Environment
Docker image: python:3.8.6-buster
Python: 3.8.6
httpcore: 0.12.2
Details
- The exact line of the code is here: https://github.com/encode/httpcore/blob/0.12.2/httpcore/_backends/asyncio.py#L181
def is_readable(self) -> bool:
transport = self.stream_reader._transport # type: ignore
sock: socket.socket = transport.get_extra_info("socket")
return is_socket_readable(sock.fileno())- If
transport.get_extra_info("socket")returnsNone,sock.fileno()crashes. - First they type of
sockshould probably beOptional[socket.socket]. - The python docs are not very detailed about the guarantees of
extra["socket"]existence but in the example code they do test it: https://docs.python.org/3/library/asyncio-protocol.html#asyncio.BaseTransport.get_extra_info - So as other code e.g. in uvicorn or aiohttp
- I wish I could provide a working example, but this only occasionally fails in my system.
- However I do have a stacktrace capture:
AttributeError 'NoneType' object has no attribute 'fileno'.
File "httpx/_client.py", line 1371, in request
response = await self.send(
File "httpx/_client.py", line 1406, in send
response = await self._send_handling_auth(
File "httpx/_client.py", line 1444, in _send_handling_auth
response = await self._send_handling_redirects(
File "httpx/_client.py", line 1476, in _send_handling_redirects
response = await self._send_single_request(request, timeout)
File "httpx/_client.py", line 1502, in _send_single_request
(status_code, headers, stream, ext,) = await transport.arequest(
File "httpcore/_async/http_proxy.py", line 124, in arequest
return await self._tunnel_request(
File "httpcore/_async/http_proxy.py", line 190, in _tunnel_request
connection = await self._get_connection_from_pool(origin)
File "httpcore/_async/connection_pool.py", line 248, in _get_connection_from_pool
if connection.is_socket_readable():
File "httpcore/_async/connection.py", line 169, in is_socket_readable
return self.connection is not None and self.connection.is_socket_readable()
File "httpcore/_async/http11.py", line 205, in is_socket_readable
return self.socket.is_readable()
File "httpcore/_backends/asyncio.py", line 179, in is_readable
return is_socket_readable(sock.fileno())
- Was going through asyncio implementation trying to locate where the
extra["socket"]gets cleared, but couldn't find anything related 🤔 For reference it is set here
Anyway, even though I do not fully understand asyncio transports, based on other examples I guess it is pretty reasonable to test if get_exra_info("socket") is not None?
Activity
florimondmanca commentedon Nov 24, 2020
I guess we should return
Trueif there's no socket retuned... ?povilasb commentedon Nov 24, 2020
I was going to say
False:)meaning
if socket not available, connection not readable🤔florimondmanca commentedon Nov 25, 2020
@povilasb Actually that's counter-intuitive, but "socket is readable" is actually a good proxy for "the connection was dropped". Think what happens when we do
sock.recv(): it blocks until the server sends some non-empty data (so it's not readable until there's data in the read buffer). When the server has disconnected and the connection is essentially broken, you'll see thatsock.recv()returns immediately withb""— the socket is readable at all times. So we're using readability as a proxy to detect broken connections; and if the transport has no socket, then we definitely should treat this as if the connection was broken, hence as if the socket (if it was there) was readable. Hope that makes sense. :)povilasb commentedon Nov 26, 2020
It does, thank you :)
Will make a PR tomorrow.