connection leaking in some network condition #1360

Activity

1. cdeler commented on Oct 14, 2020

   

   cdeler

   on Oct 14, 2020

   Contributor

   More actions

   Hello @everAwayne ,

   Yes, it's probably a bug in httpcore.

   I wrote a test case to make it reproducible. In the test case we try to access the domain, which is in our proxy blacklist.

   @pytest.mark.trio
   async def test_socket_does_not_leak_from_the_connection_has_not_been_added_to_pool(
       proxy_server: URL,
       server: Server,
   ):
       method = b"GET"
       url = (b"http", b"example.org", 80, b"/")
       headers = [(b"host", b"example.org")]
       proxy_mode = "TUNNEL_ONLY"
   
       with pytest.warns(ResourceWarning) as recorded_warnings:
           async with httpcore.AsyncHTTPProxy(proxy_server, proxy_mode=proxy_mode) as http:
               for _ in range(100):
                   try:
                       _ = await http.arequest(method, url, headers)
                   except httpcore.RemoteProtocolError:
                       pass
   
       assert len(recorded_warnings.list) == 0

   To make the test case effective we need to run pproxy with a "blocked domains" list:

   # test/utils.py changes
   @contextlib.contextmanager
   def http_proxy_server(proxy_host: str, proxy_port: int):
   
       proc = None
   
       with create_proxy_block_file(["example.org"]) as block_file_name:
           try:
               command = [
                   "pproxy",
                   "-b",
                   block_file_name,
                   "-l",
                   f"http://{proxy_host}:{proxy_port}/",
               ]
               proc = subprocess.Popen(command)
   
               _wait_can_connect(proxy_host, proxy_port)
   
               yield b"http", proxy_host.encode(), proxy_port, b"/"
           finally:
               if proc is not None:
                   proc.kill()
   
   
   @contextlib.contextmanager
   def create_proxy_block_file(blocked_domains: List[str]):
       with tempfile.NamedTemporaryFile(delete=True, mode="w+") as file:
   
           for domain in blocked_domains:
               file.write(domain)
               file.write("\n")
   
           file.flush()
   
           yield file.name
2. cdeler

   added

   bugSomething isn't workingSomething isn't working

   httpcoreIssues related to HTTPCore (core HTTP networking layer) - https://github.com/encode/httpcoreIssues related to HTTPCore (core HTTP networking layer) - https://github.com/encode/httpcore

   on Oct 14, 2020
3. cdeler

   closed this as completedin httpcore#223on Oct 30, 2020