event loop loop forever: make lots of sock_connec at the same time #378

Activity

1. Catstyle

   changed the title [-]event loop [/-] [+]event loop loop forever: make lots of sock_connec at the same time[/+] on Jan 9, 2021
2. fantix commented on Jan 21, 2021

   

   fantix

   on Jan 21, 2021

   Member

   More actions

   Hey, thanks for the report! But please provide a reproducible sample code. I've tried several batches of 10k concurrent connections on the same server but cannot reproduce the issue.
3. fantix

   added

   need more info

   on Jan 21, 2021
4. Catstyle commented on Jan 25, 2021

   

   Catstyle

   on Jan 25, 2021

   Author

   More actions

   finally got some time
   i wrote some naive codes, just to illustrate the issue

   server.py

   import asyncio
   import socket
   
   import uvloop
   uvloop.install()
   
   count = 0
   backlog = 1024
   
   
   async def connection_made(sock, loop):
       global count
       count += 1
       print(f'{loop.time()}: {count}, {sock} connected')
       await asyncio.sleep(1000)
   
   
   def accept(sock):
       loop = asyncio.get_running_loop()
       for _ in range(backlog):
           try:
               conn, addr = sock.accept()
           except (BlockingIOError, InterruptedError, ConnectionAbortedError):
               return
           conn.setblocking(False)
           asyncio.create_task(connection_made(conn, loop))
   
   
   async def main():
       sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
       sock.setblocking(False)
       sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, True)
       sock.bind(('localhost', 56789))
       sock.listen(backlog)
   
       loop = asyncio.get_running_loop()
       loop.add_reader(sock.fileno(), accept, sock)
   
       await asyncio.sleep(1000)
   
   
   if __name__ == '__main__':
       asyncio.run(main())

   client.py

   import asyncio
   import socket
   import sys
   
   import uvloop
   uvloop.install()
   
   count = 0
   
   
   async def connect(addr, loop):
       sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
       sock.setblocking(False)
       await loop.sock_connect(sock, addr)
       return sock
   
   
   async def wrapper(loop):
       while 1:
           try:
               sock = await connect(('localhost', 56789), loop)
           except BlockingIOError:
               await asyncio.sleep(0)
           else:
               break
   
       global count
       count += 1
       print(f'{loop.time()}: {count}, {sock} connected')
       await asyncio.sleep(1000)
   
   
   async def main(concurrency):
       tasks = []
       loop = asyncio.get_running_loop()
       for x in range(concurrency):
           tasks.append(asyncio.create_task(wrapper(loop)))
   
       await asyncio.gather(*tasks)
   
   
   if __name__ == '__main__':
       # python client.py 1000
       asyncio.run(main(int(sys.argv[1])))

   concurrency = 1000, works well

   267544.168: 996, <socket.socket fd=315, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 48500), raddr=('127.0.0.1', 56789)> connected
   267544.168: 997, <socket.socket fd=314, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 48498), raddr=('127.0.0.1', 56789)> connected
   267544.168: 998, <socket.socket fd=313, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 48496), raddr=('127.0.0.1', 56789)> connected
   267544.168: 999, <socket.socket fd=311, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 48494), raddr=('127.0.0.1', 56789)> connected
   267544.168: 1000, <socket.socket fd=312, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 48492), raddr=('127.0.0.1', 56789)> connected
   

   concurrency = 5000, oops

   Traceback (most recent call last):
     File "uvloop/cbhandles.pyx", line 82, in uvloop.loop.Handle._run
     File "uvloop/loop.pyx", line 1072, in uvloop.loop.Loop._sock_connect_cb
   asyncio.exceptions.InvalidStateError: invalid state
   Exception in callback Loop._sock_connect
   handle: <Handle Loop._sock_connect>
   Traceback (most recent call last):
     File "uvloop/cbhandles.pyx", line 82, in uvloop.loop.Handle._run
     File "uvloop/loop.pyx", line 1072, in uvloop.loop.Loop._sock_connect_cb
   asyncio.exceptions.InvalidStateError: invalid state
   Exception in callback Loop._sock_connect
   handle: <Handle Loop._sock_connect>
   Traceback (most recent call last):
     File "uvloop/cbhandles.pyx", line 82, in uvloop.loop.Handle._run
     File "uvloop/loop.pyx", line 1072, in uvloop.loop.Loop._sock_connect_cb
   asyncio.exceptions.InvalidStateError: invalid state
   Exception in callback Loop._sock_connect
   handle: <Handle Loop._sock_connect>
   Traceba^Z
   [1]+  Stopped                 python client.py 5000
   

   using vanilla asyncio, works well

   267761.563129303: 4996, <socket.socket fd=3588, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 38836), raddr=('127.0.0.1', 56789)> connected
   267761.563196003: 4997, <socket.socket fd=3592, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 38838), raddr=('127.0.0.1', 56789)> connected
   267761.563289903: 4998, <socket.socket fd=3598, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 38840), raddr=('127.0.0.1', 56789)> connected
   267761.563356603: 4999, <socket.socket fd=3604, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 38842), raddr=('127.0.0.1', 56789)> connected
   267761.563436503: 5000, <socket.socket fd=3602, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 38844), raddr=('127.0.0.1', 56789)> connected
   

   some kernel config
   net.core.netdev_max_backlog = 1000
   net.ipv4.tcp_max_syn_backlog = 512
5. fantix

   removed

   need more info

   on Jan 27, 2021
6. fantix

   added 4 commits that reference this issue on Feb 8, 2021

   Fix future issue in sock_connect()

   ...

   This issue will close once commit 1f8fdba is merged into the 'master' branch.

   1f8fdba

   Fix future issue in sock_connect()

   ...

   This issue will close once commit 0360dab is merged into the 'master' branch.

   0360dab

   Fix future issue in sock_connect()

   ...

   This issue will close once commit 4842175 is merged into the 'master' branch.

   4842175

   Fix future issue in sock_connect()

   ...

   This issue will close once commit acf8841 is merged into the 'master' branch.

   acf8841
7. fantix

   closed this as completedin #391on Feb 9, 2021
8. fantix

   added a commit that references this issue on Feb 9, 2021

   Fix future issue in sock_connect()

   ...

   This issue will close once commit fe3d028 is merged into the 'master' branch.

   fe3d028