Repository navigation
Leaked connections caused by timeout on pool.acquire #467
Description
- asyncpg version: 0.18.2
- PostgreSQL version: 10
- Do you use a PostgreSQL SaaS? If so, which? Can you reproduce
the issue with a local PostgreSQL install?: - Python version: 3.6.6
- Platform:
- Do you use pgbouncer?: yes
- Did you install asyncpg with pip?:
- If you built asyncpg locally, which version of Cython did you use?:
- Can the issue be reproduced under both asyncio and
uvloop?:
Timeout on pool.acquire can be cause of leaked connections. The reason is that asyncio.wait_for cancels inner task on timeout here and it is the same problem as described in #464. But in this case using of asyncio.shield outside asyncpg cannot fix the problem.
The code below reproduces the problem (timeout which reproduces the problem depends on connection time):
import asyncpg
import traceback
async def create_pool():
return await asyncpg.create_pool(
host='localhost',
port='6432',
database='db1',
user='user1',
password='12345678',
min_size=0,
max_size=3,
)
async def test_connect(pool):
conn = None
try:
async with pool.acquire(timeout=0.11) as conn: # the problem is here
await conn.fetch(
'SELECT pg_sleep(0.1);',
)
except Exception as e:
print('Error occured')
traceback.print_exc()
async def main():
pool = await create_pool()
for i in range(100):
await test_connect(pool)
await pool.close()
await asyncio.sleep(600)
import asyncio
loop = asyncio.get_event_loop()
loop.run_until_complete(main())Traceback of error:
Traceback (most recent call last):
File "<stdin>", line 4, in test_connect
File "python3.6/site-packages/asyncpg/pool.py", line 760, in __aenter__
self.connection = await self.pool._acquire(self.timeout)
File "python3.6/site-packages/asyncpg/pool.py", line 605, in _acquire
_acquire_impl(), timeout=timeout, loop=self._loop)
File "python3.6/asyncio/tasks.py", line 362, in wait_for
raise futures.TimeoutError()
concurrent.futures._base.TimeoutError
Even after pool is closed some connections are not closed while python process is running.
Activity
fixes leaking connections
stalkerg commentedon Aug 13, 2019
I don't know is it the same problem or not but my server time to time start freeze because all connection was stuck. I have some exceptions time to time from PG but it's not a timeout.
Close transport if connection times out (#468)
1st1 commentedon Nov 19, 2019
Should be fixed in master. Thanks for reporting!
Ensure pool connection is released in acquire when cancelled
Add a workaround for bpo-37658
Add a workaround for bpo-37658
Add a workaround for bpo-37658
Add a workaround for bpo-37658