You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
BaseCursor#close(..) can exhaust the system memory #560
Unfortunately, this can cause Python to run out of memory when the cursor yields hundreds of millions of records.
To Reproduce
Create a new table
Populate it with hundreds of millions of records
Create a cursor that will yield all of the records from this table. (Ex: SELECT * FROM HugeTable)
Fetch a handful of records from the cursor. (ie: cursor.fetchmany(100) )
Close the cursor
Environment
This bug is present regardless of environment; however, here's the environment that I was using:
MySQL Server
AWS RDS. (Presumably, the use Amazon Linux)
Server Version 8.0.23
MySQL Client
OS: Ubuntu Jammy
Python >= 3.7
mysqlclient 2.0.3 and mysqlclient 2.1.1
Additional Context
I can probably implement a fix for this once we confirm that it's a legitimate bug. At a high-level, I suspect the call to self.fetchall() probably needs to be replaced with a call to self.fetchmany(..). However, this might be complicated by structural details that I'm not aware of.
Activity
Discard results without converting them into Python objects. (#601)