Repository navigation
[sqlite3] Do not truncate results of user functions and aggregates on the first NUL #88985
Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/erlend-aasland'
closed_at = <Date 2021-08-06.21:08:39.691>
created_at = <Date 2021-08-04.08:29:38.206>
labels = ['extension-modules', 'type-bug', '3.9', '3.10', '3.11']
title = '[sqlite3] Do not truncate results of user functions and aggregates on the first NUL'
updated_at = <Date 2021-08-06.21:08:39.690>
user = 'https://github.com/erlend-aasland'bugs.python.org fields:
activity = <Date 2021-08-06.21:08:39.690>
actor = 'erlendaasland'
assignee = 'erlendaasland'
closed = True
closed_date = <Date 2021-08-06.21:08:39.691>
closer = 'erlendaasland'
components = ['Extension Modules']
creation = <Date 2021-08-04.08:29:38.206>
creator = 'erlendaasland'
dependencies = []
files = []
hgrepos = []
issue_num = 44822
keywords = ['patch']
message_count = 8.0
messages = ['398865', '398869', '398873', '398881', '398971', '399115', '399146', '399149']
nosy_count = 3.0
nosy_names = ['serhiy.storchaka', 'miss-islington', 'erlendaasland']
pr_nums = ['27588', '27611', '27639']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue44822'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']
Activity
erlend-aasland commentedon Aug 4, 2021
The third argument to sqlite3_result_text() is the length of the string passed as the second argument. Currently, we pass -1, so SQLite has to invoke strlen() to compute the length of the passed string. Suggesting to use PyUnicode_AsUTF8AndSize() iso. PyUnicode_AsUTF8() and pass the string size to avoid the superfluous strlen().
See also:
serhiy-storchaka commentedon Aug 4, 2021
The difference between specifying negative and non-negative third argument of sqlite3_result_text() is that in the latter case the result can contain embedded NUL characters.
Could you please add a test for string containing embedded NUL?
Letting SQLite compute string length would work incorrect in case of embedded NULs, so I think that we should raise explicit error if sz > INT_MAX.
erlend-aasland commentedon Aug 4, 2021
Of course, thanks for the heads up.
Yes, I thought about this. I do agree that raising OverflowError would be better. Thanks.
erlend-aasland commentedon Aug 4, 2021
Serhiy, I've updated the PR, if you want to take a look. Thanks for your feedback.
[-][sqlite3] Micro-optimisation: pass string size to sqlite3_result_text()[/-][+][sqlite3] Do not truncate results of user functions and aggregates on the first NUL[/+][-][sqlite3] Micro-optimisation: pass string size to sqlite3_result_text()[/-][+][sqlite3] Do not truncate results of user functions and aggregates on the first NUL[/+]serhiy-storchaka commentedon Aug 5, 2021
New changeset 8f010dc by Erlend Egeberg Aasland in branch 'main':
bpo-44822: Don't truncate
strs with embedded NULL chars returned bysqlite3UDF callbacks (GH-27588)8f010dc
miss-islington commentedon Aug 6, 2021
New changeset 2b1e713 by Miss Islington (bot) in branch '3.10':
bpo-44822: Don't truncate
strs with embedded NULL chars returned bysqlite3UDF callbacks (GH-27588)2b1e713
serhiy-storchaka commentedon Aug 6, 2021
New changeset c352412 by Erlend Egeberg Aasland in branch '3.9':
[3.9] bpo-44822: Don't truncate
strs with embedded NULL chars returned bysqlite3UDF callbacks (GH-27588). (GH-27639)c352412
erlend-aasland commentedon Aug 6, 2021
Thanks, Serhiy!