Repository navigation
Add a "strict" parameter to csv.writer and csv.DictWriter #85002
Description
| BPO | 40825 |
|---|---|
| Nosy | @ericvsmith, @serhiy-storchaka, @corona10 |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = None
created_at = <Date 2020-05-30.12:31:33.111>
labels = ['type-feature', '3.10']
title = 'Add a "strict" parameter to csv.writer and csv.DictWriter'
updated_at = <Date 2020-06-02.17:01:51.050>
user = 'https://github.com/ericvsmith'bugs.python.org fields:
activity = <Date 2020-06-02.17:01:51.050>
actor = 'eric.smith'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = []
creation = <Date 2020-05-30.12:31:33.111>
creator = 'eric.smith'
dependencies = []
files = []
hgrepos = []
issue_num = 40825
keywords = []
message_count = 10.0
messages = ['370380', '370382', '370393', '370402', '370424', '370485', '370486', '370490', '370498', '370613']
nosy_count = 3.0
nosy_names = ['eric.smith', 'serhiy.storchaka', 'corona10']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue40825'
versions = ['Python 3.10']
Activity
ericvsmith commentedon May 30, 2020
Currently, the csv library calls str() on each value it writes. This can lead to surprising behavior, see bpo-40762 for example.
On the other hand, for writing the documentation says that the values must be strings or numbers.
The proposed "strict" argument would raise a TypeError if the supplied values are not strings, numbers, or None.
See
cpython/Modules/_csv.c
Line 1203 in ba1c2c8
The documentation should be changed to note that None is allowed. Currently, None results in an empty string. I'm not proposing to change this, just document it as one of the allowed types.
How to check for "value is a number" needs to be decided.
ericvsmith commentedon May 30, 2020
For backward compatibility, strict would have to default to False.
corona10 commentedon May 30, 2020
I am +1 on with strict mode.
But I want to hear other core developers opinions.
ericvsmith commentedon May 30, 2020
I guess an isinstance check against numbers.Number would be the best way to check if an argument is a number. I'm not sure how convenient that is from C code.
serhiy-storchaka commentedon May 31, 2020
There is PyNumber_Check(). It is not direct analog of isinstance(obj, numbers.Number), it checks that the object can be explicitly converted to the real number (int or float). UUID and IPv4Address pass this check.
As a narrow check we can use isinstance(obj, (str, int, float)). It does not accept Fraction, Decimal and numpy numbers, but it is what such modules like json or plistlib accept.
ericvsmith commentedon May 31, 2020
Losing Decimal would be a problem. I use those a lot in CSV files, and I assume others do, too.
serhiy-storchaka commentedon May 31, 2020
We can check only nb_index and nb_float. It will include Fraction, Decimal and NumPy numbers and exclude UUID and IPv4Address.
ericvsmith commentedon May 31, 2020
That seems like a good plan.
serhiy-storchaka commentedon May 31, 2020
Yes, converting Decimal to float can lose precision, so we cannot require this. PyNumber_Check() is already used for QUOTE_NONNUMERIC, so it would be logical to use it in determining that the object is a number in the cvs module.
But now the problem is with determining what is a "string". There is no way to check whether the object is "string-like", because virtually all objects can be converted to string. The only standard exception is bytes and bytearray for which str() may emit BytesWarning, so this conversion is not reliable. If restrict it only to an instance of str or its subclasses, it may break other user cases, for example writing Path in CVS.
ericvsmith commentedon Jun 2, 2020
I wouldn't have a problem with isinstance(obj, str) for a string check in strict mode. If you want to write something like a Path, convert it to a string yourself. That's exactly the behavior I'd like enforced by strict: only accept numbers and actual strings.
serhiy-storchaka commentedon Aug 3, 2026
gh-155097 (PR #155099) adds a
formatterparameter tocsv.writer, passed through byDictWriter. It replaces thestr()call, and is called only for values that are neither strings norNone— i.e. exactly the values a strict mode would need to check:That is the
nb_index/nb_floatcheck discussed above:Decimal,Fractionand NumPy numbers pass,PathandUUIDdo not, andbytesis rejected rather than triggeringBytesWarning.A dedicated
strict=Truewould still be more discoverable than a four-line hook.