Repository navigation
datetime: pure Python implementation of fromisoformat() handles times with trailing spaces inconsistently with the C extension #130959
Description
Bug report
Bug description:
Discovered primarily because PyPy uses the Python version of datetime from the stdlib, and django.utils.dateparse.parse_datetime() supports wider range of values than the C version of datetime.datetime.fromisoformat(), and falls back to their own parser.
>>> datetime.datetime.fromisoformat("2012-04-23T10:20:30.400")
datetime.datetime(2012, 4, 23, 10, 20, 30, 400000)
>>> datetime.datetime.fromisoformat("2012-04-23T10:20:30.400 ")
datetime.datetime(2012, 4, 23, 10, 20, 30, 40000)
>>> datetime.datetime.fromisoformat("2012-04-23T10:20:30.400 ")
datetime.datetime(2012, 4, 23, 10, 20, 30, 4000)
>>> datetime.datetime.fromisoformat("2012-04-23T10:20:30.400 ")
datetime.datetime(2012, 4, 23, 10, 20, 30, 400)
>>> datetime.datetime.fromisoformat("2012-04-23T10:20:30.400 ")
Traceback (most recent call last):
File "<python-input-9>", line 1, in <module>
datetime.datetime.fromisoformat("2012-04-23T10:20:30.400 ")
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/mgorny/git/cpython/Lib/_pydatetime.py", line 1949, in fromisoformat
raise ValueError(
f'Invalid isoformat string: {date_string!r}') from None
ValueError: Invalid isoformat string: '2012-04-23T10:20:30.400 '
>>> datetime.datetime.fromisoformat("2012-04-23T10:20:30.400 +02:30")
datetime.datetime(2012, 4, 23, 10, 20, 30, 40000, tzinfo=datetime.timezone(datetime.timedelta(seconds=9000)))For comparison, the C extension rejects all variants containing spaces, causing Django to use its own parser.
PyPy bug report: pypy/pypy#5240
I'm going to try preparing a patch.
CPython versions tested on:
3.11, 3.13, CPython main branch
Operating systems tested on:
Linux
Activity
[-]datetime: pure Python implementation of `fromisoformat()` mishandles times with trailing spaces[/-][+]datetime: pure Python implementation of `fromisoformat()` handles times with trailing spaces inconsistently[/+][-]datetime: pure Python implementation of `fromisoformat()` handles times with trailing spaces inconsistently[/-][+]datetime: pure Python implementation of `fromisoformat()` handles times with trailing spaces inconsistently with the C extension[/+]mgorny commentedon Mar 7, 2025
Hmm, another difference is that when giving 6 digits of microseconds, C implementation accepts them but Python implementation does not:
pythongh-130959: Reject whitespace in fractions, in pure Python `from…
pganssle commentedon Mar 8, 2025
I think the spec doesn't allow for whitespace, right? If so, we should do whatever the spec says. It's slightly more likely to break things if we make changes to the C implementation than the Python implementation (since the vast majority of people are using the C implementation).
I think we can fix the issue with the Python implementation and backport the changes to 3.12+, since right now it just gives the wrong answer, and it's better to start raising exceptions here. If we reject the version with a space from the C implementation, I think we would want that to target only 3.14+, because that will take something that currently parses and gives the "correct" answer and start raising exceptions. We don't have to go as far as deprecating the behavior, since the documented scope of the function says that this should be rejected, but I don't think on net it serves users to make that kind of change in a bugfix version.
mgorny commentedon Mar 8, 2025
Yes, the standard doesn't permit whitespace, and it's not listed in "exceptions" in the documentation.
gh-130959: Reject whitespace in fractions, in pure Python `fromisofor…
pythongh-130959: Reject whitespace in fractions, in pure Python `from…
[3.13] gh-130959: Reject whitespace in fractions, in pure Python `fro…
[3.13] pythongh-130959: Reject whitespace in fractions, in pure Pytho…
gh-130959: Reject whitespace in fractions, in pure Python `fromisofor…
StanFromIreland commentedon Mar 15, 2025
Can this be closed? PR and backports have been merged.