datetime: strftime/strptime sub-second precision specifier (%Nf)

[karolinepauls]

Feature or enhancement

Note: in this issue datetime refers to datetime.datetime, while time refers to datetime.time. This issue doesn't talk about the time module.

Background

Currently, in datetime.strftime/time.strftime, the only sub-second formatting code available is %f, which produces a fixed-width string of microsecond digits, padded from the left with zeros, like "004312".

Despite %f coding for an integer, its zero-padded nature makes it usable in producing the "fraction of the second" part of a datetime string. For example "%T.%f" results in a string like 22:11:55.012345.

Problem

There is still no way for datetime.strftime/time.strftime, alone to produce milliseconds, like 22:11:55.012, or any other fraction length between 1 and 5.

Non-solutions

If you control the code, you can call d.isoformat(timespec="milliseconds"). If all you have is configuration where you specify the datetime.strftime format string, you're in no luck.

Proposal

I'm proposing an expanded variant of the %f datetime.strftime format code in the form of %Nf where N∈{1,2,3,4,5,6}. It would produce datetime microseconds, zero-padded to 6 decimal positions and truncated to N most significant positions.

Examples:

1. If %f → "012345", then:

• %6f → "012345"
• %5f → "01234"
• %4f → "0123"
• %3f → "012"
• %2f → "01"
• %1f → "0"

2. An ISO 8601 format specifying millisecond precision: %Y-%m-%dT%H:%M:%S.%3f%:z
3. A human-friendly format with 2 sub-second decimal places: %Y-%m-%d %H:%M:%S.%2f %z

Issues and choices

Truncation or rounding

For microseconds equal to 012345, should %5f produce:

• "01234" (truncation)
• "01235" (arithmetic rounding)
• some other rounding mode?

My proposed change truncates.

Format code syntax

N in %Nf theoretically conflicts with "optional minimum field width" in POSIX libc. The post by @jb2170 highlights that the conflict is very theoretical because %f is a Python extension (and libc doesn't include sub-seconds in struct tm).

This theoretical conflict could still be avoided by following alternative codes:

• %.Nf instead of %Nf. This however was deemed confusing, since having a dot in the format code suggests that the code would also produce a dot in its output, and generally, round a float rather than truncating an integer.
• %:Nf (where :N is analogous to the slicing operation) was floated, with no traction.

Overall, the 2 recently active discussion members are in favour of %Nf. I believe that either %Nf or %:Nf could be valid, which makes 3/3 recent posters in favour of %Nf.

Format code corner cases

In my initial implementation, if N falls outside of the supported [1, 6] range, a ValueError is raised. Since format code scanning operates on characters, such detection will only occur for 0, 7, 8 and 9. A code like "%10f" will not be intercepted and will hit the libc - handling always-incorrect multi-digit codes would complicate parsing in the C variant of wrap_strftime so I'm not considering that.

As an alternative to the ValueError, N=0 could produce nothing while values of 7, 8, 9 could act like N=6.

My preference: No change to initial code.

strptime leniency

In my initial implementation a %Nf code in datetime.strptime/time.strptime means that exactly N digits are expected. "%3f" will accept "111" but not "11" or "1111". This is OK since the existing %f code still accepts any number of digits in the [1, 6] range, as before.

Alternatively, a relaxation could be made instead: for a %Nf code, any number of digits from the range [1, N] could be accepted.

My preference: No relaxation.

Prior art

Rust's chrono lib permits %3f, %6f, and %9f (datetime cannot support %9f, since it doesn't go beyond microseconds).

Has this already been discussed elsewhere?

I have already discussed this feature proposal on Discourse

Links to previous discussion of this feature:

https://discuss.python.org/t/add-millisecond-formatting-support-to-datetime-strftime

Implementation

My initial implementation is in this PR:

Linked PRs

• gh-157290: datetime: strftime %Nf microsecond format #157291
• gh-157290: Add support for %Nf to datetime.str{f,p}time() #157704

[vstinner]

I don't think that we should allow precision other than 3 (ms) and 6 (us) digits.

[pganssle]

I don't think that we should allow precision other than 3 (ms) and 6 (us) digits.

I feel like this is a kind of arbitrary restriction. You are probably right that those are the main ones that people want, but I feel like it's not much extra work to support an arbitrary single digit rather than filter to just 3 and 6, and I don't see much downside to allowing it. fromisoformat allows arbitrary precision for parsing, though I think isoformat is limited to milliseconds and microseconds when outputting.

[karolinepauls]

I don't think that we should allow precision other than 3 (ms) and 6 (us) digits.

I get you - it's generally a good idea to restrict scope. For my own use case %3f is enough.

These are the reasons to support 1 <= N <= 6 (I did use LLMs to explore):

1. In industrial/trading/legacy applications the sub-second precision is often dictated by display width.
2. In predictive applications, precision may be lowered to reduce jitter.
3. ISO 8601 note by WTC allows adopters to define and require arbitrary precision. To us it means that a Python programmer implementing an integration may be required to produce an arbitrary number of digits.

This profile does not specify how many digits may be used to represent the decimal fraction of a second. An adopting standard that permits fractions of a second must specify both the minimum number of digits (a number greater than or equal to one) and the maximum number of digits (the maximum may be stated to be "unlimited").

4. like @pganssle said, fromisoformat supports any precision. When it did not, it was considered a missing feature: datetime.fromisoformat() only accepts 3 or 6 decimal places for fractional seconds #95221 (comment)

The added costs of supporting 1 <= N <= 6 are:

1. More testing (done).
2. Possibly harder to explain/document the feature

Implementation complexity does not meaningfully change.

[StanFromIreland]

I also don't see the need for a restriction. I think it costs us more than not having one. We don't know what users need, and there are use cases for other widths like centiseconds (which I know are used in various places, like LRC files). Furthermore, see Ruby's strftime's %N or GNU's date, which accept any width specifier.

I propose a slightly simpler implementation in #157704.