Skip to content

Use locale.nl_langinfo in _strptime.py #53161

Open

Description

@brettcannon
BPO 8915
Nosy @abalkin

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = 'https://github.com/abalkin'
closed_at = None
created_at = <Date 2010-06-06.03:35:08.311>
labels = ['3.7', 'type-feature', 'library']
title = 'Use locale.nl_langinfo in _strptime'
updated_at = <Date 2016-09-10.18:34:48.582>
user = 'https://github.com/brettcannon'

bugs.python.org fields:

activity = <Date 2016-09-10.18:34:48.582>
actor = 'belopolsky'
assignee = 'belopolsky'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2010-06-06.03:35:08.311>
creator = 'brett.cannon'
dependencies = []
files = []
hgrepos = []
issue_num = 8915
keywords = []
message_count = 3.0
messages = ['107181', '107445', '125958']
nosy_count = 1.0
nosy_names = ['belopolsky']
pr_nums = []
priority = 'low'
resolution = None
stage = 'needs patch'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue8915'
versions = ['Python 3.7']

Activity

  1. brettcannon commented on Jun 6, 2010

    @brettcannon
    MemberAuthor

    It might perform better to use locale.nl_langinfo to get the current locale's datetime information instead of reverse-engineering from strftime (need to benchmark to see if this is true). This would need to be conditional as the datetime info might not be exposed through nl_langinfo.

  2. added
    stdlibStandard Library Python modules in the Lib/ directory
    on Jun 6, 2010
  3. abalkin commented on Jun 10, 2010

    @abalkin
    Member

    See also bpo-8957. If this happens, I would like to add a pure python implementation strftime. See also bpo-7989.

  4. self-assigned this
    on Jun 10, 2010
  5. abalkin commented on Jan 10, 2011

    @abalkin
    Member

    I would also like to consider using OS strptime on platforms with a decent implementation.

  6. transferred this issue fromon Apr 10, 2022
  7. StanFromIreland commented on Mar 16, 2025

    @StanFromIreland
    Member

    Not fully possible, nl_langinfo doesn't support LC_TIME.

    We can partially use it, ~33% faster.

    def _getlang():
        lang = locale.setlocale(locale.LC_TIME, None)
        encoding = locale.nl_langinfo(locale.CODESET)
    
        return lang, encoding
    $ python3.14 -m timeit -s "from _strptime import _getlang" "_getlang()"
    1000000 loops, best of 5: 298 nsec per loop
    $ ./python -m timeit -s "from _strptime import _getlang" "_getlang()"
    1000000 loops, best of 5: 203 nsec per loop                    # ~33% faster

    Caching the information will result in a decent performance gain.

  8. StanFromIreland commented on Mar 16, 2025

    @StanFromIreland
    Member

    I would also like to consider using OS strptime on platforms with a decent implementation.

    This should be a separate issue.

  9. added and removed on Mar 16, 2025
  10. StanFromIreland commented on Mar 16, 2025

    @StanFromIreland
    Member

    @picnixz _strptime is currently pure python. This does not need the extension-modules label. It does however need the performance label :-)

  11. changed the title [-]Use locale.nl_langinfo in _strptime[/-] [+]Use locale.nl_langinfo in `_strptime.py`[/+] on Mar 16, 2025
  12. added
    performancePerformance or resource usage
    and removed on Mar 16, 2025
  13. picnixz commented on Mar 16, 2025

    @picnixz
    Member

    I never know which part of the date/time API is duplicated in C and which one is not so thanks.

  14. serhiy-storchaka commented on Jun 28, 2025

    @serhiy-storchaka
    Member

    nl_langinfo() supports LC_TIME.

    But there is other issue. Month and weekday names returned by nl_langinfo() and strftime() can be different.

    • In br_FR locale: "'" vs "ʼ" (U+02BC).
    • In ast_ES, ca_AD, ca_ES, ca_FR, ca_IT, ca_ES, oc_FR and wa_BE locales: "'" vs "’" (U+2019).
    • In yi_US locale: "אַ" (U+FB2E) vs "אַ" (U+05D0 U+05B7) and many others.

    I suppose this is because strftime() is implemented using wcsftime() if it is available, but nl_langinfo() needs decoding from the current locale encoding. 8-bit encodings (ISO8859-1 for br_FR and ca_FR, CP1255 for yi_US, etc) can replace some Unicode characters with other similarly looking characters.

    This issue exists also in the current code: strptime() is not always able to parse string formatted in C or other language or in Python on other platform, even if they support the same locale. strptime() should be more lenient and accept different forms of apostrophes and different form of normalization. This is a different issue, but we cannot just use nl_langinfo() without breaking existing tests until it is fixed.

    On glibc platforms we can also use private API to get Unicode result of nl_langinfo(), without using intermediate locale encoding. This may be faster and allows to avoid temporary switching the current locale. This will help also for Python implementation of strftime(). This is also a different issue, but it will help to use nl_langinfo().

  15. serhiy-storchaka commented on Jun 28, 2025

    @serhiy-storchaka
    Member

    There is other issue. On some locales there are different names for months and weekdays, it is not a matter of some normalization:

    • be_BY.utf8: "чэрвеня" vs "červienia".
    • tt_RU.utf8: "июнь" vs "yün".
    • nan_TW@latin: "6goe̍h" vs "六月".
    • sr_RS@latin: "jun" vs "јун".
    • ug_CN@latin: "Seper" vs "ئىيۇن".
    • uz_UZ@cyrillic: "Июн" vs "Iyun".
    • sd_IN@devanagari: "जूनि" vs "جون".

    It seems that the modifier (@latin, @cyrillic, @devanagari) is just ignored in some cases (this is the locale module bug). But I do not know why there is a difference for be_BY.utf8 and tt_RU.utf8.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

performancePerformance or resource usagestdlibStandard Library Python modules in the Lib/ directorytype-featureA feature request or enhancement

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions