Skip to content

Python implementation of json.loads() accepts invalid unicode escapes #125660

Closed
Closed
Python implementation of json.loads() accepts invalid unicode escapes#125660
Labels
stdlibStandard Library Python modules in the Lib/ directorytype-bugAn unexpected behavior, bug, or error

Description

@nineteendo
nineteendo
opened on Oct 17, 2024

Bug report

Bug description:

While reviewing #125652 and reading the documentation of int(), I realised this condition in json.decoder is insufficient:

cpython/Lib/json/decoder.py

Line 61 in f203d1c

if len(esc) == 4 and esc[1] not in 'xX': 

>>> import sys
>>> sys.modules["_json"] = None
>>> import json
>>> json.loads(r'["\u 000", "\u-000", "\u+000", "\u0_00"]')
['\x00', '\x00', '\x00', '\x00']

CPython versions tested on:

3.13

Operating systems tested on:

macOS

Linked PRs

Activity

  1. nineteendo
    added
    type-bugAn unexpected behavior, bug, or error
    on Oct 17, 2024

  2. nineteendo commented on Oct 17, 2024

    @nineteendo
    nineteendo
    on Oct 17, 2024
    ContributorAuthor

    cc @serhiy-storchaka

  3. nineteendo commented on Oct 17, 2024

    @nineteendo
    nineteendo
    on Oct 17, 2024
    ContributorAuthor

    Maybe something like this? Although it might be a better idea to use a stricter function.

    esc = s[end:end + 4].strip()
if "_" not in esc and len(esc) == 4 and esc[0] not in "+-" and esc[1] not in "xX":
  4. Eclips4
    added
    stdlibStandard Library Python modules in the Lib/ directory
    on Oct 17, 2024
  5. Eclips4
    added this to JSON issueson Oct 17, 2024

  6. serhiy-storchaka commented on Oct 17, 2024

    @serhiy-storchaka
    serhiy-storchaka
    on Oct 17, 2024
    Member

    Either this, or simply use regexp.

  7. nineteendo commented on Oct 18, 2024

    @nineteendo
    nineteendo
    on Oct 18, 2024
    ContributorAuthor

    Let's use a regex, unicode digits aren't allowed either:

    >>> int("\uff10", 16)
0

  8. nineteendo commented on Oct 18, 2024

    @nineteendo
    nineteendo
    on Oct 18, 2024
    ContributorAuthor

    Could there be other code that suffers from this problem?

  9. taleinat commented on Oct 18, 2024

    @taleinat
    taleinat
    on Oct 18, 2024
    Contributor

    Could there be other code that suffers from this problem?

    Possibly, but that's out of context for this issue. If someone finds such issues they should be reported separately.

  10. serhiy-storchaka
    added a commit that references this issue on Oct 18, 2024

    gh-125660: Reject invalid unicode escapes for Python implementation o…

    df75136
  11. miss-islington
    added 2 commits that reference this issue on Oct 18, 2024

    pythongh-125660: Reject invalid unicode escapes for Python implementa…

    025b9e8

    pythongh-125660: Reject invalid unicode escapes for Python implementa…

    49905fb
  12. serhiy-storchaka
    added 3 commits that reference this issue on Oct 19, 2024

    pythongh-125660: Enable setting persistent_id and persistent_load of …

    322e2aa

    [3.13] gh-125660: Reject invalid unicode escapes for Python implement…

    d9dafc7

    [3.12] gh-125660: Reject invalid unicode escapes for Python implement…

    2746ec4
  13. serhiy-storchaka
    closed this as completedon Oct 21, 2024
  14. github-project-automation
    moved this to Done in JSON issueson Oct 21, 2024
  15. ebonnal
    added a commit that references this issue on Jan 12, 2025

    pythongh-125660: Reject invalid unicode escapes for Python implementa…

    a05b129
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    stdlibStandard Library Python modules in the Lib/ directorytype-bugAn unexpected behavior, bug, or error

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions