Skip to content

Improve exception message in ast.literal_eval #77069

Description

@Rosuav
BPO 32888
Nosy @briancurtin, @Rosuav, @serhiy-storchaka
PRs
  • bpo-32888: Improve exception message in ast.literal_eval #340
  • bpo-38396: dump(node) instead of repr(node) on ast.literal_eval #16620
  • gh-77069: add context info to ast.literal_eval error messages #17662
  • Dependencies
  • bpo-32892: Remove specific constant AST types in favor of ast.Constant
  • bpo-32893: ast.literal_eval() shouldn't accept booleans as numbers in AST
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = None
    created_at = <Date 2018-02-20.17:26:21.508>
    labels = ['3.8', 'library']
    title = 'Improve exception message in ast.literal_eval'
    updated_at = <Date 2019-12-19.15:26:59.362>
    user = 'https://github.com/Rosuav'

    bugs.python.org fields:

    activity = <Date 2019-12-19.15:26:59.362>
    actor = 'BTaskaya'
    assignee = 'none'
    closed = False
    closed_date = None
    closer = None
    components = ['Library (Lib)']
    creation = <Date 2018-02-20.17:26:21.508>
    creator = 'Rosuav'
    dependencies = ['32892', '32893']
    files = []
    hgrepos = []
    issue_num = 32888
    keywords = ['patch']
    message_count = 6.0
    messages = ['312423', '312424', '312427', '312429', '312442', '320993']
    nosy_count = 3.0
    nosy_names = ['brian.curtin', 'Rosuav', 'serhiy.storchaka']
    pr_nums = ['340', '16620', '17662']
    priority = 'normal'
    resolution = None
    stage = 'patch review'
    status = 'open'
    superseder = None
    type = None
    url = 'https://bugs.python.org/issue32888'
    versions = ['Python 3.8']

    Activity

    1. Rosuav commented on Feb 20, 2018

      @Rosuav
      ContributorAuthor

      When a non-literal is given to literal_eval, attempt to be more
      helpful with the message, rather than calling it 'malformed'.

    2. added
      stdlibStandard Library Python modules in the Lib/ directory
      on Feb 20, 2018
    3. serhiy-storchaka commented on Feb 20, 2018

      @serhiy-storchaka
      Member

      I afraid this makes an error message more confusing and misleading.

      >>> ast.literal_eval('~2')
      Traceback (most recent call last):
        File "<stdin>", line 1, in <module>
        File "/home/serhiy/py/cpython/Lib/ast.py", line 93, in literal_eval
          return _convert(node_or_string)
        File "/home/serhiy/py/cpython/Lib/ast.py", line 92, in _convert
          return _convert_signed_num(node)
        File "/home/serhiy/py/cpython/Lib/ast.py", line 65, in _convert_signed_num
          return _convert_num(node)
        File "/home/serhiy/py/cpython/Lib/ast.py", line 56, in _convert_num
          raise ValueError('%s not allowed in literal' % type(node).__name__)
      ValueError: UnaryOp not allowed in literal

      This is not true since + and - are allowed:

      >>> ast.literal_eval('-2')
      -2
      
      >>> ast.literal_eval('2+3')
      Traceback (most recent call last):
        File "<stdin>", line 1, in <module>
        File "/home/serhiy/py/cpython/Lib/ast.py", line 93, in literal_eval
          return _convert(node_or_string)
        File "/home/serhiy/py/cpython/Lib/ast.py", line 92, in _convert
          return _convert_signed_num(node)
        File "/home/serhiy/py/cpython/Lib/ast.py", line 65, in _convert_signed_num
          return _convert_num(node)
        File "/home/serhiy/py/cpython/Lib/ast.py", line 56, in _convert_num
          raise ValueError('%s not allowed in literal' % type(node).__name__)
      ValueError: BinOp not allowed in literal

      But:

      >>> ast.literal_eval('2+3j')
      (2+3j)
      
      >>> ast.literal_eval('"a"+"b"')
      Traceback (most recent call last):
        File "<stdin>", line 1, in <module>
        File "/home/serhiy/py/cpython/Lib/ast.py", line 93, in literal_eval
          return _convert(node_or_string)
        File "/home/serhiy/py/cpython/Lib/ast.py", line 85, in _convert
          left = _convert_signed_num(node.left)
        File "/home/serhiy/py/cpython/Lib/ast.py", line 65, in _convert_signed_num
          return _convert_num(node)
        File "/home/serhiy/py/cpython/Lib/ast.py", line 56, in _convert_num
          raise ValueError('%s not allowed in literal' % type(node).__name__)
      ValueError: Str not allowed in literal

      But Str is allowed:

      >>> ast.literal_eval('"ab"')
      'ab'
    4. Rosuav commented on Feb 20, 2018

      @Rosuav
      ContributorAuthor

      Hmm, I think I see what I broke there. It was part of the merge conflict resolution - I moved the check into the function, which is actually incorrect. It wasn't misleading like that in the original patch. Will fix that.

    5. Rosuav commented on Feb 20, 2018

      @Rosuav
      ContributorAuthor

      Actually, it's a bit more complicated than I thought. Current proposed solution: Track the context of each conversion, thus allowing different errors to be distinguished.

    6. Rosuav commented on Feb 20, 2018

      @Rosuav
      ContributorAuthor

      (BTW, by "proposed" I mean that the change that I describe is in the PR.)

    7. serhiy-storchaka commented on Jul 3, 2018

      @serhiy-storchaka
      Member

      Please wait until bpo-32892 and bpo-32893 be committed or rejected before merging this PR.

    8. transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      3.8 (EOL)end of lifestdlibStandard Library Python modules in the Lib/ directory

      Projects

      No projects

        Milestone

        No milestone

        Relationships

        None yet

        Development

        No branches or pull requests

        Issue actions