end_offset is ignored in subclasses of SyntaxError (3.12-only)

Bug report

Bug description:

When I tried to implement a subclass of SyntaxError, the exception details ignored the end_offset-variable and just showed one caret at the position of offset.

raise SyntaxError("Error message", (None, 1, 5, 'a = sin(3)', 1, 9))

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<string>", line 1
    a = sin(3)
        ^^^^
SyntaxError: Error message

class CustomSyntaxError(SyntaxError):
    pass
raise CustomSyntaxError("Error message", (None, 1, 5, 'a = sin(3)', 1, 9))

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<string>", line 1
    a = sin(3)
        ^
CustomSyntaxError: Error message

CPython versions tested on:

3.11, 3.12

Operating systems tested on:

Windows

Linked PRs

It works correctly for me on 3.13a1:

>>> class CustomSyntaxError(SyntaxError):
...     pass
... 
>>> raise CustomSyntaxError("Error message", (None, 1, 5, 'a = sin(3)', 1, 9))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
    raise CustomSyntaxError("Error message", (None, 1, 5, 'a = sin(3)', 1, 9))
  File "<string>", line 1
    a = sin(3)
        ^^^^
CustomSyntaxError: Error message

Also worked correctly for me on 3.10.10

raise SyntaxError("Error message", (None, 1, 5, 'a = sin(3)', 1, 9))
Traceback (most recent call last):
  File "<pyshell#0>", line 1, in <module>
    raise SyntaxError("Error message", (None, 1, 5, 'a = sin(3)', 1, 9))
  File "<string>", line 1
    a = sin(3)
        ^^^^
SyntaxError: Error message


class CustomSyntaxError(SyntaxError):
    pass
raise CustomSyntaxError("Error message", (None, 1, 5, 'a = sin(3)', 1, 9))
Traceback (most recent call last):
  File "<pyshell#4>", line 1, in <module>
    raise CustomSyntaxError("Error message", (None, 1, 5, 'a = sin(3)', 1, 9))
  File "<string>", line 1
    a = sin(3)
        ^^^^
CustomSyntaxError: Error message

For me (Python 3.11.6 on Windows 10) the console does not print the carets in the original error message, only when using the traceback package:

Python 3.11.6 (tags/v3.11.6:8b6ee5b, Oct  2 2023, 14:57:12) [MSC v.1935 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import traceback
>>> class CustomSyntaxError(SyntaxError):
...     pass
...
>>> try:
...     raise CustomSyntaxError("Error message", ('<py>', 1, 5, 'a = sin(3)', 1, 9))
... except Exception as e:
...     traceback.print_exc()
...     print('-'*20)
...     raise e
...
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "<py>", line 1
    a = sin(3)
        ^^^^
CustomSyntaxError: Error message
--------------------
Traceback (most recent call last):
  File "<stdin>", line 6, in <module>
  File "<stdin>", line 2, in <module>
  File "<py>", line 1
    a = sin(3)
        ^
CustomSyntaxError: Error message
>>> raise SyntaxError("Error message", ('<py>', 1, 5, 'a = sin(3)', 1, 9))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<py>", line 1
    a = sin(3)
        ^^^^
SyntaxError: Error message

See also this gist for the behavior in a Ipython console or Jupyter Notebook. There neither SyntaxError nor its subclass showed the desired result.
Maybe it has something to do with the error handling in the console, e.g. the console in PyCharm Community Edition works fine for me.

Python 3.11.1, 3.12.2 (Windows 10)
IDLE

import sys
import traceback
class CustomSyntaxError(SyntaxError):
    pass

try:
    raise CustomSyntaxError("Error message", ('<py>', 1, 5, 'a = sin(3)', 1, 9))
except Exception as e:
    traceback.print_exc()
    print('-' * 20, file=sys.stderr)
    raise e from None

Traceback (most recent call last):
  File "<pyshell#4>", line 2, in <module>
  File "<py>", line 1
    a = sin(3)
        ^^^^
CustomSyntaxError: Error message
--------------------
Traceback (most recent call last):
  File "<pyshell#4>", line 6, in <module>
    raise e from None
  File "<pyshell#4>", line 2, in <module>
    raise CustomSyntaxError("Error message", ('<py>', 1, 5, 'a = sin(3)', 1, 9))
  File "<py>", line 1
    a = sin(3)
        ^^^^
CustomSyntaxError: Error message

========= RESTART: D:\path\test.py =========
Traceback (most recent call last):
  File "D:\path\test.py", line 10, in <module>
    raise CustomSyntaxError("Error message", ('<py>', 1, 5, 'a = sin(3)', 1, 9))
  File "<py>", line 1
    a = sin(3)
        ^^^^
CustomSyntaxError: Error message
--------------------
Traceback (most recent call last):
  File "D:\path\test.py", line 14, in <module>
    raise e from None
  File "D:\path\test.py", line 10, in <module>
    raise CustomSyntaxError("Error message", ('<py>', 1, 5, 'a = sin(3)', 1, 9))
  File "<py>", line 1
    a = sin(3)
        ^^^^
CustomSyntaxError: Error message

cmd

C:\path\Python312>python
Python 3.12.2 (tags/v3.12.2:6abddd9, Feb  6 2024, 21:26:36) [MSC v.1937 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
random, math imported
>>> import sys
>>> import traceback
>>>
>>>
>>> class CustomSyntaxError(SyntaxError):
...     pass
...
>>>
>>> try:
...     raise CustomSyntaxError("Error message", ('<py>', 1, 5, 'a = sin(3)', 1, 9))
... except Exception as e:
...     traceback.print_exc()
...     print('-' * 20, file=sys.stderr)
...     raise e from None
...
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "<py>", line 1
    a = sin(3)
        ^^^^
CustomSyntaxError: Error message
--------------------
Traceback (most recent call last):
  File "<stdin>", line 6, in <module>
  File "<stdin>", line 2, in <module>
  File "<py>", line 1
    a = sin(3)
        ^
CustomSyntaxError: Error message
>>> ^Z


C:\path\Python312>python D:\path\test.py
Traceback (most recent call last):
  File "D:\path\test.py", line 10, in <module>
    raise CustomSyntaxError("Error message", ('<py>', 1, 5, 'a = sin(3)', 1, 9))
  File "<py>", line 1
    a = sin(3)
        ^^^^
CustomSyntaxError: Error message
--------------------
Traceback (most recent call last):
  File "D:\path\test.py", line 14, in <module>
    raise e from None
  File "D:\path\test.py", line 10, in <module>
    raise CustomSyntaxError("Error message", ('<py>', 1, 5, 'a = sin(3)', 1, 9))
  File "<py>", line 1
    a = sin(3)
        ^
CustomSyntaxError: Error message

PyCharm 2023.3.5 (Professional Edition)

Traceback (most recent call last):
  File "D:\path\test.py", line 10, in <module>
    raise CustomSyntaxError("Error message", ('<py>', 1, 5, 'a = sin(3)', 1, 9))
  File "<py>", line 1
    a = sin(3)
        ^^^^
CustomSyntaxError: Error message
--------------------
Traceback (most recent call last):
  File "D:\path\test.py", line 14, in <module>
    raise e from None
  File "D:\path\test.py", line 10, in <module>
    raise CustomSyntaxError("Error message", ('<py>', 1, 5, 'a = sin(3)', 1, 9))
  File "<py>", line 1
    a = sin(3)
        ^
CustomSyntaxError: Error message

test.py

import sys
import traceback


class CustomSyntaxError(SyntaxError):
    pass


try:
    raise CustomSyntaxError("Error message", ('<py>', 1, 5, 'a = sin(3)', 1, 9))
except Exception as e:
    traceback.print_exc()
    print('-' * 20, file=sys.stderr)
    raise e from None

Python 3.13.0a5 (windows 10) correct print

C:\path\Python313>python
Python 3.13.0a5 (tags/v3.13.0a5:076d169, Mar 12 2024, 21:29:03) [MSC v.1938 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
random, math imported
>>> import sys
>>> import traceback
>>>
>>>
>>> class CustomSyntaxError(SyntaxError):
...     pass
...
>>>
>>> try:
...     raise CustomSyntaxError("Error message", ('<py>', 1, 5, 'a = sin(3)', 1, 9))
... except Exception as e:
...     traceback.print_exc()
...     print('-' * 20, file=sys.stderr)
...     raise e from None
...
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
    raise CustomSyntaxError("Error message", ('<py>', 1, 5, 'a = sin(3)', 1, 9))
  File "<py>", line 1
    a = sin(3)
        ^^^^
CustomSyntaxError: Error message
--------------------
Traceback (most recent call last):
  File "<stdin>", line 6, in <module>
    raise e from None
  File "<stdin>", line 2, in <module>
    raise CustomSyntaxError("Error message", ('<py>', 1, 5, 'a = sin(3)', 1, 9))
  File "<py>", line 1
    a = sin(3)
        ^^^^
CustomSyntaxError: Error message
>>> ^Z


C:\path\Python313>python D:\path\test.py
Traceback (most recent call last):
  File "D:\path\test.py", line 10, in <module>
    raise CustomSyntaxError("Error message", ('<py>', 1, 5, 'a = sin(3)', 1, 9))
  File "<py>", line 1
    a = sin(3)
        ^^^^
CustomSyntaxError: Error message
--------------------
Traceback (most recent call last):
  File "D:\path\test.py", line 14, in <module>
    raise e from None
  File "D:\path\test.py", line 10, in <module>
    raise CustomSyntaxError("Error message", ('<py>', 1, 5, 'a = sin(3)', 1, 9))
  File "<py>", line 1
    a = sin(3)
        ^^^^
CustomSyntaxError: Error message

Can confirm on 3.12.4:

Python 3.12.4 (main, Jun  6 2024, 18:26:44) [Clang 15.0.0 (clang-1500.1.0.2.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from jsonyx import loads
>>> loads("NaN")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/wannes/Library/CloudStorage/OneDrive-Personal/Personal/GitHub/pyvz2/alpha/src/jsonyx/__init__.py", line 248, in loads
    return JSONDecoder(allow=allow).loads(s, filename=filename)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/wannes/Library/CloudStorage/OneDrive-Personal/Personal/GitHub/pyvz2/alpha/src/jsonyx/__init__.py", line 123, in loads
    return self._scanner(filename, s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 1
    NaN
    ^
jsonyx._decoder.JSONSyntaxError: NaN is not allowed

It works correctly on 3.13.0b4:

Python 3.13.0b4 (v3.13.0b4:567c38b4eb, Jul 18 2024, 07:28:29) [Clang 15.0.0 (clang-1500.3.9.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from jsonyx import loads
>>> loads("NaN")
Traceback (most recent call last):
  File "<python-input-1>", line 1, in <module>
    loads("NaN")
    ~~~~~^^^^^^^
  File "/Users/wannes/Library/CloudStorage/OneDrive-Personal/Personal/GitHub/pyvz2/alpha/src/jsonyx/__init__.py", line 248, in loads
    return JSONDecoder(allow=allow).loads(s, filename=filename)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/wannes/Library/CloudStorage/OneDrive-Personal/Personal/GitHub/pyvz2/alpha/src/jsonyx/__init__.py", line 123, in loads
    return self._scanner(filename, s)
           ~~~~~~~~~~~~~^^^^^^^^^^^^^
  File "/Users/wannes/Library/CloudStorage/OneDrive-Personal/Personal/GitHub/pyvz2/alpha/src/jsonyx/_decoder.py", line 398, in scanner
    raise exc.with_traceback(None) from None
  File "<string>", line 1
    NaN
    ^^^
jsonyx._decoder.JSONSyntaxError: NaN is not allowed

The bug is affecting 3.12.6 (haven't tested on 3.12.7) so I'll mark it as 3.12-only.

Could you do a bisect?

I don't really have time for now (I'm triaging the pending issues), so I'll leave it to anyone willing to do it.

This also wasn't working on 3.10 and 3.11. It was fixed in 3.13 by #110702.
~~Should we back port it to fix this?~~ We should fix it differently.

I don't want to choose between end_offset and storing the column number.

This fixes the problem:

diff --git a/Python/pythonrun.c b/Python/pythonrun.c
index 5f3d249df4..cf84573a8e 100644
--- a/Python/pythonrun.c
+++ b/Python/pythonrun.c
@@ -538,7 +538,6 @@ parse_syntax_error(PyObject *err, PyObject **message, PyObject **filename,
         *offset = hold;
     }
 
-    if (Py_TYPE(err) == (PyTypeObject*)PyExc_SyntaxError) {
     v = PyObject_GetAttr(err, &_Py_ID(end_lineno));
     if (!v) {
         PyErr_Clear();
@@ -570,11 +569,6 @@ parse_syntax_error(PyObject *err, PyObject **message, PyObject **filename,
             goto finally;
         *end_offset = hold;
     }
-    } else {
-        // SyntaxError subclasses
-        *end_lineno = *lineno;
-        *end_offset = -1;
-    }
 
     v = PyObject_GetAttr(err, &_Py_ID(text));
     if (!v)

The weird thing is that this check isn't even necessary:

>>> class CustomSyntaxError(SyntaxError):
...     def __init__(self, msg, filename, lineno, offset, text, end_lineno, end_offset):
...         self.msg = msg
...         self.filename = filename
...         self.lineno = lineno
...         self.offset = offset
...         self.text = text
...         # self.end_lineno = end_lineno
...         # self.end_offset = end_offset
... 
>>> raise CustomSyntaxError("Error message", None, 1, 5, 'a = sin(3)', 1, 9)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<string>", line 1
    a = sin(3)
        ^
CustomSyntaxError: Error message

[3.12] gh-111609: `end_offset` is ignored in subclasses of SyntaxError (

gh-111609: Test `end_offset` in SyntaxError subclass (#127830)

pythongh-111609: Test `end_offset` in SyntaxError subclass (pythonGH-…

pythongh-111609: Test `end_offset` in SyntaxError subclass (python#12…

pythongh-111609: Test `end_offset` in SyntaxError subclass (pythonGH-…

[3.13] gh-111609: Test `end_offset` in SyntaxError subclass (GH-127830)…

This can be closed.

Navigation Menu

Repository navigation

`end_offset` is ignored in subclasses of SyntaxError (3.12-only) #111609

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

end_offset is ignored in subclasses of SyntaxError (3.12-only) #111609

Description

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

Activity

sobolevn commented on Nov 1, 2023

harshitrathor3 commented on Nov 1, 2023

stbech commented on Nov 2, 2023

zedzhen commented on Apr 8, 2024

zedzhen commented on Apr 8, 2024

nineteendo commented on Jul 19, 2024

picnixz commented on Dec 2, 2024

nineteendo commented on Dec 2, 2024

picnixz commented on Dec 2, 2024

nineteendo commented on Dec 3, 2024

nineteendo commented on Dec 3, 2024

nineteendo commented on Dec 3, 2024

nineteendo commented on Mar 17, 2025

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

`end_offset` is ignored in subclasses of SyntaxError (3.12-only) #111609