Repository navigation
Store classes themselves in module and class namespaces - #915
craigbarratt merged 2 commits into
Conversation
Bind a class to its name once it has been created, like an assignment, instead of keeping it in an EvalLocalVar that leaked through globals(), module attributes, lambdas and native code. This removes the unwrapping added by custom-components#902. An annotation naming the class being defined is evaluated once the class exists, before its decorators run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comparison with master and CPython 3.14 (22 examples)Unchanged: fixed by #902, or the same as masterThe #875 example: class Foo:
pass
class Bar:
z: Foo | None = None
Bar.z
#master None
#cpython None
#this pr NoneA self-referencing annotation class Node:
nxt: Node = None
Node.__annotations__["nxt"].__name__
#master 'Node'
#cpython 'Node'
#this pr 'Node'A from dataclasses import dataclass, InitVar, fields
@dataclass
class Node:
val: int = 0
parent: InitVar[Node] = None
def __post_init__(self, parent):
pass
[f.name for f in fields(Node)]
#master ['val']
#cpython ['val']
#this pr ['val']Fixed by this PR
class Local:
pass
globals()["Local"] is Local
#master False
#cpython True
#this pr TrueA lambda returning a module-level class class Foo:
pass
get = lambda: Foo
get() is Foo
#master False
#cpython True
#this pr True
class Base:
pass
class Sub(Base):
pass
@pyscript_compile
def check():
return issubclass(Sub, Base)
check()
#master False
#cpython True
#this pr True
class Foo:
pass
@pyscript_compile
def native():
return Foo | None
native() == (Foo | None)
#master TypeError: unsupported operand type(s) for |: 'EvalLocalVar' and 'NoneType'
#cpython True
#this pr TrueA nested class class Outer:
class Inner:
pass
(Outer.Inner | None) == (Outer.Inner | None)
#master TypeError: unsupported operand type(s) for |: 'EvalLocalVar' and 'NoneType'
#cpython True
#this pr TrueA class referring to itself with class Node:
nxt: Node | None = None
Node.nxt
#master TypeError: unsupported operand type(s) for |: 'EvalLocalVar' and 'NoneType'
#cpython None
#this pr NoneA from dataclasses import dataclass
@dataclass
class Node:
val: int = 0
nxt: Node | None = None
Node(1, Node(2)).nxt.val
#master TypeError: unsupported operand type(s) for |: 'EvalLocalVar' and 'NoneType'
#cpython 2
#this pr 2The name after a class body raises try:
class A:
raise ValueError
except ValueError:
pass
"A" in globals()
#master True
#cpython False
#this pr FalseAn Enum with a nested class from enum import Enum
class Color(Enum):
RED = 1
class Helper:
pass
Color.RED.value
#master NameError: name 'Helper' is not defined
#cpython 1
#this pr 1An annotation with a type parameter in a function (#796) def first[T](items: list[T]) -> T:
head: T = items[0]
return head
first([5, 6])
#master NameError: name 'T' is not defined
#cpython 5
#this pr 5
import typing
class Node:
nxt: Node = None
typing.get_type_hints(Node)["nxt"] is Node
#master False
#cpython True
#this pr TrueA from dataclasses import dataclass, fields
from typing import ClassVar
@dataclass
class Node:
registry: ClassVar[dict[str, Node]] = {}
val: int = 0
[f.name for f in fields(Node)]
#master NameError: name 'Node' is not defined
#cpython ['val']
#this pr ['val']Breaking: worked on master, now an error, as in CPythonA method using a nested class by its bare name class Outer:
class Inner:
v = 4
def make(self):
return Inner.v
Outer().make()
#master 4
#cpython NameError: name 'Inner' is not defined
#this pr NameError: name 'Inner.v' is not definedA default value referring to its own class class Node:
def m(self, x=Node):
return x
Node().m() is Node
#master True
#cpython NameError: name 'Node' is not defined
#this pr NameError: name 'Node' is not definedA class attribute referring to its own class class Node:
kind = Node
Node.kind.__name__
#master 'Node'
#cpython NameError: name 'Node' is not defined
#this pr NameError: name 'Node' is not defined
def drop():
global gone
del gone
drop()
"ok"
#master 'ok'
#cpython NameError: name 'gone' is not defined
#this pr NameError: name 'gone' is not defined
try:
from json import nope
r = "ok"
except AttributeError:
r = "AttributeError caught"
r
#master 'AttributeError caught'
#cpython ImportError: cannot import name 'nope' from 'json' (.../json/__init__.py)
#this pr ImportError: cannot import name 'nope' from 'json' (.../json/__init__.py)As in CPython 3.14, a typo in an annotation is no longer reportedA typo in an annotation class C:
x: Undefinedd = 3
C.x
#master NameError: name 'Undefinedd' is not defined
#cpython 3
#this pr 3Better than master, still different from CPythonAn annotation naming a class defined later class A:
b: B = None
class B:
pass
A.__annotations__["b"]
#master NameError: name 'B' is not defined
#cpython <class 'B'>
#this pr 'B' |
craigbarratt
left a comment
There was a problem hiding this comment.
Thanks @dmamelin. Storing the class itself is the right fix, and it's nice to see the unwrapping go.
One small thing: eval_pending_annotations only sees the class name, so an annotation that also uses a class attribute stays a string. For example, class A: Alias = int; x: A | Alias gives 'A | Alias', while CPython gives A | int. Evaluating with {**cls.__dict__, cls_name: cls} fixes it.
@HoneyBearTech, thanks for the tests from #907, which this PR includes. Since it supersedes #907, there's no need to rebase that one.
|
@craigbarratt Good catch, thanks, fixed. One change from your suggestion: I put the class's own names after the class name, |
|
Thanks, looks great! |
I'd been thinking about reworking how classes are stored for a while, and #902 pushed me to a complete solution. I didn't get to present it before #902 was merged, so this PR removes part of it: the code that unwraps classes. Its import and
globalscope changes stay, and all of its tests still pass. Thanks to @HoneyBearTech, for the push, and for the tests in #902 and #907.Root cause
A class defined at module level was kept in the global symbol table as an
EvalLocalVarwrapper. That table is also the module's__dict__and the globals of@pyscript_compilefunctions, so the wrapper leaked wherever the table was read. #902 unwraps it in the places that read it, and #907 does the same for nested classes, butglobals(),getattr(mod, ...), lambdas and native code still get the wrapper.Changes
EvalLocalVaris only used, like CPython's cells, for function locals shared with closures. If the class body, metaclass or a decorator raises, the name keeps its previous binding.nxt: Node | NoneinNode) is evaluated once the class exists, before class decorators run, so@dataclassseesInitVar[Node]orClassVar[...]. An annotation naming something still not defined is kept as source text, as withfrom __future__ import annotations.global x; del xfor an unboundxraisesNameError, andfrom x import missingraisesImportError, for any name, not only for classes.|with another class from the same script (#875) #902 is removed (unwrap_definedand its uses inEvalAttrSet,ast_attribute,assign_name,ast_deleteandast_importfrom):eval.pyis +45/−61 lines: +22/−61 for the change itself, and 23 more lines to evaluate self-referencing annotations once the class exists (without them,nxt: Nodewould stay the string'Node', and@dataclasswouldn't see anInitVar[Node]). All 39 CPython-comparison scenarios from Fix TypeError when a class type hint uses|with another class from the same script (#875) #902 pass without it (29 of them fail on master before Fix TypeError when a class type hint uses|with another class from the same script (#875) #902), since there's no wrapper left to unwrap.Compared with master
On 22 examples, with CPython 3.14 as reference, 20 give the same result as CPython here and 3 on master. The full comparison is in the first comment.
globals(), lambdas,issubclassandX | Nonein@pyscript_compile, nested classes (Store nested classes themselves on the outer class (#875) #907),nxt: Node | NoneinNode,@dataclassreferring to its own class (includingInitVar/ClassVar),typing.get_type_hints(), the name after a class body raises, an Enum with a nested class, andhead: Tin a generic function (part of Errors with type annotation handling in 1.7.0 #796).'B', since CPython 3.14 evaluates annotations lazily (on master it raisesNameError).Breaking changes
All of these follow CPython.
A method can't use a nested class by its bare name:
A class can't be used as a value in its own body:
delof an unbound global raisesNameError:A missing name in
from x import yraisesImportErrorinstead ofAttributeError:Also as in CPython 3.14, a typo in a class or module annotation is no longer reported when the class or module is loaded:
Tests
All in
tests/test_unit_eval.py:globalNameTestscases and 2 nested-class cases, compared with CPython, and anevalTestscase for annotations;evalTestscase that deleted an unbound global is adjusted.Closes #907, whose tests are included.
Risk
Every
classstatement now binds its name differently, and functions no longer capture module-level classes, so this affects all user classes, not only the cases above. Checked: the full test suite, the 39 scenarios from #902, the 22 examples compared with CPython, pre-commit, and my production Home Assistant with its scripts, apps and modules.