Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,19 +1,21 @@
# typedpython: compiled
import array


@compiled
def dot_arr(a: array.array[float], b: array.array[float]) -> float:
s: float = 0.0
for i in range(len(a)):
s += a[i] * b[i]
return s


@compiled
def scale_arr(a: array.array[float], k: float) -> None:
for i in range(len(a)):
a[i] = a[i] * k


@compiled
def dot_list(a: list[float], b: list[float]) -> float:
s: float = 0.0
for i in range(len(a)):
Expand Down
16 changes: 14 additions & 2 deletions benchmarks/typedpython/run.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,14 +10,15 @@
The "TypedPython @compiled" column runs only with `--typedpython <dir>` (the directory that holds
the `typedpython` package, python-multiplatform-ksp/src/main/python) and `--typedpython-python` (an
interpreter with the gate's dependencies, e.g. that module's .venv): each benchmark is copied with
the `# typedpython: compiled` marker, compiled through `typedpython.pipeline`, and its `main()` is
`@compiled` above every top-level def and class, compiled through `typedpython.pipeline`, and its `main()` is
run from the built extension. Without the option the column reports n/a.

binary-trees declares `__slots__` (TypedPython compiles fixed-layout classes only, SPEC N-11). The
speedup is reported against that CPython version; `CPython (no __slots__)` runs the same program
with the `__slots__` line removed, so what adding the slots changes in CPython is recorded too.
"""
import argparse
import ast
import datetime
import json
import os
Expand Down Expand Up @@ -87,6 +88,17 @@ def build() -> None:
"""


def with_compiled(source: str) -> str:
"""`source` with `@compiled` above every top-level def and class (above their other decorators)."""
tree = ast.parse(source)
lines = source.split("\n")
firsts = [min([n.lineno] + [d.lineno for d in n.decorator_list]) for n in tree.body
if isinstance(n, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef))]
for first in reversed(firsts):
lines.insert(first - 1, "@compiled")
return "\n".join(lines)


def build_compiled(tp_dir: Path, tp_python: str) -> dict[str, tuple[str, list[str]] | str]:
"""Compile each benchmark with TypedPython; name -> (extension, compiled functions) or a reason."""
built: dict[str, tuple[str, list[str]] | str] = {}
Expand All @@ -95,7 +107,7 @@ def build_compiled(tp_dir: Path, tp_python: str) -> dict[str, tuple[str, list[st
work = TP_BUILD / name
work.mkdir(parents=True, exist_ok=True)
source = work / py
source.write_text("# typedpython: compiled\n" + (ROOT / "py" / py).read_text())
source.write_text(with_compiled((ROOT / "py" / py).read_text()))
result = subprocess.run([tp_python, "-c", COMPILE_SNIPPET, str(source), str(work / "out")],
capture_output=True, text=True, env=env)
if result.returncode != 0:
Expand Down
2 changes: 1 addition & 1 deletion docs/SPEC.md
Original file line number Diff line number Diff line change
Expand Up @@ -724,7 +724,7 @@ All in `PM/commonTest`, so they run wherever the interpreter loads.
with `.diagnostics`) before the module runs. Results are cached in memory by file content hash, failures
included. `Status: implemented`, `python-multiplatform-ksp/src/test/python/test_importgate.py`.
- **N-8** TypedPython compilation to C (INTENT §1.4; design §4.3; #41). `@compiled` (a builtin name, no
import, #42) and a first-line `# typedpython: compiled` mark functions; the compiler lowers them
import, #42) is the only mark for functions (the first-line module pragma was removed, #353); the compiler lowers them
through the typed IR (`typedpython/ir.py`) to C against the CPython C API and builds an extension with
the platform C compiler. Compiled functions return what CPython returns (IR contract: checked i64
with deopt to the interpreted function, Python floor division/modulo, exact int/float comparison,
Expand Down
6 changes: 2 additions & 4 deletions docs/design/typedpython.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ TypedPython 의 목표는 **새 언어를 만들지 않고, 파이썬 문법 그
|---|---|---|---|
| 0 | 서드파티 패키지 (의존성) | 없음, 검사하지 않는다 | 바이트코드 (pypackpack 이 받은 그대로) |
| 1 `checked` | **사용자 프로젝트의 파이썬 코드 전부, 기본값** | 빌드 타임 타입 오류, `Any` 유출 0 (§4.2) | 바이트코드 |
| 1 `compiled` | `@compiled` (내장 이름, import 없음) 또는 모듈 선언 `# typedpython: compiled` | `checked` + §3.1 금지 목록 | C 확장 모듈 (NEON · AVX 벡터화, §5.3) |
| 1 `compiled` | `@compiled` (내장 이름, import 없음) | `checked` + §3.1 금지 목록 | C 확장 모듈 (NEON · AVX 벡터화, §5.3) |
| 2 `kernel` | `@kernel` | 객체·힙 할당 없음, 고정 dtype | SPIR-V / MSL (직접) / WGSL / PTX, 빌드 타임 (§5) |

- **사용자 코드는 표시하지 않아도 검사된다.** 타입 오류가 있으면 Kotlin 컴파일 에러처럼 빌드가
Expand Down Expand Up @@ -685,9 +685,7 @@ upstream 대비 수치 오차와 커널 시간을 기준선으로 박아 둔다.
인터프리터 시작 시 `builtins.compiled` 를 설치하고(#42), 일반 CPython 에서는 `typedpython` wheel 의
`typedpython_builtins.pth` 가 `builtins` 에 없을 때만 항등 함수를 설치한다(런타임이 넣은 것을 덮어쓰지
않는다). 타입 검사기에는 `__builtins__.pyi` 로 알린다, Pyrefly 1.3.2 는 검색 경로의 그 파일을 추가
내장으로 읽는다(실측). 모듈 단위 표기는 `# typedpython: compiled` 주석을 유지한다: `# pyright: strict` 와 같은
도구 pragma 형태이고, 실행 시 이름 조회가 필요 없어 일반 CPython 에서 부작용이 없으며, 데코레이터 없이
모듈 전체를 지정할 수 있다.
내장으로 읽는다(실측). 모듈 단위 pragma(`# typedpython: compiled`)는 2026-10-30 메인테이너 결정으로 제거했다(#353). `@compiled` 가 컴파일 대상을 표시하는 유일한 수단이다.
2. ~~고정폭 정수의 오버플로 의미~~, **결정됨:** `int` 는 i64 로 컴파일하고 오버플로 시 인터프리터 재실행으로 큰 정수로 승격(의미 보존).
3. ~~pypackpack Cython 슬롯 인터페이스 소유~~, **결정됨:** pypackpack 이 정한다. TypedPython 은 생성한 C 소스, 빌드 플래그, C 를 낸 모듈 목록을 넘긴다. 확장이 원본 소스를 품으므로 별도 `.py` 폴백은 없다(Cython 폐기 후 갱신).
4. ~~GPU 착지를 torchnative 커스텀 op 로 할 것인가~~, **결정됨 (2026-10-04, 메인테이너):** 커널은
Expand Down
3 changes: 2 additions & 1 deletion docs/release/desktop/typedpython/kernels.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# typedpython: compiled
import sys


@compiled
def dot(n: int) -> float:
s = 0.0
for i in range(n):
Expand All @@ -10,6 +10,7 @@ def dot(n: int) -> float:
return s


@compiled
def collatz_steps(limit: int) -> int:
total = 0
for start in range(1, limit):
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
# typedpython: compiled
"""Issue #45: compiled TypedPython code calling Kotlin through the binder.

Every name imported below is a binder proxy (`fixture/app/TypedPythonBridge.kt`, published by
Expand All @@ -21,6 +20,7 @@
from fixture.app import TpSample, tpAccumulate, tpCheck, tpTouch


@compiled
def drive(n: int, scale: float, token: object) -> float:
"""Per step: a Kotlin constructor call that returns a typed proxy, a method call on that proxy,
the proxy handed back to a Kotlin function, and a PyObject handed to Kotlin -- inside native
Expand All @@ -35,6 +35,7 @@ def drive(n: int, scale: float, token: object) -> float:
return total


@compiled
def checked(n: int, limit: float) -> float:
"""`tpCheck` throws in Kotlin once `acc` passes `limit`; the caller sees what Python sees."""
acc: float = 0.0
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ def _build(paths: list[Path], cache: Path, out: Path | None) -> int:
print(f"{m.status:7} {m.name}: {what}{why}")
rel = m.path.relative_to(root).as_posix()
if m.kind == "not_marked":
print(f" {rel}: not marked for compilation (no `# typedpython: compiled`, no @compiled)")
print(f" {rel}: not marked for compilation (no @compiled)")
for item in m.skipped_items:
where = f"{item.file}:{item.line}" if item.line is not None else item.file
print(f" {where}: {item.name}: {item.message}")
Expand Down Expand Up @@ -112,6 +112,8 @@ def _demo(module: Path, module_argv: list[str], out: Path) -> int:
return EXIT_FAILURE

def interpreted() -> object:
import builtins # the module's `@compiled` is a builtin name; typedpython_builtins.pth installs it
builtins.compiled = getattr(builtins, "compiled", lambda f, /: f)
spec = importlib.util.spec_from_file_location(module.stem + "_interpreted", module)
assert spec is not None and spec.loader is not None
mod = importlib.util.module_from_spec(spec)
Expand Down
26 changes: 7 additions & 19 deletions python-multiplatform-ksp/src/main/python/typedpython/frontend.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,7 @@
`lower(path)` reads one module and returns an `ir.Module`:

- Which functions: module-level `def`s decorated with the builtin `@compiled` (a bare name, no
import, #42), or every module-level `def` when the first non-empty line is
`# typedpython: compiled`.
import, #42). That decorator is the only mark; there is no module-level pragma (#353).
- A function is lowered only when **all** of it is expressible in the IR; otherwise it is left out
and `Module.skipped[name]` says why, with the line. There is no partial lowering.
- Types: parameters and returns from their annotations (`int`/`float`/`bool`, `list[float]` and
Expand Down Expand Up @@ -80,8 +79,6 @@
from typedpython import gate, ir, pyrefly, rebinding, resume, taint
from typedpython.ir import Type

MARKER = "# typedpython: compiled"

I64_MIN, I64_MAX = -2**63, 2**63 - 1
TOP = (I64_MIN, I64_MAX)
EXACT_INT_FLOAT = 2**53 # |n| <= this: int/int true division is exact-rounded in C (ir.BinOp)
Expand Down Expand Up @@ -249,7 +246,7 @@ def lower(path: Path) -> ir.Module:

def _parallel_marked(tree: ast.Module, source: str) -> set[str]:
"""Module-level functions the user marked for SPEC N-15: the comment line `# typedpython: parallel`
directly above the `def` (above its decorators if it has any). A comment, like the module marker,
directly above the `def` (above its decorators if it has any). A comment,
so the interpreted module is unchanged and needs no runtime name."""
lines = source.splitlines()
out: set[str] = set()
Expand Down Expand Up @@ -434,17 +431,10 @@ def _binding_targets(stmt: ast.stmt) -> list[ast.expr]:
SLOT_ACCESS_DUNDERS = ("__getattribute__", "__getattr__", "__setattr__", "__delattr__", "__new__")


def _whole_module(source: str) -> bool:
return next((line.strip() for line in source.splitlines() if line.strip()), "") == MARKER


def is_opted_in(path: Path) -> bool:
"""Whether the front end considers the module for compilation at all: the `# typedpython: compiled`
marker as its first non-empty line, or a module-level `def` / `class` decorated `@compiled`
(the same tests `_targets` and `_classes` apply)."""
"""Whether the front end considers the module for compilation at all: a module-level `def` / `class`
decorated `@compiled` (the same test `_targets` and `_classes` apply)."""
source = Path(path).read_text()
if _whole_module(source):
return True
return any(
isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef))
and any(isinstance(d, ast.Name) and d.id == "compiled" for d in node.decorator_list)
Expand All @@ -454,14 +444,13 @@ def is_opted_in(path: Path) -> bool:
def _classes(
tree: ast.Module, source: str, info: _ModuleInfo,
) -> tuple[dict[str, ir.ClassDecl], dict[str, str]]:
whole = _whole_module(source)
decls: dict[str, ir.ClassDecl] = {}
skipped: dict[str, str] = {}
for node in tree.body:
if not isinstance(node, ast.ClassDef):
continue
marked = any(isinstance(d, ast.Name) and d.id == "compiled" for d in node.decorator_list)
if not (whole or marked):
if not marked:
continue
result = _class_decl(node, tree, info)
if isinstance(result, str):
Expand Down Expand Up @@ -608,13 +597,12 @@ def _class_of(kind: str, classes: dict[str, ir.ClassDecl]) -> tuple[str, bool] |
def _targets(
tree: ast.Module, source: str, info: _ModuleInfo,
) -> list[tuple[ast.FunctionDef | ast.AsyncFunctionDef, str | None]]:
whole_module = _whole_module(source)
found: list[tuple[ast.FunctionDef | ast.AsyncFunctionDef, str | None]] = []
for node in tree.body:
if not isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):
continue
marked = [d for d in node.decorator_list if isinstance(d, ast.Name) and d.id == "compiled"]
if not (whole_module or marked):
if not marked:
continue
reason = None
if marked and info.bound("compiled"):
Expand Down Expand Up @@ -2499,4 +2487,4 @@ def _breaks(body: tuple[ir.Stmt, ...]) -> bool:
return False


__all__ = ["lower", "MARKER"]
__all__ = ["lower"]
24 changes: 24 additions & 0 deletions python-multiplatform-ksp/src/test/python/conftest.py
Original file line number Diff line number Diff line change
@@ -1,12 +1,36 @@
import ast
import builtins
import sys
from pathlib import Path
import textwrap

import pytest

# The `typedpython_builtins.pth` of an installed wheel; the test interpreter has no such install.
builtins.compiled = getattr(builtins, "compiled", lambda f, /: f)

# Prepended to every child-process script that loads a module whose source uses `@compiled`: a plain
# CPython has the builtin only when `typedpython_builtins.pth` is installed, so the child installs it.
COMPILED_SHIM = "import builtins\nif not hasattr(builtins, 'compiled'):\n builtins.compiled = lambda f: f\n"

FIXTURES = Path(__file__).parent / "fixtures"


def decorate_compiled(source: str) -> str:
"""`source` with `@compiled` put above every module-level def / class (above its other decorators),
the set a whole-module mark used to cover. A source that does not parse is returned unchanged."""
try:
tree = ast.parse(source)
except SyntaxError:
return source
lines = source.split("\n")
firsts = [min([n.lineno] + [d.lineno for d in n.decorator_list]) for n in tree.body
if isinstance(n, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef))]
for first in reversed(firsts):
lines.insert(first - 1, "@compiled")
return "\n".join(lines)


@pytest.fixture
def write(tmp_path):
"""Write a dedented source file under tmp_path and return its path."""
Expand Down
Loading