Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ There is no unit test framework. The automated checks are `test:protocol` (plain
**Error hints.** `src/runtime/hints.py` (loaded first) maps an exception to a hint for the unit's common mistakes, matched on type *and* message; anything else gets none. Run, REPL, the visualizer (`errorHint`) and the doctest runner report it as data (`{kind, ...names}`); `CoreStrings.errorHint` words it, `ErrorHintNote` shows it below Python's unchanged message. New kinds need a case in `check-runtime-hints.py` and `check-error-hints.mjs`.

**Python execution lives in a web worker.** `src/pyodide-worker.js` loads Pyodide from the jsDelivr CDN and inlines the Python files under `src/runtime/` via `?raw` imports:
- `session.py` — invalidates cached modules between runs and runs the main file by `exec` in a plain dict, `_glpython_repl_namespace`, which REPL inputs (`_glpython_repl`) reuse. It's not `runpy`, so the file's functions and the REPL share one global namespace. Also has `_glpython_error_message` (a Python-style last traceback line), which `visualizer.py` uses too.
- `session.py` — invalidates cached modules between runs and runs the main file by `exec` in `_glpython_repl_namespace`, which REPL inputs (`_glpython_repl`) reuse. It's not `runpy`, so the file's functions and the REPL share one global namespace. That namespace is a fresh module installed as `sys.modules["__main__"]` (`_glpython_new_main`), so `import __main__` sees the file; a visualizer trace and a replay install one of their own and restore the previous one. Also has `_glpython_error_message` (a Python-style last traceback line), which `visualizer.py` uses too.
- `gturtle.py` — the `gturtle` module students import; it just records drawing commands, which are returned as JSON and rendered by `GraphicsWindow.tsx`.
- `input_bridge.py` — patches `input()` to block the worker via `Atomics.wait` on a `SharedArrayBuffer`. Only works when the whole browsing context is cross-origin isolated, so it's reported as an optional capability (`capabilities.input`) — don't assume it's available; without it, `input()` raises `RuntimeError: input() is not available here` (hint `input-unavailable`). The page asks in the console (`input_request` → the `>>>` line becomes the program's input line, `programInput`); after the answer the bridge prints prompt + answer as one line (Pyodide's `batched` stdout only passes whole lines, so the prompt isn't printed before the wait), and a cancelled wait (Stop, Ctrl+D) raises `EOFError`. Before blocking, the worker sends the output it still holds (`activeStream.flush()`). `check-runtime-input.py` checks the bridge with a stub for `js`.
- `visualizer.py` — a hand-written tree-walking interpreter over `ast.parse()` output. Real CPython tracing can't expose intermediate expression values, so this interpreter records fine-grained trace steps (keyed by AST `nodeId`) for supported node types via `_STMT_HANDLERS` / `_EXPR_HANDLERS` dispatch dicts; unsupported nodes fall back to compiling/exec'ing that single node (correct result, but one opaque step). Adding fine-grained support for a construct = adding one handler.
Expand Down
6 changes: 5 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -254,7 +254,11 @@ as Reset does, with `{ id, type: 'reset' }`), and learn when Python is busy thro
### REPL

Below the console, a `>>>` input line runs Python in the namespace of the
last Run, so learners can call the file's functions directly. An expression
last Run, so learners can call the file's functions directly. As in
`python3 -i main.py`, that namespace is the module `__main__`: the file runs
with `__name__ == "__main__"`, and `import __main__; __main__.x` sees its
variables, in the REPL, the visualizer and the Tests' Run too (before any
Run, the REPL's own namespace is `__main__`). An expression
shows its value, except `None`, as in CPython's REPL (so a call of a function
without `return`, `print(...)` or `help(...)` shows only what it printed); a
statement such as `a = 5` shows
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@
"test:protocol": "npm run build && node scripts/check-embed-protocol.mjs",
"test:hints": "node scripts/check-error-hints.mjs",
"test:console": "node scripts/check-console-browser.mjs",
"test:runtime": "python3 -B scripts/check-runtime-columns.py && python3 -B scripts/check-runtime-values.py && python3 -B scripts/check-runtime-doctests.py && python3 -B scripts/check-runtime-coverage.py && python3 -B scripts/check-runtime-calls.py && python3 -B scripts/check-runtime-hints.py && python3 -B scripts/check-runtime-fidelity.py && python3 -B scripts/check-runtime-replay.py && python3 -B scripts/check-runtime-imports.py && python3 -B scripts/check-runtime-repl.py && python3 -B scripts/check-runtime-input.py && python3 -B scripts/check-runtime-lists.py",
"test:runtime": "python3 -B scripts/check-runtime-columns.py && python3 -B scripts/check-runtime-values.py && python3 -B scripts/check-runtime-doctests.py && python3 -B scripts/check-runtime-coverage.py && python3 -B scripts/check-runtime-calls.py && python3 -B scripts/check-runtime-hints.py && python3 -B scripts/check-runtime-fidelity.py && python3 -B scripts/check-runtime-replay.py && python3 -B scripts/check-runtime-imports.py && python3 -B scripts/check-runtime-repl.py && python3 -B scripts/check-runtime-input.py && python3 -B scripts/check-runtime-lists.py && python3 -B scripts/check-runtime-main.py",
"prepare": "npm run build"
},
"peerDependencies": {
Expand Down
20 changes: 20 additions & 0 deletions scripts/check-console-browser.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
// visualizer), and without isolation stops with an error and its hint.
// The visualizer (P39): a named list indexed in one step, a long list
// abbreviated in the step and the variables and shown in full on request.
// The run file is the module __main__ (P37), also in the REPL after it.
//
// Drives the built embed (dist-embed/, from npm run build:embed) in Chrome
// through its protocol, as the LiaScript macro does; Pyodide comes from its
Expand Down Expand Up @@ -281,6 +282,25 @@ try {
}
await page.close()
}
// P37: the run file is __main__, in Pyodide too.
{
const page = await openEmbed({})
const frame = page.frameLocator('#embed')
try {
await page.evaluate((source) => window.send('run', source), 'x = 2\nimport __main__\nprint("x ist", __main__.x)\nif __name__ == "__main__":\n print("als Programm")\n')
await frame.locator('.console').getByText('als Programm').waitFor({ timeout: 60000 })
assert.match(await consoleText(frame), /x ist 2/, 'after a Run')
await enterRepl(frame, 'y = 5')
await enterRepl(frame, '__main__.y * 10')
await frame.locator('.console').getByText('50', { exact: true }).waitFor({ timeout: 30000 })
console.log('__main__: the run file and the REPL: ok')
} catch (error) {
failed += 1
console.error(error.message)
console.error('console:', JSON.stringify(await consoleText(frame).catch(() => '?')))
}
await page.close()
}
// Without isolation (as in LiaScript): Python's error and the hint.
{
const page = await openEmbed({})
Expand Down
131 changes: 131 additions & 0 deletions scripts/check-runtime-main.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
"""Regression test: the file that runs is the module "__main__" (P37), as in
"python3 main.py": "import __main__; __main__.x" sees its variables, after a
Run, in the REPL after it and in the visualizer; "if __name__ == '__main__':"
runs; and a function of another file still reads the caller's variables with
"sys._getframe(1).f_globals", as the course does (after a Run and in the
REPL; in the visualizer that frame is the interpreter's), or through
"__main__". A visualizer run and a replay don't replace the REPL's own
"__main__".

Runs the runtime with CPython (no Pyodide needed):
python3 -B scripts/check-runtime-main.py
"""
import contextlib
import io
import json
import os
import subprocess
import sys
import tempfile

ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
RUNTIME = os.path.join(ROOT, "src", "runtime")
sys.path.insert(0, RUNTIME) # for "import gturtle"

# Loaded as Pyodide loads them, so their frames are named "<exec>" there too.
runtime = {"__name__": "glpython_runtime"}
for name in ("hints.py", "session.py", "visualizer.py"):
with open(os.path.join(RUNTIME, name), encoding="utf-8") as handle:
exec(compile(handle.read(), "<exec>", "exec"), runtime)

failures = []


def expect(label, got, expected):
if got != expected:
failures.append(f"{label}: got {got!r}, expected {expected!r}")


MAIN = '''import sys
x = 2
import __main__
print(__main__.x)
print(__main__)
print(__main__ is sys.modules["__main__"], __name__)
if __name__ == "__main__":
print("als Programm")
import hilfe
print(hilfe.lies("x"), hilfe.lies_main("x"))
'''
HILFE = '''import sys

def lies(name):
return sys._getframe(1).f_globals[name]

def lies_main(name):
import __main__
return getattr(__main__, name)
'''
folder = os.path.realpath(tempfile.mkdtemp(prefix="glpython-main-"))
for path, source in (("main.py", MAIN), ("hilfe.py", HILFE)):
with open(os.path.join(folder, path), "w", encoding="utf-8") as handle:
handle.write(source)
main_path = os.path.join(folder, "main.py")

# A Run prints what CPython prints.
cpython = subprocess.run([sys.executable, "-B", "main.py"], cwd=folder, capture_output=True, text=True)
expected = cpython.stdout.replace(f"'{folder}/", "'/")
sys.path.insert(0, folder)
printed = io.StringIO()
with contextlib.redirect_stdout(printed):
runtime["_glpython_run_project"](main_path)
expect("Run", printed.getvalue().replace(f"'{folder}/", "'/"), expected)
expect("Run, the whole output", expected, "2\n<module '__main__' from '/main.py'>\nTrue __main__\nals Programm\n2 2\n")


def repl(source):
with contextlib.redirect_stdout(io.StringIO()):
return json.loads(runtime["_glpython_repl"](source)).get("value")


# The REPL after the Run: the same module, in both directions.
expect("REPL: __main__.x", repl("__main__.x"), "2")
repl("y = 5")
expect("REPL: __main__.y", repl("__main__.y"), "5")
repl("__main__.z = 7")
expect("REPL: z", repl("z"), "7")
expect("REPL: _getframe from another file", repl('hilfe.lies("y")'), "5")
expect("REPL: __main__ from another file", repl('hilfe.lies_main("y")'), "5")
repl_main = sys.modules["__main__"]
expect("REPL: the namespace is the module's", repl_main.__dict__ is runtime["_glpython_repl_namespace"], True)

# The visualizer runs the file as "__main__" too (its __file__ is the
# editor's name), and gives the REPL's module back afterwards.
trace_source = MAIN.replace("print(__main__)\n", "").replace('hilfe.lies("x"), ', "")
result = json.loads(runtime["_glpython_trace_program"]("main.py", {"main.py": trace_source, "hilfe.py": HILFE}, None))
expect("visualizer", (result["output"], result["errorMessage"]), ("2\nTrue __main__\nals Programm\n2\n", None))
expect("visualizer: the REPL's __main__ is back", sys.modules["__main__"] is repl_main, True)

# A traced REPL input (a dry run) sees the file's variables through
# __main__, and changing them changes nothing in the session.
result = json.loads(runtime["_glpython_run_visual_repl"]("__main__.y = 99", json.dumps({"main.py": MAIN, "hilfe.py": HILFE}), "main.py"))
expect("visual REPL: no error", result["errorMessage"], None)
expect("visual REPL: a dry run", (repl("y"), sys.modules["__main__"] is repl_main), ("5", True))
result = json.loads(runtime["_glpython_run_visual_repl"]("print(__main__.y)", json.dumps({"main.py": MAIN, "hilfe.py": HILFE}), "main.py"))
expect("visual REPL: __main__.y", result["output"], "5\n")

# A replay runs the file and the inputs in a "__main__" of its own, then
# restores the session's.
runtime["_glpython_replay_begin"](json.dumps([main_path, os.path.join(folder, "hilfe.py")]))
with contextlib.redirect_stdout(io.StringIO()):
runtime["_glpython_replay_file"](main_path)
replayed = json.loads(runtime["_glpython_replay_input"]("__main__.x")).get("value")
expect("replay: __main__.x", replayed, "2")
expect("replay: a module of its own", sys.modules["__main__"] is not repl_main, True)
runtime["_glpython_replay_end"]()
expect("replay: the session's __main__ is back", sys.modules["__main__"] is repl_main, True)

# Before any Run, the REPL is "__main__", as Python's own.
fresh = {"__name__": "glpython_runtime"}
for name in ("hints.py", "session.py"):
with open(os.path.join(RUNTIME, name), encoding="utf-8") as handle:
exec(compile(handle.read(), "<exec>", "exec"), fresh)
with contextlib.redirect_stdout(io.StringIO()):
fresh["_glpython_repl"]("a = 1")
value = json.loads(fresh["_glpython_repl"]("import __main__; print(__main__.a)"))
expect("REPL before a Run", value.get("error"), None)

if failures:
print("\n".join(failures))
sys.exit(f"{len(failures)} __main__ check(s) failed.")
print("Runtime __main__ checks passed.")
2 changes: 1 addition & 1 deletion scripts/check-runtime-replay.py
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ def session(files, inputs, label):
run_output = ""
if not files:
# A worker where nothing ran yet.
runtime["_glpython_repl_namespace"] = {"__name__": "__main__", "__builtins__": builtins}
runtime["_glpython_repl_namespace"] = runtime["_glpython_new_main"]()
else:
try:
_, run_output = captured(runtime["_glpython_run_project"], main)
Expand Down
26 changes: 20 additions & 6 deletions src/runtime/session.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,22 @@
import sys
import traceback

# A fresh module "__main__", installed as sys.modules["__main__"], and its
# namespace: as in Python, the file that runs (or the REPL before any Run)
# is that module, so "import __main__; __main__.x" sees its variables.
def _glpython_new_main(main_path=None):
import types
module = types.ModuleType("__main__")
if main_path is not None:
module.__file__ = main_path
module.__builtins__ = builtins
sys.modules["__main__"] = module
return module.__dict__

# The namespace the last Run executed the main file in. REPL inputs run in it
# afterwards (like "python -i"), so learners can call the file's functions;
# the next Run replaces it with a fresh one.
_glpython_repl_namespace = {"__name__": "__main__", "__builtins__": builtins}
_glpython_repl_namespace = _glpython_new_main()

# The project's files are mounted at "/" (and gturtle.py next to them), so a
# REPL input can import them even before the first Run - e.g. right after a
Expand Down Expand Up @@ -100,7 +112,7 @@ def _glpython_run_project(main_path):
global _glpython_repl_namespace, _glpython_last_error_report
# Assigned before running, so a file that fails halfway still leaves its
# earlier definitions callable.
_glpython_repl_namespace = {"__name__": "__main__", "__file__": main_path, "__builtins__": builtins}
_glpython_repl_namespace = _glpython_new_main(main_path)
_glpython_last_error_report = None
try:
_glpython_exec_main(main_path, _glpython_repl_namespace)
Expand Down Expand Up @@ -188,19 +200,20 @@ def _glpython_replay_project_modules(paths):
def _glpython_replay_begin(project_paths_json):
global _glpython_replay_namespace, _glpython_replay_saved, _glpython_replay_used_input
paths = set(json.loads(project_paths_json))
original_main = sys.modules.get("__main__")
saved = _glpython_replay_project_modules(paths)
for name in saved:
del sys.modules[name]
_glpython_replay_saved = (saved, paths, builtins.input)
_glpython_replay_saved = (saved, paths, builtins.input, original_main)
builtins.input = _glpython_replay_blocked_input
_glpython_replay_used_input = False
# Inputs typed before any Run ran in an empty namespace.
_glpython_replay_namespace = {"__name__": "__main__", "__builtins__": builtins}
_glpython_replay_namespace = _glpython_new_main()

# Python's traceback, as the Run reported it, or "" if the file ran through.
def _glpython_replay_file(main_path):
global _glpython_replay_namespace
_glpython_replay_namespace = {"__name__": "__main__", "__file__": main_path, "__builtins__": builtins}
_glpython_replay_namespace = _glpython_new_main(main_path)
try:
_glpython_exec_main(main_path, _glpython_replay_namespace)
return ""
Expand All @@ -217,10 +230,11 @@ def _glpython_replay_end():
global _glpython_replay_namespace, _glpython_replay_saved
if _glpython_replay_saved is None:
return
saved, paths, original_input = _glpython_replay_saved
saved, paths, original_input, original_main = _glpython_replay_saved
for name in _glpython_replay_project_modules(paths):
del sys.modules[name]
sys.modules.update(saved)
sys.modules["__main__"] = original_main
builtins.input = original_input
_glpython_replay_namespace = None
_glpython_replay_saved = None
18 changes: 15 additions & 3 deletions src/runtime/visualizer.py
Original file line number Diff line number Diff line change
Expand Up @@ -536,12 +536,16 @@ def _glpython_fallback_exec(node, scope):
ast.fix_missing_locations(module)
code = compile(module, "<glpython>", "exec")
env = _glpython_flatten_scope(scope)
before = dict(env)
exec(code, env)
# Write back any names the fallback statement touched so subsequent
# hand-rolled statements see the effects (best-effort: module-level and
# local names only, matching this interpreter's simplified scoping model).
# A dunder name only if the statement set it ("import __main__").
for name, value in env.items():
if not name.startswith("__") and (name in scope.vars or scope.parent is None):
if name.startswith("__") and (name == "__builtins__" or (name in before and before[name] is value)):
continue
if name in scope.vars or scope.parent is None:
scope.vars[name] = value

def _glpython_summary_for_node(node):
Expand Down Expand Up @@ -1550,7 +1554,11 @@ def _glpython_fill_scope_from_repl(scope, trees, main_path, values):
for name, value in values.items():
if name == "__builtins__":
continue
if isinstance(value, _glpython_types.FunctionType):
if isinstance(value, _glpython_types.ModuleType) and value.__dict__ is values:
# "import __main__" in the file: the traced module, so that the input
# stays a dry run (see _glpython_trace_program).
value = sys.modules["__main__"]
elif isinstance(value, _glpython_types.FunctionType):
found = defs.get((value.__code__.co_filename, value.__name__, value.__code__.co_firstlineno))
if found is not None:
# _GlpythonFunction takes the file its "def" lives in from here.
Expand Down Expand Up @@ -1647,6 +1655,9 @@ def _glpython_trace_program(main_path, sources, fill_scope, report_value=False):
# swallowing all of that run's output ("Keine Ausgabe" bug).
original_stdout = sys.stdout
original_stderr = sys.stderr
# The traced file is the module "__main__" while it runs, as in a Run (see
# _glpython_new_main); the REPL's own one comes back afterwards.
original_main = sys.modules.get("__main__")
capture = _GlpythonCapture()
sys.stdout = capture
sys.stderr = capture
Expand All @@ -1657,7 +1668,7 @@ def _glpython_trace_program(main_path, sources, fill_scope, report_value=False):
truncated = False
value = None
try:
module_vars = {"__name__": "__main__", "__file__": main_path}
module_vars = _glpython_new_main(main_path)
_glpython_module_scope = _GlpythonScope(module_vars, parent=None, kind="module")
if fill_scope is not None:
fill_scope(_glpython_module_scope, trees)
Expand Down Expand Up @@ -1690,6 +1701,7 @@ def _glpython_trace_program(main_path, sources, fill_scope, report_value=False):
finally:
sys.stdout = original_stdout
sys.stderr = original_stderr
sys.modules["__main__"] = original_main

return json.dumps({
"trace": _glpython_trace_log,
Expand Down
Loading