Visualizer: a statement call of your own function ends as None

CLAUDE.md

@@ -57,6 +57,7 @@ There is no unit test framework. The automated checks are `test:protocol` (plain
 - **Visualizing a REPL entry replays the session** (`session.py`'s `_glpython_replay_*`, driven by the worker's `replayReplEntry`): the worker keeps only text (the last Run's files and output, each input since with what it printed and its result JSON), reruns the file and the earlier inputs in a namespace of their own and compares each with the original (object addresses masked); then it traces the entry and compares its output, error and value too. `input()` raises during a replay. Any difference, or `input()`, ends in a `repl-replay-note` instead of a trace. The project's modules are set aside and restored, so the live session is untouched. `check-runtime-replay.py` checks this under CPython.
   - `_GlpythonFunction` calls a stub with the same parameters first, so a call with wrong arguments raises Python's own `TypeError` (same message) at the call site.
   - Columns: `ast` counts UTF-8 bytes, `tokenize` characters, the UI UTF-16 code units. Every offset into a line goes through `_glpython_text_before` (the text in front of it, for slicing) or `_glpython_ui_column` (a column for the UI); never slice a line by an `ast` offset directly.
+  - A call that is a whole statement ends as its value, `None` too, if the callee is the learner's own (`_glpython_is_own_function`: a `_GlpythonFunction`, or code from a project file, `<glpython>` or `<repl>`); a built-in's `None` (`print`, `xs.append`, `gturtle`, the stdlib) keeps the call text.
   - Values are shown through `_glpython_safe_repr`, which writes types as learners do (`int`, not `<class 'int'>`); the program's own output keeps Python's repr.
   - It abbreviates long values (`[1, 2, 3, 4, 5, …, 100]`; levels in `_GLPYTHON_ABBREVIATION_LEVELS`) and notes each full text. The next recorded step gets `fullValues` (`{shown text: index}`), the trace result `values` (`{text, cut, length, kind}`). The UI (`fullValuesUpTo`, `ValueText`, `LocalsValue`, `FullValueDialog`) opens one on click.
   - Lists that read as a tree (`listtree`: first element a string): `session.py`'s `_glpython_tree_of` gives `{label, children}`/`{leaf}`, for the REPL's value (`tree`) and, noted like the full values, for the panel's values (step `trees`, result `trees`). The UI draws them with `TreeView` (`src/TreeView.tsx`: HTML nodes, SVG edges, a tree wider than its place scrolls with its root centred and hints at the hidden edges), behind capability `listTree`. The positions come from our port of van der Ploeg's tidy-tree algorithm (`src/treeLayout.ts`, in characters of the code font and in rows; every node one row high), with d3-flextree's changes; `test:layout` compares it with `d3-flextree`, which is a devDependency for that check only and must stay out of the builds.

README.md

@@ -99,6 +99,16 @@ Deep chains of open calls, such as a recursion without a base case, collapse
 their middle boxes into one "… N more calls of f(x) …" marker
 (`visualizerCollapsedCalls`), so the layout stays usable.
 
+A call ends as its value, `None` too: also a call that is a whole statement
+(a line of the program, a REPL input, a doctest example), when it calls the
+learner's own function - a `def` or lambda of the project's files, of a
+library loaded with it, a method of their class - so a function without a
+matching `return` visibly gives `None`. Only a built-in, a method of a
+built-in type or a function of a module that comes with Python or glpython
+(`print(...)`, `xs.append(x)`, `random.shuffle(xs)`, `gturtle`) keeps its call
+text as a statement, where `None` would only be noise. The console still
+shows nothing for `None`, as CPython's REPL.
+
 ### Box and substitution mode
 
 The visualizer's toolbar is one line: first step, previous step, next step,

package.json

@@ -40,7 +40,7 @@
     "test:layout": "node --experimental-strip-types --no-warnings scripts/check-tree-layout.mjs",
     "test:tree": "node --experimental-strip-types --no-warnings scripts/check-tree-mode.mjs",
     "test:svg": "node --experimental-strip-types --no-warnings scripts/check-tree-svg.mjs",
-    "test:runtime": "python3 -B scripts/check-runtime-columns.py && python3 -B scripts/check-runtime-values.py && python3 -B scripts/check-runtime-doctests.py && python3 -B scripts/check-runtime-coverage.py && python3 -B scripts/check-runtime-calls.py && python3 -B scripts/check-runtime-hints.py && python3 -B scripts/check-runtime-fidelity.py && python3 -B scripts/check-runtime-replay.py && python3 -B scripts/check-runtime-imports.py && python3 -B scripts/check-runtime-repl.py && python3 -B scripts/check-runtime-input.py && python3 -B scripts/check-runtime-lists.py && python3 -B scripts/check-runtime-main.py && python3 -B scripts/check-runtime-trees.py && python3 -B scripts/check-runtime-box-reasons.py",
+    "test:runtime": "python3 -B scripts/check-runtime-columns.py && python3 -B scripts/check-runtime-values.py && python3 -B scripts/check-runtime-doctests.py && python3 -B scripts/check-runtime-coverage.py && python3 -B scripts/check-runtime-calls.py && python3 -B scripts/check-runtime-hints.py && python3 -B scripts/check-runtime-fidelity.py && python3 -B scripts/check-runtime-replay.py && python3 -B scripts/check-runtime-imports.py && python3 -B scripts/check-runtime-repl.py && python3 -B scripts/check-runtime-input.py && python3 -B scripts/check-runtime-lists.py && python3 -B scripts/check-runtime-main.py && python3 -B scripts/check-runtime-trees.py && python3 -B scripts/check-runtime-box-reasons.py && python3 -B scripts/check-runtime-none-calls.py",
     "prepare": "npm run build"
   },
   "peerDependencies": {

scripts/check-console-browser.mjs

@@ -735,6 +735,76 @@ try {
     }
     await page.close()
   }
+  // P49: a call of the learner's own function that returns None shows None
+  // as its last step - a line of the program, a REPL input, a failing
+  // doctest example - in each mode; print(...) and xs.append(...) keep their
+  // call text. The console prints nothing for None.
+  {
+    const anwenden = "def anwenden(op, x, y):\n    \"\"\"\n    >>> anwenden('*', 3, 4)\n    12\n    \"\"\"\n    if op == '+':\n        return x + y\n"
+    const page = await openEmbed({}, '?features=run,repl,visualize,tests,tree&windowHeight=900')
+    const frame = page.frameLocator('#embed')
+    const mode = (name) => frame.locator('.visualizer-mode button', { hasText: name }).click()
+    // The root box's code at each step, a tree as its text.
+    const lines = () => frame.locator('.visualizer-view > .visualizer-canvas > .frame-box > .frame-code').evaluate((code) => {
+      for (const tree of code.querySelectorAll('.value-tree-scroll')) tree.replaceWith(`[${tree.getAttribute('aria-label')}]`)
+      return code.innerText
+    })
+    const steps = async () => {
+      await measured(frame)
+      const first = frame.locator('.visualizer-controls button').nth(0)
+      if (await first.isEnabled()) await first.click()
+      const seen = [await lines()]
+      const next = frame.locator('.visualizer-controls button').nth(2)
+      while (await next.isEnabled()) {
+        await next.click()
+        seen.push(await lines())
+      }
+      return seen
+    }
+    // The line of the root box's code that holds the call (the box's own
+    // line numbers are separate lines of innerText).
+    const lineOf = (text, call) => text.split('\n').find((line, index, all) => line === call || all[index - 1] === call)
+    for (const name of ['Kasten', 'Ersetzung', 'Baum']) {
+      try {
+        await page.evaluate((source) => window.send('visualize', source), `${anwenden}\nxs = []\nanwenden('*', 3, 4)\nprint('x')\nxs.append(1)\n`)
+        await frame.locator('.visualizer-step-count', { hasText: 'Vor der' }).waitFor({ timeout: 60000 })
+        await mode(name)
+        const seen = await steps()
+        const last = seen.at(-1).split('\n')
+        assert.equal(last[last.indexOf('10') + 1], 'None', 'a line of the program: anwenden(...) ends as None')
+        assert.equal(last[last.indexOf('11') + 1], "print('x')", 'print(...) keeps its text')
+        assert.equal(last[last.indexOf('12') + 1], 'xs.append(1)', 'xs.append(...) keeps its text')
+        assert.ok(seen.every((text) => !text.includes('[None]')), 'no tree of None alone')
+
+        await page.evaluate((source) => window.send('run', source), anwenden)
+        await frame.locator('.console').getByText('(Keine Ausgabe)').waitFor({ timeout: 60000 })
+        await enterRepl(frame, "anwenden('*', 3, 4)")
+        const entry = frame.locator('.repl-entry').last()
+        await entry.locator('.repl-entry-visualize:not([disabled])').waitFor({ timeout: 30000 })
+        assert.equal(await entry.locator('.repl-value').count(), 0, 'the console shows nothing for None')
+        assert.equal(await entry.innerText().then((text) => text.replace('Visualisieren', '').trim()), ">>> anwenden('*', 3, 4)", 'nothing printed')
+        await entry.locator('.repl-entry-visualize').click()
+        await mode(name)
+        const repl = (await steps()).map((text) => lineOf(text, "anwenden('*', 3, 4)") ?? text)
+        assert.equal(repl[0], "anwenden('*', 3, 4)", 'the REPL input starts as the call')
+        assert.equal(repl.at(-1).split('\n').at(-1), 'None', 'the REPL input ends as None')
+
+        await page.evaluate((source) => window.send('test', source), anwenden)
+        await frame.getByRole('button', { name: 'Beispiel visualisieren' }).click()
+        await frame.locator('.visualizer-step-count', { hasText: 'Vor der' }).waitFor({ timeout: 60000 })
+        await mode(name)
+        const example = await steps()
+        // The call ends as None, the comparison below it.
+        assert.deepEqual(example.at(-1).split('\n').slice(-2), ['None', 'Erhalten: (Keine Ausgabe) · Erwartet: 12'], 'the failing example ends as None')
+        assert.equal(lineOf(example[0], "anwenden('*', 3, 4)"), "anwenden('*', 3, 4)", 'the example starts as the call')
+        console.log(`None of an own call, «${name}»: a line, a REPL input, a doctest example: ok`)
+      } catch (error) {
+        failed += 1
+        console.error(`None of an own call, «${name}»: ${error.message}`)
+      }
+    }
+    await page.close()
+  }
   // P37: the run file is __main__, in Pyodide too.
   {
     const page = await openEmbed({})

scripts/check-runtime-none-calls.py

@@ -0,0 +1,144 @@
+"""Regression test (P49): a call that is a whole statement and returns None.
+A call of the learner's own function shows its None as the last step - in a
+program, a REPL input and a doctest example alike: a "def" of the file, of
+another file of the project, a lambda, a method of their class, a function
+from an earlier REPL input. A built-in, a method of a built-in type and a
+function of a module glpython or Python brings along (gturtle, random)
+keep their call text. A call whose value is used shows None as before.
+
+Runs the runtime with CPython (no Pyodide needed):
+    python3 -B scripts/check-runtime-none-calls.py
+"""
+import contextlib
+import io
+import json
+import os
+import sys
+import types
+
+ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+RUNTIME = os.path.join(ROOT, "src", "runtime")
+sys.path.insert(0, RUNTIME)  # for "import gturtle"
+
+runtime = {"__name__": "glpython_runtime"}
+for name in ("hints.py", "session.py", "visualizer.py"):
+    with open(os.path.join(RUNTIME, name), encoding="utf-8") as handle:
+        exec(compile(handle.read(), "<exec>", "exec"), runtime)
+
+failures = []
+
+
+def expect(label, got, expected):
+    if got != expected:
+        failures.append(f"{label}: got {got!r}, expected {expected!r}")
+
+
+def call_values(result, path, source):
+    """(source text, shown value) of each call's last step in the file, in
+    order; the value is None where the call keeps its text."""
+    lines = source.split("\n")
+    return [
+        (lines[step["line"] - 1][step["col"]:step["endCol"]], step["valueText"])
+        for step in result["trace"]
+        if step["kind"] == "eval" and step["path"] == path and step["label"].endswith("(...)")
+    ]
+
+
+LIB = '''def melde(text):
+    print(text)
+'''
+
+MAIN = '''import random
+import gturtle
+import lib
+
+
+def anwenden(op, x, y):
+    if op == '+':
+        return x + y
+
+
+class Zaehler:
+    def erhoehe(self):
+        self.n = 1
+
+
+nichts = lambda: None
+xs = []
+anwenden('*', 3, 4)
+lib.melde('hallo')
+nichts()
+Zaehler().erhoehe()
+print('x')
+xs.append(1)
+random.shuffle(xs)
+gturtle.forward(10)
+ergebnis = anwenden('*', 3, 4)
+f = anwenden
+f('*', 1, 2)
+'''
+SOURCES = {"main.py": MAIN, "lib.py": LIB}
+
+# The other file as the worker mounts it, at "/lib.py".
+lib = types.ModuleType("lib")
+lib.__file__ = "/lib.py"
+exec(compile(LIB, "/lib.py", "exec"), lib.__dict__)
+sys.modules["lib"] = lib
+
+result = json.loads(runtime["_glpython_trace_program"]("main.py", SOURCES, None))
+expect("program: no error", result["errorMessage"], None)
+expect("program: the calls' values", [value for text, value in call_values(result, "main.py", MAIN) if text != "Zaehler()"], [
+    "None",  # anwenden('*', 3, 4)
+    "None",  # lib.melde('hallo'): a file of the project
+    "None",  # nichts(): a lambda
+    "None",  # Zaehler().erhoehe(): a method of the learner's class
+    None,  # print('x')
+    None,  # xs.append(1)
+    None,  # random.shuffle(xs)
+    None,  # gturtle.forward(10)
+    "None",  # ergebnis = anwenden('*', 3, 4): the value is used
+    "None",  # f('*', 1, 2): the learner's function under another name
+])
+
+# A REPL input (and a failing doctest example, which is traced the same way
+# against the Tests run's namespace): the file's functions as a Run left them.
+REPL_MAIN = '''def anwenden(op, x, y):
+    """
+    >>> anwenden('*', 3, 4)
+    12
+    """
+    if op == '+':
+        return x + y
+
+
+def zeige(x):
+    """
+    >>> zeige(3)
+    3
+    """
+    print(x)
+'''
+for namespace_name, namespace in (("repl", "_glpython_repl_namespace"), ("test", "_glpython_test_namespace")):
+    values = runtime.setdefault(namespace, {})
+    values.clear()
+    values["__name__"] = "__main__"
+    exec(compile(REPL_MAIN, "/main.py", "exec"), values)
+    with contextlib.redirect_stdout(io.StringIO()):
+        exec(compile("def leer():\n    pass\n", "<repl>", "exec"), values)
+    for source, expected in (("anwenden('*', 3, 4)", "None"), ("zeige(3)", "None"), ("leer()", "None"), ("print(3)", None), ("anwenden('+', 3, 4)", "7")):
+        traced = json.loads(runtime["_glpython_run_visual_repl"](source, json.dumps({"main.py": REPL_MAIN}), "main.py", namespace_name))
+        calls = [value for text, value in call_values(traced, "<repl>", source) if text == source]
+        expect(f"{namespace_name}: {source}", calls[-1:], [expected])
+
+# The console shows nothing for None, as CPython's REPL.
+runtime["_glpython_repl_namespace"].clear()
+runtime["_glpython_repl_namespace"]["__name__"] = "__main__"
+exec(compile(REPL_MAIN, "/main.py", "exec"), runtime["_glpython_repl_namespace"])
+with contextlib.redirect_stdout(io.StringIO()) as printed:
+    answer = json.loads(runtime["_glpython_repl"]("anwenden('*', 3, 4)"))
+expect("console: anwenden('*', 3, 4)", (answer.get("value"), printed.getvalue()), (None, ""))
+
+if failures:
+    print("\n".join(failures))
+    sys.exit(f"{len(failures)} None call check(s) failed.")
+print("Runtime None call checks passed.")

src/runtime/visualizer.py

@@ -935,6 +935,22 @@ def __call__(self, *args, **kwargs):
       _glpython_last_call_return = return_info
       _glpython_call_depth -= 1
 
+# The files of the traced project: their functions are the learner's own
+# (set by _glpython_trace_program). "<glpython>" is code of theirs the
+# visualizer compiled itself (a class body's methods), "<repl>" an earlier
+# REPL input's.
+_glpython_own_files = set()
+
+# Whether a callee is a function of the learner's own (a "def" or lambda of
+# the program, of one of its files or of a loaded library, also a method of
+# their class), not a built-in, a method of a built-in type or a function of
+# a module glpython or Python brings along (gturtle, random, ...).
+def _glpython_is_own_function(func):
+  if isinstance(func, _GlpythonFunction):
+    return True
+  code = getattr(getattr(func, "__func__", func), "__code__", None)
+  return code is not None and code.co_filename in _glpython_own_files
+
 def _eval_call(node, scope, suppress_none_result=False):
   global _glpython_pending_call_node, _glpython_last_call_frame_id
   func_ids = getattr(node, "_glpython_func_ids", None) or []
@@ -968,14 +984,15 @@ def _eval_call(node, scope, suppress_none_result=False):
   closes_frame_id = _glpython_last_call_frame_id
   _glpython_pending_call_node = None
   callee_name = getattr(func, "__name__", _glpython_node_source(node.func))
-  # A bare "f(...)" statement whose result nobody uses shouldn't have its
-  # call text replaced with the word "None" once it finishes - that's just
-  # visual noise, not insight (this is the overwhelmingly common case for
-  # side-effecting calls like print(...) or a student function with no
-  # return). Calls whose value IS used (an assignment, part of a bigger
-  # expression, etc.) still show "None" normally, since that's genuinely
-  # useful feedback there.
-  value_text = None if (suppress_none_result and value is None) else _glpython_safe_repr(value)
+  # A bare "f(...)" statement whose result nobody uses keeps its call text
+  # once it finishes when the callee is a built-in or a method of a built-in
+  # type (print(...), xs.append(x)): the word "None" would only be noise
+  # there. A call of the learner's own function shows its "None", even as a
+  # statement - it's what a function without a matching "return" gives, and
+  # what a doctest example or a REPL input compares or shows. Calls whose
+  # value is used (an assignment, part of a bigger expression, etc.) always
+  # show it.
+  value_text = None if (suppress_none_result and value is None and not _glpython_is_own_function(func)) else _glpython_safe_repr(value)
   return_info = _glpython_last_call_return if closes_frame_id is not None else None
   _glpython_record(node, "eval", f"{callee_name}(...)", value_text, _glpython_current_func(), closes_frame_id=closes_frame_id, extra=return_info)
   return value
@@ -1062,10 +1079,9 @@ def _exec_annassign(node, scope):
     _glpython_record(node, "exec", f"{_glpython_node_source(node.target)} = {_glpython_safe_repr(value)}", None, _glpython_current_func(), _glpython_snapshot_locals(scope))
 
 def _exec_expr(node, scope):
-  # A bare expression statement's value is always discarded by Python
-  # itself, so if it's a plain call ("f(...)" on its own line), tell
-  # _eval_call not to show "-> None" when that's all it computed - see the
-  # comment on _eval_call's suppress_none_result parameter.
+  # A bare expression statement's value is discarded by Python itself, so
+  # if it's a plain call ("f(...)" on its own line), tell _eval_call it may
+  # keep the call text instead of showing "None" - see the comment there.
   if isinstance(node.value, ast.Call):
     return _eval_call(node.value, scope, suppress_none_result=True)
   return _glpython_eval(node.value, scope)
@@ -1718,6 +1734,8 @@ def _glpython_trace_program(main_path, sources, fill_scope, report_value=False):
   _glpython_frame_stack = [0]
   _glpython_pending_call_node = None
   _glpython_last_call_frame_id = None
+  _glpython_own_files.clear()
+  _glpython_own_files.update(["<glpython>", _GLPYTHON_REPL_PATH, *("/" + path for path in sources if path != _GLPYTHON_REPL_PATH)])
 
   if "/" not in sys.path:
     sys.path.insert(0, "/")