computer: Add scoped host modules

.changeset/scoped-host-modules.md

@@ -0,0 +1,5 @@
+---
+"@cloudflare/computer": minor
+---
+
+Add `createScopedModule()` for host modules that hold state for a run, such as a browser session. A scope lasts from when a run's code starts in an isolate until it waits in `ctx.interrupt()` or ends. The backend starts one instance for each scope, on the scope's first call to the module, and closes it when the scope ends, with a `log()` on a run's end that appends to its stderr. Scoped modules are experimental.

docs/17_isolate_javascript.md

@@ -276,7 +276,7 @@ Caller source can import four kinds of module, and all of them are fixed when th
 | Built in | Always installed | The isolate, backed by the Workspace | `node:fs`, `node:fs/promises` |
 | Node.js | `nodejs_compat` in `compatibilityFlags`, the default | The isolate, provided by the runtime | `node:path`, `node:crypto` |
 | Source | `modules: { name: "source" }` | The isolate | a bundled library |
-| Host | `modules: { "codemode:name": { fn } }`, or a factory | The Durable Object | `codemode:git`, `codemode:container`, `codemode:tools`, your own |
+| Host | `modules: { "codemode:name": { fn } }`, a factory, or `createScopedModule()` | The Durable Object | `codemode:git`, `codemode:container`, `codemode:tools`, your own |
 
 ```ts
 import { createArtifactsModule } from "@cloudflare/computer/modules/artifacts";
@@ -324,7 +324,7 @@ Modules code can import:
 - Node.js built-ins: `node:path`, `node:url`, ... Bare names such as `path` work too.
 ```
 
-A factory adds its own text through a `description` property, as the prebuilt modules do. The backend reads it each time it describes itself, not once when it is constructed, so a factory can define `description` as a getter and describe something that exists only later, as `codemode:tools` does with the agent's tools. An object of functions is listed by its export names; say more about it in the `exec` tool's backend description if the model needs it.
+A factory adds its own text through a `description` property, as the prebuilt modules do. The backend reads it each time it describes itself, not once when it is constructed, so a factory can define `description` as a getter and describe something that exists only later, as `codemode:tools` does with the agent's tools. A scoped module takes `description` as an option. An object of functions, or a scoped module without a description, is listed by its export names; say more about it in the `exec` tool's backend description if the model needs it.
 
 ### Built-in filesystem
 
@@ -429,6 +429,43 @@ The run's code never sees metadata: it isn't in `process.env` or the input, so c
 
 Specifiers and the export names of an object are checked at construction. A factory's export names are checked when the backend connects and the factory runs. An object must export at least one function. A factory may build none, as `codemode:tools` does for an agent with no tools: the backend then leaves that module out of the connection, so only code that imports it fails, with a message saying the module exports nothing, and every other run still starts. Every export name must be a JavaScript identifier name other than `default` or `then`. A reserved word such as `delete` is allowed, and caller code renames it on import: `import { delete as remove } from "codemode:files"`. Importing a name the module does not export fails when the module graph links, before any code runs.
 
+#### Scoped modules
+
+A module that holds state for a run, such as a browser session it opens on the first call, can be made with `createScopedModule()`. The state lives for one scope of the run: from when the run's code starts in an isolate until it waits in `ctx.interrupt()` or ends. A run that resumes starts again from the top in a new scope. The backend makes one instance of the module for each scope, on the scope's first call to the module, and closes it when the scope ends:
+
+```ts
+import { createScopedModule } from "@cloudflare/computer";
+
+modules: {
+  "codemode:browser": createScopedModule({
+    description: "A browser: `open(url)` and `text()`.",
+    exports: ["open", "text"],
+    async start(scope, host) {
+      const session = await browsers.connect({ conversation: scope.metadata.conversation });
+      return {
+        functions: {
+          open: ([url]) => session.open(String(url)),
+          text: () => session.text(),
+        },
+        async close(end) {
+          await session.close();
+          if (end.reason !== "paused") end.log("Closed the browser.\n");
+        },
+      };
+    },
+  }),
+}
+```
+
+`start` gets the scope's `{ id, metadata, signal }`, where `id` and `metadata` are the run's and stay the same in every scope, and the same services a factory gets. It returns the instance's `functions` with an optional `close`. The backend needs the export names before any run starts, so the module lists them in `exports`, which is checked at construction like an object's export names. If the functions `start` returns don't match `exports` exactly, the call fails and says so. Calls a scope makes while its instance starts wait for it, and if `start` throws, they fail. `scope.signal` aborts when the scope is cancelled or releases its isolate, and a call that was cancelled or timed out by the time `start` returns fails rather than running. A call answered from a replayable run's journal doesn't start an instance, so a scope whose calls to the module are all replayed never starts one.
+
+An instance closes only if it started, and then once, after the scope's host calls settle. `close` gets one of these reasons:
+
+- `"paused"`: the run is waiting in `ctx.interrupt()` and has released its isolate. Its code starts again from the top when it resumes, in a new scope with a new instance, so let go of everything. An answer that arrives during `close` resumes the run once `close` is done.
+- `"completed"`, `"failed"`, or `"cancelled"`: the run ended. `close` runs before its exit is published, and only these reasons have `log(text)`, which appends text to the run's stderr ahead of its exit. Logged text is saved with the run's terminal state, so a run interrupted before it ends never resumes against output it didn't write. Text past what's left of the run's `maxStdioBytes` is cut, and ends with a line saying `Host module output truncated.` Text logged after `close` returns is dropped.
+
+A run killed while it waits has no instance open, so nothing closes. If the Durable Object restarts during a scope, the scope's instances go with it, unclosed. Every instance of a scope closes at once, with a `signal` and a `deadline`, and the backend waits for them up to `maxHostCallMs`, when `signal` aborts. A `close` that throws or runs late is reported with `console.error`, and the run goes on as it would have. Scoped modules are experimental and may change in any release.
+
 ### `codemode:git`
 
 ```js

packages/computer/README.md

@@ -34,6 +34,7 @@ removed in a future release; each one names its replacement.
 | Resuming interrupted isolate JavaScript: `onInterrupt: "restart"` and `"replay"`, `workspace.runtime.recover()`, `unsafeToRepeat()`, and waiting for answers with `ctx.interrupt()`, `pending()` and `respond()` | `@cloudflare/computer`, `@cloudflare/computer/backends/worker-javascript` | Experimental |
 | `WorkerShellBackend` and the shell command groups | `@cloudflare/computer/backends/worker-shell`, `@cloudflare/computer/shell/*` | Experimental |
 | The `codemode:container` host module | `@cloudflare/computer/modules/container` | Experimental |
+| Scoped host modules: `createScopedModule()` | `@cloudflare/computer`, `@cloudflare/computer/backends/worker-javascript` | Experimental |
 | `workspace.push()`, `workspace.pull()`, and the sync protocol between the durable object and `computerd` | `@cloudflare/computer` | Experimental |
 | The `computerd` daemon and its container image | `ghcr.io/cloudflare/computer-computerd-linux-x64` | Experimental |
 | The RPC wire protocol between the durable object and `computerd` (`@cloudflare/computer-rpc`) | Internal | Experimental |

packages/computer/src/backends/worker-javascript/module-graph.ts

@@ -1,6 +1,7 @@
 import { parse } from "acorn";
 
 import type { WorkspaceRuntimeCapability } from "../../runtime/capability";
+import { isScopedModule, type WorkspaceScopedModule } from "../../runtime/scoped-module";
 import type {
   WorkspaceModule,
   WorkspaceModuleFactory,
@@ -85,6 +86,7 @@ const RESERVED_EXPORT_NAMES = new Set(["default", "then"]);
 export interface ParsedModules {
   readonly source: Readonly<Record<string, string>>;
   readonly host: ReadonlyMap<string, WorkspaceModuleFactory>;
+  readonly scoped: ReadonlyMap<string, WorkspaceScopedModule>;
   /** One markdown bullet per importable module, for a model. */
   readonly description: string;
 }
@@ -93,19 +95,22 @@ const FILESYSTEM_DESCRIPTION =
   "- `node:fs/promises` (also `node:fs`): the workspace's files. `readFile`, `writeFile`, `mkdir`, `rm`, `readdir`, `stat`, `lstat`, `readlink`, `symlink`, `chmod`, and `access`. Async only.";
 
 /**
- * Parse the backend's `modules` option: split it into bundled source
- * and host module factories, check every specifier and every object's
- * export names, and describe each module for a model. A factory's
- * export names are checked when the backend connects and it runs.
+ * Parse the backend's `modules` option: split it into bundled source,
+ * host module factories, and scoped host modules, check every
+ * specifier and the export names of every object and scoped module,
+ * and describe each module for a model. A factory's export names are
+ * checked when the backend connects and it runs.
  *
  * @param modules - The modules passed to the backend.
- * @returns Source modules, host module factories, and their description.
+ * @returns Source modules, host module factories, scoped modules, and
+ *   their description.
  * @throws When a specifier or an object's export names are not allowed.
  *   The host configured the backend wrongly and no execution can use it.
  */
 export function parseModules(modules: Readonly<Record<string, WorkspaceModule>>): ParsedModules {
   const source: Record<string, string> = Object.create(null);
   const host = new Map<string, WorkspaceModuleFactory>();
+  const scoped = new Map<string, WorkspaceScopedModule>();
   const lines: Array<string | (() => string)> = [FILESYSTEM_DESCRIPTION];
   for (const [specifier, module] of Object.entries(modules)) {
     const name = `\`${specifier}\``;
@@ -137,6 +142,13 @@ export function parseModules(modules: Readonly<Record<string, WorkspaceModule>>)
         `Host module ${JSON.stringify(specifier)} must use a simple codemode:* name, such as "codemode:git".`,
       );
     }
+    if (isScopedModule(module)) {
+      assertExportNames(specifier, module.exports);
+      scoped.set(specifier, module);
+      const exports = module.exports.map((key) => `\`${key}\``).join(", ");
+      lines.push(`- ${name}: ${module.description ?? `exports ${exports}.`}`);
+      continue;
+    }
     if (typeof module === "function") {
       host.set(specifier, module);
       lines.push(() => `- ${name}: ${module.description ?? "a host module."}`);
@@ -155,6 +167,7 @@ export function parseModules(modules: Readonly<Record<string, WorkspaceModule>>)
   return {
     source,
     host,
+    scoped,
     get description() {
       return lines.map((line) => (typeof line === "string" ? line : line())).join("\n");
     },
@@ -174,20 +187,29 @@ export function assertHostModuleExports(
   functions: WorkspaceModuleFunctions,
 ): void {
   const names = Object.keys(functions);
+  assertExportNames(specifier, names);
+  for (const name of names) {
+    if (typeof functions[name] !== "function") {
+      throw new Error(
+        `Host module ${JSON.stringify(specifier)} export ${JSON.stringify(name)} must be a function.`,
+      );
+    }
+  }
+}
+
+// Check a host module's export names.
+function assertExportNames(specifier: string, names: readonly string[]): void {
   if (names.length === 0) {
     throw new Error(`Host module ${JSON.stringify(specifier)} must export a function.`);
   }
+  if (new Set(names).size !== names.length)
+    throw new Error(`Host module ${JSON.stringify(specifier)} exports a name twice.`);
   for (const name of names) {
-    if (!EXPORT_NAME.test(name) || RESERVED_EXPORT_NAMES.has(name)) {
+    if (typeof name !== "string" || !EXPORT_NAME.test(name) || RESERVED_EXPORT_NAMES.has(name)) {
       throw new Error(
         `Host module ${JSON.stringify(specifier)} export ${JSON.stringify(name)} must be a JavaScript identifier other than "default" or "then".`,
       );
     }
-    if (typeof functions[name] !== "function") {
-      throw new Error(
-        `Host module ${JSON.stringify(specifier)} export ${JSON.stringify(name)} must be a function.`,
-      );
-    }
   }
 }
 
@@ -196,7 +218,8 @@ export interface BuildModuleGraphOptions {
   cwd: string;
   capability: WorkspaceRuntimeCapability;
   configuredModules: Readonly<Record<string, string>>;
-  hostModules: ReadonlyMap<string, WorkspaceModuleFunctions>;
+  /** Each host module's export names, by specifier. */
+  hostModules: ReadonlyMap<string, readonly string[]>;
   /** Host modules whose factory built no functions on this connection, so they are left out. */
   emptyHostModules?: ReadonlySet<string>;
   /** Whether the Dynamic Worker has the Node.js built-ins. */
@@ -365,9 +388,9 @@ export async function buildModuleGraph(options: BuildModuleGraphOptions) {
   modules["node:fs"] = { js: nodeFsModule() };
 
   const toCapabilities = relativeSpecifier(storedName("module"), CAPABILITIES_MODULE);
-  for (const [specifier, functions] of options.hostModules) {
+  for (const [specifier, names] of options.hostModules) {
     modules[storedName(specifier)] = {
-      js: hostModule(toCapabilities, specifier, Object.keys(functions)),
+      js: hostModule(toCapabilities, specifier, names),
     };
   }
   for (const [specifier, source] of Object.entries(options.configuredModules)) {