Reach the files of a box through a file service in the box

[splitbrain-bot]

Part of #109 (step 4).

Changes

• File service. A new top-level package files/ builds boxes-files, a process in the box image. The
  orchestrator starts it as an exec, as it starts an adapter, and speaks HTTP/2 over its standard input and output.
  It runs as the agent. The operations are list, find-repos, stat, read, raw, write, remove, mkdir, create and size.
  The path rules of review/fs.ts moved into it. A write needs If-Match with the expected hash or
  If-None-Match: *, and goes through a temp file. Every reply carries the protocol version.
• Contract. shared/files.ts holds the protocol version, the routes, the caps and validRelativePath.
• Client. orchestrator/src/files.ts keeps one connection per container and starts the file service at the first
  request. It checks the shape and the size of each reply, applies timeouts, tries a read once more after a lost
  connection, and never tries a write again. A file service of another version, or an image without it (exit code
  126 or 127), gives a 409 that names the version found and the version needed.
• Users. The review (all reads and writes, REVIEW.md, the raw download), the attachments (upload and download)
  and the disk usage go through the file service. The orchestrator no longer reads box files on the data volume.
  The chown calls on review and attachment files and chownFdToAgent are gone.
• Disk usage. The orchestrator measures a box before each stop (60 s limit, the old value stays on a timeout), and
  a running box again after 15 minutes when the box list or the details ask. A migration adds disk_bytes and
  disk_measured_at to boxes. The API has diskMeasuredAt, and the box details show it.
• Probe. node …/boxes-files is part of the box in gateway/background.ts, so the idle reaper can stop a box
  with a connected file service, and the box-level stop leaves it running.
• Build. The box image builds with the repository root as its context, with a new files-build stage. Its test
  stage runs boxes-files </dev/null. publish.yml uses the new context and runs the files tests.
  orchestrator/Dockerfile copies files/src, because the test helper files-in-process.ts imports it.
• Docs. New doc/files.md. development.md, storage.md, review.md, rests.md, boxes.md, orchestrator.md,
  threads.md and the README are updated.

Changes in behavior

• The raw download, the attachment upload and download, deleting the review and deleting a comment start a stopped
  box (section 8.5 of the concept).
• The repository at the workspace root is named workspace in the review. Before, it had the box id, from the host
  directory name.
• A box with an image from before this change gets the 409 at each file operation until it runs the new image.

Tests

• files: 53 passed (path rules, operations, the HTTP/2 server, and the real main.ts over stdio).
• Orchestrator: tsc --noEmit is clean, 856 passed, 1 skipped. The review, attachment and app tests run the real
  file service in process. New client tests cover reuse of the connection, the read retry, no write retry, the
  version error, the missing command, the caps and the reply shape.
• Proxy: 73 passed.
• Dashboard: npm run check is clean. The full suite had two failures in the box info view, which are fixed; the view
  tests passed again (18). The e2e harness runs the file service in process.
• The bundle built in the layout of the files-build stage runs through the link and exits 0 on an empty input.

Not tested against a real Docker daemon. The box image build did not run.

• The first CI run failed in e2e/ux.test.ts ("a run of reasoning and tool rows stays a list"), which this PR does
  not touch. Its second iteration measured the row gaps while the second turn still rendered, because both of its
  waits were already satisfied by the first turn. The second commit makes the test wait for the second turn: 6 of 6
  runs and the whole file pass locally.

Notes for the next steps

• Step 5, agent set and ready state.
  • installAgentSet(tar) becomes a new route of the file service. If it starts a child process, for example tar,
    the probe rule in gateway/background.ts must also cover the children of the file service. Today it matches
    only the process itself, because the disk walk runs in process.
  • Today the first file operation starts the file service. Step 5 starts it as the first exec after start, as part
    of the ready state.
  • Today a version mismatch fails each file operation with a 409. Section 8.3 wants the box status error with this
    message, and a note when the last pull failed. versionError in files.ts builds the message.
  • agents.ts still writes agents/<id> and calls chownToAgent. This goes with the /boxes/agent mount.
• Step 6, home seed. createWorkspace, createHome and createNix in workspaces.ts still chown the new
  directories. They are storage creation and move behind the runtime in step 7.
• Step 7, the Runtime interface.
  • The spawner in files.ts calls dk.spawnAdapterExec. It becomes Runtime.spawn(boxId, ['boxes-files']).
    fileService() is keyed by the container id and becomes keyed by the box id.
  • measureDisk in boxes.ts uses row.container_id, and stopHeld reads containerState before it measures.
    Both become Runtime.state and the box id.
  • The disk measurement has its own 60 s timeout (SIZE_TIMEOUT_MS). Section 8.4 says it has the timeout of a run.
  • setFileServiceSpawnerForTests and files-in-process.ts are the seams for the tests; a fake runtime can use the
    same in-process server.
• Step 8, conformance tests.
  • The file service end to end: start a box, spawn boxes-files, write and read a file, and check that the file
    belongs to BOX_UID.
  • The "version none" message depends on the exit code 126 or 127 of an exec whose command is missing. A
    conformance test can check this code on each runtime.
• Step 9, Kubernetes.
  • An exec through the API server reports its exit code on the status channel. The runtime must give it as
    exited, or the missing-command check in refuseMissingCommand never matches.
  • An idle file service connection can end after the idle timeout of the container runtime. The next operation
    reconnects, and a read in progress is tried once more.
  • The file service spawns with stdin open for the whole connection. The exec transport must keep stdin open and pass
    the end of stdin when the orchestrator closes it.