Contain the extraction child: it runs as the service user and can reach the service's secrets

Problem

The extraction child (python -I -m processor._child, spec §3) parses untrusted documents with pypdf, lxml / libxml2, bs4 and openpyxl. v1 already treats it as untrusted in two ways:

Scrubbed environment: the child's environment holds only PATH and LANG, so the broker credential and GOOGLE_APPLICATION_CREDENTIALS are not in it.
Safe result channel: the parent decodes the child's result with an unpickler that resolves no global but ExtractOutcome. A compromised child cannot make the parent run code (test: test_a_compromised_child_cannot_run_code_in_the_parent).

Neither measure contains a child that gets code execution through a parser bug. The child runs as the same Unix user as the consumer, so it can:

read whatever that user can: the service's env file and GCS service-account key, and the operator/agent files in that user's home;
read the parent's environment through /proc/<parent>/environ;
open network connections, the broker included.

This was found in the v1 code review (CR 2, fe62727) and accepted for the MVP on 2026-09-30. The input is pages of known watched items that Replicator fetches, not arbitrary uploads. That narrows the exposure but does not close it.

Proposed containment (in layers)

Landlock in the child. This VM's kernel offers ABI 6, and Landlock needs no privilege and no unit change. After RLIMIT_AS and before importing the extractors, the child restricts itself:
- Filesystem: read-only on the interpreter, venv and system library paths; no writes; nothing under /etc/processor, the service user's home, or /proc/<ppid>.
- Network: no TCP connect or bind (ABI ≥ 4).
- Scoping: no signals to other processes and no abstract unix sockets (ABI 6).
A setting decides whether a kernel without Landlock fails closed (the production default) or open (dev).
Parent PR_SET_DUMPABLE=0. This closes /proc/<pid>/environ and ptrace to same-uid processes.
A dedicated service user (processor, not the operator/agent user):
- /etc/processor owned by root and read by systemd (EnvironmentFile=);
- the GCS key through LoadCredential=.
This separates the service from the agent's own credentials, even though the child still shares the service's uid (so layer 1 is still needed). It needs an operator step at install.
Optional: a seccomp filter for the child.

Done when

Tests show the child cannot open the service env file, the key file, the parent's /proc/<pid>/environ, or a file in the service user's home.
Tests show the child cannot connect to 127.0.0.1:6379.
The whole parity corpus still passes through the contained child.
Spec §3 and docs/DEPLOYMENT.md describe the containment, and the known-limitation note in spec §3 is removed.

Refs: #1, spec §3 (the child process), src/processor/_child.py, src/processor/child.py.

#1 fix: CR 2 decisions — escapes dead-letter at the cap, DataCorruption

Hand-off from the burn-down epic #24, 2026-10-05. This is next. #23 shipped in #32.

Why now. It's the last item before cutover (watcher#326). Landing it during shadow lets Watcher's comparator check the contained child against real traffic, not only the parity fixtures.

Operator decisions (2026-10-05, at the check-in):

One PR, all layers (Landlock, a non-dumpable parent, a dedicated service user). Not split.
Disable the Docker socket now, pulled forward from Index processor on co-index's shared SocratiCode store (init-socraticode, STORE=external) #5. The operator runs sudo systemctl disable --now docker.socket. Contain the extraction child: it runs as the service user and can reach the service's secrets #2 adds a test that the child can't reach /run/docker.sock, and Index processor on co-index's shared SocratiCode store (init-socraticode, STORE=external) #5's box is then done.

Measured on co-processor, 2026-10-05T23:15Z:

Kernel 6.12.93. Landlock ABI 6 (landlock_create_ruleset(NULL, 0, VERSION) returns 6); LSMs capability,landlock,selinux.
The unit runs as exedev, which has passwordless sudo and is in group docker. NoNewPrivileges=yes already blocks sudo from the service and its children. It does not block /run/docker.sock (root:docker 0660, docker.socket enabled and active), which is the worst exposure today: code execution in the child means root through Docker.
/run/tailscale/tailscaled.sock is root:root 0666: any local uid can query tailscaled's LocalAPI (read-only for a non-operator).
The interpreter is /usr/bin/python3.12 (system). sys.path under -I is /usr/lib/python3.12*, .venv/lib/python3.12/site-packages, and /home/exedev/processor/src (an editable install via _editable_impl_processor.pth).
/home/exedev is 0750. Secrets in reach of the service user: /etc/processor/.env (dir 700), the GCS writer key it points to, and the repo's .env (GH tokens, 600).
NSS hosts: files dns, so dropping AF_UNIX doesn't affect name resolution.

Touchpoints (checked against main at 80165dd):

src/processor/_child.py main(), in order today: oom_score_adj → RLIMIT_AS → move the result channel → pickle.load → import the transform → run.
src/processor/child.py run_in_child (-I, env=_child_env(), cwd="/") and its docstring (lines 3–20).
deploy/processor.service: User/Group, WorkingDirectory, EnvironmentFile, ExecStart, NoNewPrivileges, ProtectSystem=full.
Spec §3, lines 204–208: the "not a sandbox" note to remove. DEPLOYMENT.md: install, "Deploy a change" (line 174), env file. AGENTS.md: "Extraction runs in a killable child … Not a sandbox", and "Code on main is the deployed code".

Traps found while drafting:

Layer 3 breaks the deploy model. /home/exedev is 0750, so a processor user can't reach the checkout, .venv or src/ (the editable install). Today's invariant "this checkout is the deployed code" (AGENTS.md, DEPLOYMENT, and step 3 of the epic's verify recipe) can't survive as it is. Options:
- (a) Deploy to a root-owned tree (e.g. /opt/processor: a git worktree or git archive of main, a non-editable install, its own venv), and change "Deploy a change" to build it. Recommended: the service can then read nothing of exedev's.
- (b) chmod o+x /home/exedev (traverse only). A smaller change, but every world-readable file under exedev's home whose path is known becomes readable, and the editable src/ stays agent-writable code the service executes.
- (c) Something else the plan argues for.
This changes the operator's deploy command, so the plan states the new deploy and install steps and the operator approves them before building. The epic's verify recipe will need a new step 3 (e.g. the deployed tree's commit equals origin/main). Say what it is.
Landlock on 6.12 probably doesn't restrict connect() to a pathname unix socket. Its filesystem rights cover MAKE_SOCK (creating one), not connecting to one that exists. [stated from Landlock's docs; not run here, so prove it with a test]. So layer 1 alone leaves the child able to reach tailscaled.sock, and docker.sock until it's disabled. The child needs no sockets at all. Options:
- a seccomp filter in the child that refuses socket()/socketpair() (the issue's optional layer 4; a small classic-BPF program through prctl(PR_SET_SECCOMP));
- RestrictAddressFamilies=AF_INET AF_INET6 on the unit, which covers the whole service; check that nothing in the parent needs AF_UNIX, since journald gets an inherited fd and NSS has no resolve module.
Recommendation: the seccomp filter, scoped to the child. Either way, a test that the child can't connect to a unix socket the service user could otherwise reach.
Derive Landlock's allowlist; don't hard-code it. The paths move with trap 1. Allow read and execute on sys.base_prefix's library tree, each non-empty sys.path entry, the dynamic loader and /etc/ld.so.cache, and whatever else the parity corpus shows it needs (6.12 has no Landlock audit log, so run the corpus and see what it refuses). Never allow the repo root, which holds .env. Write nothing. Leave /proc out (the oom_score_adj write happens before the restriction).

Things to weigh:

No new dependency. _child imports only the stdlib at module level (its docstring). Landlock is 3 syscalls (444–446) plus prctl(PR_SET_NO_NEW_PRIVS), through ctypes. Load ctypes and libc before restricting. A PyPI binding would mean a wheelhouse and supply-chain change.
The fail-closed setting (the issue's): the production default refuses to run extraction without Landlock ABI ≥ 6. Dev and CI need to say which mode they run. CI's runner kernel may lack ABI 6, so check, and skip the containment tests with a stated reason rather than letting them pass vacuously.
PR_SET_DUMPABLE=0 on the parent closes /proc/<parent>/environ to same-uid processes. With layer 3, the child shares the service's uid, so this still matters.
LoadCredential= for the GCS key: Environment=GOOGLE_APPLICATION_CREDENTIALS=%d/<name> (%d is the credentials directory). Check that pydantic-settings and co-core read it unchanged.
The env file moves to root-owned and read by systemd (EnvironmentFile= is read before the user switch). The service user doesn't need to read it.

Done when (the issue's, plus):

Tests show the child can't open: the service env file, the key file, /proc/<parent>/environ, a file in the service user's home, or the repo's .env.
Tests show the child can't TCP-connect to 127.0.0.1:6379, and can't connect to a unix socket.
The whole parity corpus, including tests/fixtures/parity/real/, passes through the contained child.
The operator has disabled docker.socket, and a test or check shows the child can't reach /run/docker.sock.
Spec §3, DEPLOYMENT.md (install, deploy, env, credentials) and AGENTS.md describe the containment and the new deploy model. The "not a sandbox" note is removed.
Deployed: the service runs as the new user. A live shadow command after the restart is ack: complete with Outcome log records no digests: add input_digest, output_digest, size, empty and stored to every command outcome #26's digests, and Watcher still reports matches.

Gates (operator):

docker.socket off (decided; any time).
Approve the plan's deploy and install changes before building (trap 1).
The install step: create the user, move secrets, LoadCredential=, the first deploy to the new tree.

Deploy: this changes src/ and deploy/. The restart is the new install, so the plan gives the rollback (the old unit and tree) in case the first shadow command fails. During shadow the risk is bounded: a failed command stays pending and Watcher doesn't act on it.

Out of scope: #5 (except its docker.socket box), #29, #21.

Ship: issue → plan in docs/plans/ (operator review: gate 2) → branch in a worktree → TDD → PR → FF merge → install (gate 3). The closing comment gives each box with its evidence (one live record), the known limits, and the follow-ups (or "none").

Kickoff prompt:

Pick up processor#2. Read its hand-off comment from epic #24 first.

Gate 1 done: docker.socket is off (2026-10-06T00:01:57Z, the coordinator, with the operator's authorization). systemctl is-enabled docker.socket returns disabled, and docker.socket, docker.service and containerd are all inactive. Nothing depended on them. A same-uid connect('/run/docker.sock') now returns ECONNREFUSED. The socket file is still there (RemoveOnStop=no), with nothing listening, and it isn't recreated at boot. processor was unaffected (NRestarts=0). The Done-when check "the child can't reach /run/docker.sock" should test that the connect fails, not that the file is missing.

#2 docs: plan — Layer 3 follows the cohort deploy standard (status#9 …

#2 feat: _contain — Landlock, seccomp socket filter, undumpable paren…

#2 feat: the child contains itself; processor run is undumpable, fail…

#2 feat: releases — scripts/deploy.sh and the unit as user processor …

#2 docs: spec §2/§3/§7, DEPLOYMENT, AGENTS, SKILLS — containment, the…

#2 fix: the allowlist's library dirs come from the loader's mapped ob…

#2 fix: CR 1 — a boot canary: one real extraction through the contain…

#2 fix: CR 2 — grant read only: the contained child executes nothing,…

#2 fix: CR 3 — a failed first deploy restarts the service on the unit…

#2 fix: CR 4 — contain() refuses a process with a second thread, whic…

#2 fix: CR 5 — the deploy's verify reads the new MainPID's records fr…

#2 fix: CR 6 — prctl and syscall arguments reach the kernel as full-w…

#2 docs: CR 8 — the undumpable parent's debugging cost, and deploy.sh…

#2 fix: CR 9 — check the key as the unit receives it: ACL-readable by…

#2 docs: CR 10 — the plan records what build and review amended

#2 fix: CR 11 — releases are copied, never hardlinked, from the uv cache

#2 fix: CR 12 — the install chmods /etc/processor's files by name, no…

#2 fix: CR 11 — wrap the uv stub's log line (E501; CR 11's commit ski…

#2 fix: CR 13 — a failed archive or wheelhouse copy says so, rather t…

#2 fix: CR 14 — releases/ is created 755 explicitly, so the service u…

Reopened. The merge of #33 closed this through "Closes #2". The code is merged and Stage A is live, but the Done-when still needs Stage B: the service running as processor, plus live evidence.

Merged: PR #33, FF to main at fe19a2b (2026-10-06 16:55:48Z). The commits:

the plan (gate 2, approved 2026-10-06);
_contain (Landlock, seccomp, undumpable parent);
the child wiring and the boot check;
releases with scripts/deploy.sh, and the unit as processor;
docs;
four review rounds: CR 1–14, with CR 7 held because it would rewrite history.

Stage A, live since 16:55:51Z. The old deploy path: uv sync --frozen --no-dev and a restart, still as exedev from ~/processor.

starting: child_containment: required, landlock_abi: 6, build: dev. The boot canary passed: one real extraction through the contained child.
scripts/smoke_scratch_bus.py passed, with "child_containment": "required".
Parent: NoNewPrivs: 1, and a same-uid read of /proc/<pid>/environ is refused (undumpable).
docker.socket is disabled (gate 1).
The next shadow command will be added here.

Stage B (gate 3, the operator): DEPLOYMENT.md § Install, then the checks in § Containment, which include the key-as-credential pair. Rollback is in § Rollback. The closing comment will give each box with its evidence.

Follow-ups, to file at close: processor's CI gate and its drift check (status#11 and status#12). The feedback for the cohort skill is in the plan and linked from Provisioner#23.

Stage A, the first live shadow command under containment: 01M492PB6W3P8N0EEA85HZX2ED at 2026-10-06T17:03:02.704Z, ack: complete.

input_digest 2e38aa5e…, output_digest sha256:b8f6d0f1… (17,327 bytes, Watcher's recorded fingerprint for that input), processor_version 0.19.7+1.
total_ms 838.

Stage B (gate 3) is next.

Done, 2026-10-06. Shipped in #33 (FF, main = fe19a2b). Stage A was live at 16:55:51Z; Stage B was installed at 19:05:30Z. The service runs as processor from release fe19a2b6a06f.

Each box, with its evidence

Box	Evidence
The child can't open the env file, the key file, `/proc/<parent>/environ`, or a file in the service user's home	`tests/test_child.py`, under `required`, every denial paired with an uncontained control. On `co-processor` before the install, against the real `/etc/processor/.env` and `co-gcs-processor-writer.json`. Live after the install: the key as the unit receives it, `/run/credentials/processor.service/gcs-writer-key`, is readable by `processor` (the control) and gives `PermissionError` to a contained process. The parent is undumpable: `sudo -u processor cat /proc/<pid>/environ` is refused
…or the repo's `.env`	`test_the_child_cannot_read_the_real_secrets[…/.env]`, run from the main checkout after the install: passed
The child can't TCP-connect to `127.0.0.1:6379`, or connect to a unix socket	The tests: EPERM at `socket()` from seccomp. `tests/test_contain.py` pins that Landlock alone lets a pathname unix socket through (trap 2)
The whole parity corpus passes through the contained child, `real/` included	`test_the_whole_parity_corpus_is_identical_through_the_child`: 22 inputs (`cases.json` plus 9 real). It also passed from a real release build, and CI ran it under `required` on Landlock ABI 7
`docker.socket` off, and the child can't reach `/run/docker.sock`	`systemctl is-enabled docker.socket` says `disabled` (gate 1, 2026-10-05). `test_the_child_cannot_reach_the_hosts_sockets[/run/docker.sock]` gets EPERM, and `processor` isn't in group `docker`
Spec §3, DEPLOYMENT and AGENTS.md describe it; the "not a sandbox" note is gone	Spec §2, §3 and §7. DEPLOYMENT: `/etc/processor/`, releases, `scripts/deploy.sh`, Containment, Install, Rollback. AGENTS.md
Deployed: runs as the new user; a live shadow command is `ack: complete` with #26's digests; Watcher still matches	One live record, below: as `processor`, `ack: complete`, and its `output_digest` equals Watcher's recorded fingerprint for that input (`tests/fixtures/parity/real/export.json`)

The live record, the first command handled as processor:

{"timestamp":"2026-10-06T19:21:02.698838Z","command_id":"01M49AK0VC91BHX7V70XS9JJ4W","action":"ack","reason":"complete","input_digest":"6abb3bf14eba7131833a56a2c9e2d8afa2a00d753273945dc51f09de1e5567bd","output_digest":"sha256:4cc814c794ec78b4c9c9d707065ab625a2f15b2d28825c24281fde394e0dfd04","output_size_bytes":11791,"empty":false,"processor_version":"0.19.7+1","total_ms":1128.5}

Its start record: {"build":"fe19a2b6a06f","child_containment":"required","landlock_abi":6}.

Under Stage A there were two more ack: complete records, with Watcher's fingerprint, at 17:03:02Z and 19:05:02Z. The second came after /etc/processor was made root-only, and was served on the old process's in-memory credentials.

What it is

The child contains itself (processor._contain, stdlib ctypes). Landlock gives it read-only access to an allowlist derived from sys.path and the loaded libraries, with no execve, no TCP, and scoped signals and abstract sockets. seccomp blocks every socket and io_uring. It refuses to contain a process with a second thread.
The parent is undumpable. It refuses to start without containment, and runs a canary extraction at boot.
The service user and the deploy: user processor; /etc/processor root-only; the key through LoadCredential=; ProtectSystem=strict and ProtectHome=yes. Releases follow the cohort standard (broker#22 / status#9): scripts/deploy.sh with verify, the smoke run as the service, switch-back on failure, and copied rather than hardlinked from the uv cache.

Known limits

The child shares the kernel: every syscall but the socket family is reachable.
It can read its own code, the stdlib and the shared libraries.
It can spend CPU up to the timeout and memory up to RLIMIT_AS.
It decides its own command's output, which only Watcher's comparison checks.
It shares the service's uid. Landlock and the undumpable parent carry that, not uid separation.
exedev owns the releases and has sudo, so that boundary is the service against its code, not the operator against production (as Status's R2).

Follow-ups

deploy.sh: gate a deploy on the commit's CI (status#11's CI gate) #34: a CI gate on deploy (status#11).
A drift signal when live lags origin/main (status#12's drift check) #35: a drift signal when live lags origin/main (status#12).
The cohort-skill feedback (14 points) is in the plan, linked from Provisioner#23.
Not done here: ticking Index processor on co-index's shared SocratiCode store (init-socraticode, STORE=external) #5's docker.socket box, which waits on the operator.

Navigation Menu

Repository navigation

Contain the extraction child: it runs as the service user and can reach the service's secrets #2

Problem

Proposed containment (in layers)

Done when

Each box, with its evidence

What it is

Known limits

Follow-ups

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Contain the extraction child: it runs as the service user and can reach the service's secrets #2

Description

Problem

Proposed containment (in layers)

Done when

Activity

gregoryfoster commented on Oct 5, 2026

gregoryfoster commented on Oct 6, 2026

gregoryfoster commented on Oct 6, 2026

gregoryfoster commented on Oct 6, 2026

gregoryfoster commented on Oct 6, 2026

Each box, with its evidence

What it is

Known limits

Follow-ups

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions