Skip to content

[security] dryRun can still submit a real application — the guard only inspects clicks, and press_key Enter is prompted for #246

Closed

Description

@serge-ivo

dryRun exists so a user can test an application safely. apply-loop.ts:505 states the contract:

Dry run's whole promise is "this will not submit", so an unverifiable click is refused rather
than trusted.

The runtime guard is deliberately placed where the model cannot override it
(job-apply.ts:173) — the right design. But it only inspects one action type, and its
blocklist is an English regex.

Gap 1 — press_key is not guarded at all

export function dryRunBlockReason(action) {
    if (!action || action.action !== "click") return null;   // :513
    …
}

press_key is offered to the brain unconditionally (:387, "Press a keyboard key like Enter or
Tab"
) and maps straight through:

case "press_key": return { thought, action: { action: "key", key: str(a.key) || "Enter" } };  // :537

Pressing Enter inside a form field is a standard way to submit a form. In a dry run the brain
can do it and nothing stops it.

This is not a hypothetical route the model would have to invent — the prompt actively teaches
it
, twice:

  • :470 — "…or press_key ArrowDown then Enter" for custom comboboxes.
  • :471 — "…press_key "ArrowDown" then press_key "Enter" to accept the first match" for
    autocomplete fields.

So Enter-pressing is normal, encouraged behaviour during form filling. On the last field of a
single-step form, that submits a real job application during a run the user asked to be a test.

Gap 2 — the blocklist misses common submit labels

/\bsubmit\b|\bfinish\b|\bdone\b|\bcomplete\b|\bconfirm\b|send application|submit application|easy apply|quick apply|one[- ]?click|1[- ]?click/i

Not matched: a button labelled just "Apply", or "Send", or anything not in English —
Enviar, Postuler, Bewerben, 応募する. easy apply and quick apply are listed but bare
apply is not.

Deny-by-pattern fails open for everything unlisted, on forms that can be in any language.

Why this matters more than the odds suggest

The action is irreversible and external: a real application, to a real employer, under the user's
name, during a run they explicitly labelled a test. And the guard exists precisely because the
prompt is not trusted — :486 already tells the model "DO NOT SUBMIT … (Submit clicks are blocked
anyway.)". That parenthetical is currently only true of clicks.

Suggested shape

  • Guard every action type in dry run, not just click. At minimum block key when the key is
    Enter/NumpadEnter. navigate is worth a look too (a GET-based submit is unlikely but the
    same reasoning applies).
  • Invert the click rule. Rather than blocking labels that look like submit, consider blocking
    any click the snapshot identifies as a submit control (type=submit, role=button inside the
    form's action row), or requiring the brain to call finish(ready) once it believes the form is
    complete and blocking all further interaction after that point. An allowlist of safe actions is
    the only version that holds in a language the regex author didn't think of.
  • Add \bapply\b and \bsend\b to the regex as an immediate mitigation, but treat it as a
    stopgap — it does not fix Gap 1 and does not fix non-English forms.

Verification

  • A dry run where the brain calls press_key("Enter") is blocked with the same
    re-issue-and-continue behaviour clicks get today.
  • A dry run against a form whose submit button reads "Apply" (or "Enviar") does not submit.
  • A real (non-dry) run is unaffected — every one of these must still work normally.
  • Mutation-check: remove each guard and the corresponding test must fail.

Activity

  1. serge-ivo commented on Aug 6, 2026

    @serge-ivo
    ContributorAuthor

    Code audit: closing as already fixed in the current job-apply path. The workflow delegates actions through runApplyLoop, and runApplyLoop now blocks dry-run Enter/Return before deps.act is called, including first-action Enter on a pre-filled one-page form. It also blocks post-fill submit clicks for apply/send/submit/finish/done/complete/confirm/accept/agree labels. Tests exist in workers/api/src/lib/apply-loop.test.ts for post-fill Enter and first-action Enter. The remaining workflow-level helper is click-only, but it is a secondary stateless guard; the action path is guarded by the stateful loop before runner execution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions