[browser] First-party Web Task catalog agent #73

Activity

1. serge-ivo

   added

   browser-agentsBrowser automation generalizationBrowser automation generalization

   on Aug 1, 2026
2. serge-ivo

   added a commit that references this issue on Aug 6, 2026

   feat(agents): a browser agent that cannot click Pay is one you can po…

   ...

   2fd8d9f
3. serge-ivo commented on Aug 6, 2026

   

   serge-ivo

   on Aug 6, 2026

   ContributorAuthor

   More actions

   Shipped in 2fd8d9f — but deliberately narrower than this ticket asked for, so the reasoning matters more than the diff.

   What changed about the scope

   Everything needed to build the general agent already landed (#70 engine, #71 BrowserTaskWorkflow, #72 /browse + browser.task, plus a run_browse trigger action and a Run control on the Board). So the open question was never "can we" — it was "should the storefront carry it". Three things said no:

   1. A free-text objective box is not a promise. A catalog listing has to say what it will do for you; "whatever you type" is a text field, and nobody subscribes to a text field. It is also unreviewable — there is no version of "any objective on any site" a reader can judge safe before spending tokens on it.
   2. The catalog already has this ticket's own example. facebook-friend-confirmer is a published browser-task agent that accepts friend requests. A blanker second copy is a duplicate — and this epic's own non-goals rule out exactly that shape (mass-action automation, arbitrary-domain autonomy without approval).
   3. Epic: first-party exemplary agents → mature the platform → open to third parties (demand-gated) #58 is mature-first-party-then-open. Maturity means the agent is good, not that the engine is reachable. With 14 published agents and no external users, a 15th mediocre one makes the storefront worse.

   So this ships one web task worth someone else's money instead.

   Portal Watch (portal-watch, migration 0087)

   Reads the accounts that only tell you things after you log in — energy and water, council rates, an insurance renewal, a visa or case status, a supplier or school portal — in the subscriber's own signed-in browser, and reports the actual figures and dates.

   That is the exact intersection where a server-side connector cannot help: no API, no useful email, and the thing granting access is a session on your machine. It is a recurring chore rather than a demo, and a run_browse cron trigger does the monthly round unattended.

   Read-only is the product, not a limitation. config.browserTask.readOnly is declared on the agent row and read from there and nowhere else — not the request body, not the trigger config, not the instance config — and the workflow's act layer refuses every committing click before it reaches the page. So it cannot pay, submit, send or delete even if the model decides to, a subscriber asks it to, or hostile page text tells it to. Same fixed-at-declaration-time principle an actionable ticket uses for its action. readOnly and the per-run dryRun share one commit-verb guard, because two would eventually disagree about what "committing" means and the weaker one would be the hole; readOnly wins when both are set, so a permanent property is never described to the model as a rehearsal it could retry.

   Two limits stated plainly, because an unverifiable safety claim is worth nothing:

   • The guard covers committing clicks. Typing and Enter still work (a watcher needs a search box), so the guarantee is "it cannot click Pay", not a formal sandbox.
   • It is fail-safe, so a filter button literally labelled "Apply" is refused too.

   100% declarative. There is no portal-watch code anywhere in the monorepo. The agent is one row: declared capabilities (#141) pick the browser runtime and the BROWSER_TASK brain, settingsSchema carries the two things that differ per subscriber, and identity.goal is the objective. A sibling (shipment tracker, case-status watcher) is another row, not another PR.

   The three handoffs come from the shared loop unchanged: captcha/2FA → live takeover; a widget it can't operate → one human step; a value it wasn't given → ask-and-hold (the prompt forbids fabricating).

   Also added: config.browserTask.startUrlSetting, so the agent nominates which typed setting holds the start URL. The URL is the one thing that differs per subscriber, and without it the URL had to be retyped on every manual run and duplicated into every cron config, where the two could disagree about what the agent watches.

   Deliberately scoped out

   • Change detection / notify-on-change. Today a run reports what it sees; it doesn't say "this changed since last time", which is what would make it worth subscribing rather than merely useful. The right shape is a browse pipeline step so the result can flow into dedupe_upsert with emitOn:"update" and out through the delivery pump — that belongs with opening the step vocabulary (Epic: first-party exemplary agents → mature the platform → open to third parties (demand-gated) #58), not a bespoke branch of the workflow. (Note apply doesn't notify on completion either, so this isn't a hole I introduced.)
   • The experimental/admin flag this ticket asked for. The thing that flag was protecting against is a general engine that can do anything on any site — which is precisely what got scoped out. A read-only agent that cannot commit is seeded published like every other first-party agent. Happy to gate it if you disagree.

   Tests — 39 new, all green locally

   • browser-task-loop.test.ts (23) drives the real loop, prompt, tool mapping and guard with only the browser and the model faked, composing the act layer exactly as the workflow does — a guard that ran after the runner call would still return the right string and be useless. Covers all three handoffs and both mutation directions of the guard.
   • portal-watch-seed.test.ts (16) parses what the migration will actually insert and resolves it through the real registry. A typo there doesn't fail a build — it makes the agent fall back to legacy derivation and come up as a chat agent with no browser. Mutation-checked.

   Full suite 4145 passed, pnpm typecheck clean, biome clean on changed files, pnpm docs:drift clean (no new routes; /browse is in the intentionally-excluded set).

   What remains before this closes

   1. CI has not run yet. GitHub created zero check suites for 2fd8d9f and for eea3f10 before it — Actions isn't dispatching on this repo right now; not commit-specific. Every CI step was run locally instead (check-agents-allowlist, check-qa-config, check-test-isolation, docs:drift, test, typecheck, biome) and all pass.
   2. Acceptance criterion 1 is unverified by me — subscribe → pags up → real logged-in portal → the agent acts and hands off. That needs a real machine, a real account and a BYOK key, so it's an owner-only check.
   3. Minor UX edge, not mine to fix: the Board's "Run browser task" control disables Run until you type a URL, so the portal_url setting only feeds the cron/API paths on a manual first run (localStorage pre-fills it afterwards). store/console/src/tabs/BoardTab.tsx is owned by a concurrent session, so I left it alone — one-line fix to fall back to the setting.

   Leaving open pending (1) and (2).
4. serge-ivo commented on Aug 7, 2026

   

   serge-ivo

   on Aug 7, 2026

   ContributorAuthor

   More actions

   Closing as delivered. Verified rather than taken from the comment:

   • 2fd8d9f is on main — "a browser agent that cannot click Pay is one you can point at a real account ([browser] First-party Web Task catalog agent #73)".
   • workers/api/migrations/0087_seed_portal_watch_agent.sql is present, so the agent is a seeded catalog row.
   • workers/api/src/workflows/browser-task.ts exists and BROWSER_TASK is in KNOWN_WORKFLOWS (agent-capabilities.ts:362), so the declared capability resolves.
   • Children [browser] Extract the generic browser-task engine from the apply loop #70, [browser] BrowserTaskWorkflow + objective-driven system prompt #71, [browser] POST /v1/instances/:id/browse + browser.task runner type #72 are all closed.

   The scope narrowing (Portal Watch instead of a general free-text Web Task agent) is argued in the shipping comment and the residue was explicitly reassigned: change-detection goes to the browse pipeline step under #58, and the experimental flag became moot once the general engine was scoped out.

   The one thing left dangling is an open offer — "Happy to gate it if you disagree" — which is a question for the owner, not tracked work. Reopen or file fresh if the answer is yes.
5. serge-ivo

   closed this as completedon Aug 7, 2026