Skip to content

Outbound Fleet reporting: report token scope, metrics --push, push after each Command (#2069) - #2086

Merged
JacobStephens2 merged 3 commits into
issue-2036from
issue-2069
Nov 12, 2026
Merged

JacobStephens2 merged 3 commits into
issue-2036from
issue-2069

Conversation

@JacobStephens2

@JacobStephens2 JacobStephens2 commented Nov 12, 2026 •

Copy link
Copy Markdown
Owner

Summary

A machine the watching Console cannot reach inbound, such as a sleeping or NATted laptop VM, now pushes its own Run metrics summary outward. It uses a holder-bound report token, both by hand and after each Command. The Console shows that machine read-only in Fleet Metrics.

sequenceDiagram
    participant Worker as lima-thirdshift (worker)
    participant Hub as always-on Console
    Note over Hub: console token create lima-thirdshift --scopes report
    Worker->>Hub: console report add hub <address> (token on stdin): GET /api/v1/whoami
    Hub-->>Worker: holder lima-thirdshift, scopes [report]
    Note over Worker: metrics --push, or Command finished (5 s, warn-only)
    Worker->>Hub: POST /api/v1/metrics/report {machine, view, summary}
    Hub->>Hub: machine == holder? not this Console's own name? save the holder's reported slot (locked, renamed in place)
    Hub-->>Worker: 200 {machine, pushed_at}
Loading
 Scope
   read, security, act, run, admin      # Scope::PERSON: a person's scopes, unchanged defaults
+  report                               # tokens only, alone, holder must be a Fleet name
 logs/fleet-metrics/<digest>.json
   Slot::Read(person)                   # #2068: a peer read, kept for one person
+  Slot::Reported                       # pushed by the machine itself; any reader of this Console sees it
 Fleet Metrics machine status
   ok | cached | unavailable | refused | incompatible | link-only
+  reported (read-only, pushed_at age; failed Commands named without detail links)

Worker side:

  • console report add|list|remove, CLI-only, stored in console.report in User config and [report] in Credentials.
  • metrics --push [<destination>], which takes the window and filter flags.
  • Started::finish pushes after a top-level Command. A skipped Pass and a child Run don't push.

ADR 0026 and the parity inventory list metrics --push and console report add|list|remove as CLI-only. The report endpoint's CLI pair is metrics --push.

Docs: docs/guide.md gains "Reporting outward" and the report row in the scope table. site/docs/index.html is regenerated. site/index.html, site/how-it-works/ and site/architecture/ are updated.

Evidence

Execution evidence

  • Before (base issue-2036, 3722e41):
    $ thirdshift console token create lima-thirdshift --scopes report
    thirdshift: unknown scope "report"; expected read,security,act,run,admin
    $ thirdshift metrics --push
    thirdshift: usage: thirdshift metrics [<command>] [--json] [--fleet | --machine <name>] …
    
    After:
    $ thirdshift console token create lima-thirdshift --scopes report
    ts_c68…
    $ thirdshift metrics --push
    thirdshift: no report destinations; pair one with thirdshift console report add <name> <address>
    
  • Acceptance tests: cargo test --test console metrics_report, 4 passed.
    • a_report_token_writes_only_its_holders_snapshot_and_reaches_nothing_else: uploads its holder's report; 403 for another machine; 403 on every OpenAPI operation needing read, security, act, run or admin (more than 40 checked).
    • metrics_push_shows_the_worker_read_only_on_the_receiving_fleet: pushes to a loopback Console; /metrics and ?machine=<holder> show reported, read-only, (just now), failures without links; a revoked token gives a 401 push failure.
    • a_finished_command_reports_its_metrics_and_an_offline_console_only_warns: a scripted Run pushes. With the Console stopped, the next Run exits 0 and logs warning: metrics not reported to hub: ….
    • a_report_from_this_hosts_own_name_is_refused_or_shown: the Spec reviewer's test. At 1eadbce it failed with report accepted with 200 but never shown; it now passes (422).
  • Parity: cargo test --bin thirdshift console:: passes, including every_cli_command_has_an_api_twin_or_reasoned_exclusion, adr_lists_every_cli_only_reason and every_console_route_has_a_cli_equivalent.
  • Site and docs: npm run docs:check --prefix site passes. npm test --prefix site passes 68 of 68. npm test --prefix console passes.
  • Affected-tests selection: scripts/affected-tests refs/remotes/origin/issue-2036 selected the full Rust suite. It ran with cargo test --no-fail-fast and every test passed except two pre-existing flakes:
    • process findings::merged_fix_is_rechecked_after_a_later_fix_fails fails 4 of 6 runs on the unmodified base issue-2036 too.
    • triage_pass recommendation_only_pass_reserves_without_claiming_or_publishing_and_notifies hit its "agent never started" deadline under full-suite load and passes alone.

Visual evidence

Publishing captures to a secret Gist was refused by this session's permission check, so no images are embedded. The captures were taken with Playwright Chromium at 1440×900 and 390×844. Each build served one hub home holding the same pushed laptop-vm snapshot (two failed and two succeeded Spec runs, /metrics?window=7d):

  • Before (base build), /tmp/before-desktop.NQJG88.png and /tmp/before-mobile.Vq8Dzv.png: the base reads the pushed file as its owner's own snapshot. It shows laptop-vm: cached, offline · last seen … · no longer in this Console's Fleet, and Where runs failed promises that "each name opens its details".
  • After, /tmp/after-desktop.yHmp5b.png and /tmp/after-mobile.fTFa4m.png: laptop-vm: reported, read-only · pushed 2026-11-12 07:26 UTC (just now), in the same muted colour as cached. Where runs failed lists widgets#72 (reported) on laptop-vm, widgets#73 (reported) on laptop-vm with no links, and says a cached or reported machine's details cannot be opened from here. The phone layout is unchanged apart from the longer intro text.

Unaddressed findings

Standards

  • 7. MetricsArgs.push and push_to as a data clump. Partly addressed: push_to is now documented (src/args.rs, MetricsArgs). The fields stay separate because the parity identity keys on the action alone (src/console/parity.rs: Command::Metrics(asked) => if asked.push { "push" } …), and push_to is an optional target of that action.
  • 8. MetricsReport::local leaves machine blank. Partly addressed: its doc now says the sender names each destination's holder. The summary is computed once and named per destination (src/console/report.rs, push), because each destination may have issued its token to a different holder; rebuilding it per destination would recompute trends from the logs for each one.
  • 12. FleetPage::age duplicates status::age. Declined. The Metrics page writes "N minutes ago" as page copy beside a "pushed " stamp, while src/logs/status.rs age produces the Inbox's compact "N min". Sharing one would change one page's wording.

Spec

  • 2. A skipped Pass does not push. Declined. A skipped Pass did no work, and no trend counts it: src/logs/metrics/outcomes.rs:60-64, idle_or_pass_through ("a skipped Pass: neither did work of its own, so no trend counts it"), and docs/guide.md:342. A push after it would resend an unchanged summary. The guide states the rule.

Security review outcome

No unaddressed introduced findings.

Review coverage

  • Fixed point: refs/remotes/origin/issue-2036, reviewed at 1eadbce.
  • Excluded as generated output: site/docs/index.html, generated from docs/guide.md. Both reviewers accepted the exclusion: npm run docs:check --prefix site is clean, and npm test --prefix site passes.
  • Changed after review, not read by a reviewer: tests/console/api.rs (report added to the allowed x-scope values), plus the review fixes in 6f037bd and e66994a.
  • Findings: Standards 13, of which 10 were fixed and 3 are listed above. Spec 3, of which 2 were fixed (own-name refusal, locked slot merge) and 1 is listed above.

Merge Danger

Door: two-way

Data written by this change:

  • the console.report User config key;
  • the [report] Credentials section;
  • reported snapshots, with a new optional pushed_at field, beside the existing files under logs/fleet-metrics/.

A build without this change reads a reported snapshot as a cached, retired machine, as the Before captures show. It rejects a User config that contains console.report.

Blast Radius: Fleet

What it touches:

  • Every top-level Command: on a machine with report destinations, each one now ends with an HTTP push of at most 5 seconds per destination. It only warns on failure and never changes the exit code.
  • Fleet pairing: now goes through the shared fleet::whoami probe. identity is optional, so older Consoles still pair.
  • Scopes: person scopes are unchanged; Scope::ALL is renamed Scope::PERSON.
  • Snapshot writes: each snapshot write now takes a per-file lock.

Closes #2069

Built with claude 2.1.314 · opus · default effort

…--push, a push after each Command, and reported machines read-only on Metrics (#2069)
…ch snapshot slot's merge, share the whoami probe, list writer and view keys, name person scopes (#2069)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant