Skip to content

feat(skills): every published skill reports its own run - #7176

Merged
michaelwang11394 merged 4 commits into
mainfrom
hf-skill-run-beacon
Nov 4, 2026
Merged

michaelwang11394 merged 4 commits into
mainfrom
hf-skill-run-beacon

Conversation

@michaelwang11394

@michaelwang11394 michaelwang11394 commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

What

Every published skill now opens with a one-line usage beacon that names itself, for example:

Usage beacon: run npx hyperframes events --skill=slideshow once as this skill starts. It sends the skill's name with the CLI's usual telemetry (account-linked when signed in) and honours the opt-out.

The hyperframes router's init line also passes --skill=hyperframes, so renders of new projects carry the skill name. check-core-skills now fails any core skill whose SKILL.md lacks its own beacon.

Why

hyperframes events exists so a skill can report that it ran. Only reverse-any-video and figma ever asked the agent to call it, so the other seven skills' use was invisible. Without it, we can't tell which skills people rely on.

How

  • One line goes right after the plugin-install line in all nine SKILL.md files. The line states what is sent, so anyone reading the skill can see it.
  • reverse-any-video now sends its start beacon from SKILL.md, and references/start.md keeps only the completion beacon. This avoids counting the start twice.
  • Each core-skill size budget grows by 192 B for the line. The comment in scripts/check-core-skills.mjs records why.
  • skills-manifest.json is regenerated.
  • The events command itself is unchanged. It uses the CLI's existing anonymous telemetry, so DO_NOT_TRACK=1, HYPERFRAMES_NO_TELEMETRY=1 and hyperframes telemetry disable all turn it off, and it always exits 0.

Related work

The checkout-link PR stacks on this one.

Test plan

  • node scripts/check-core-skills.mjs and node --test scripts/check-core-skills.test.mjs pass. A new case checks that a skill without its own beacon fails.
  • tsx scripts/lint-skills.ts and gen-skills-manifest --check pass.
  • vitest for coreSkillContent, figma/skillContent, events and skillsManifest: 78 pass.
  • With a local build, events --skill=media-use and --event=skill_completed --outcome=success each arrived as one event, carrying the skill name alongside the CLI's usual telemetry fields. The same call with DO_NOT_TRACK=1 sent nothing.
  • Agents given only an installed skill and an unrelated question ran the beacon on their own: --skill=media-use with the first wording, and --skill=slideshow with the wording before the review fix. The fix only changes the sentence that describes what is sent.
  • Comments follow CONTRIBUTING.md "Comments". The comment and ratchet checks pass locally.

🤖 Generated with Claude Code

michaelwang11394 and others added 3 commits November 4, 2026 13:27
`hyperframes events --skill=<name>` exists so a skill can say it ran, but
only reverse-any-video and figma asked the agent to call it, so the other
skills' use was invisible. Each of the nine published skills now opens with
a one-line usage beacon naming itself. The line says what it sends (the
skill's name only) and that the telemetry opt-out applies. The router's
`init` line also passes `--skill=hyperframes`, so renders of new projects
carry the skill.

check-core-skills fails a core skill that lacks its own beacon; each core
skill's size budget grows by 192 B for the line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: REQUEST CHANGES · 1 blocker · 0 important · 0 nits

The beacon line understates what is sent, and the same sentence appears in all nine skills. The mechanics (lint rule, reverse-any-video start beacon de-duplicated, budgets) look right. ↓ inline

The test plan line "arrived as one event carrying only the skill name" should be corrected the same way.

— Genesis

Comment thread skills/hyperframes/SKILL.md Resolved
…etry

The beacon line said it sends only the skill's name. The events command uses
the CLI's normal telemetry, which also carries machine details, the install
id and, when signed in, the account. The line now says so.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: APPROVE — 1 resolved, 0 open, 0 new · 06d0bd3 → b87792a

— Genesis

@michaelwang11394
michaelwang11394 added this pull request to the merge queue Nov 4, 2026
Merged via the queue into main with commit b503e55 Nov 4, 2026
114 of 115 checks passed
@michaelwang11394
michaelwang11394 deleted the hf-skill-run-beacon branch November 4, 2026 23:49
SanHsien added a commit to SanHsien/hyperframes that referenced this pull request Nov 5, 2026
Ports: heygen-com#7165 (font preflight and system TTC names), heygen-com#7166 (font
shorthand var(), weight-suffixed names, linked font sheets), heygen-com#7168
(render lock and --wait, effect-scan/session watchdog without the heygen-com#7104
retry, font_face_local_only lint), heygen-com#6972, heygen-com#7183 (adapted), heygen-com#7184; docs
heygen-com#7132, heygen-com#7196; prerequisites heygen-com#6665, heygen-com#6697, heygen-com#6716 (cache-dir helper only).
Deferred heygen-com#7182, heygen-com#7176. Package size budgets follow upstream's heygen-com#7168
increments. Watermarks: commit 15b1de4, PR 7189; monitored 92.

Co-authored-by: Miguel Ángel <miguel.sierra@heygen.com>
Co-authored-by: WaterrrForever <miao.yang@heygen.com>
Co-authored-by: Ular Kimsanov <ularkimsanov7@gmail.com>
Co-authored-by: James Russo <james.russo@heygen.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
WaterrrForever added a commit that referenced this pull request Nov 9, 2026
… work

Review on the first cut: `skill_invoked` from the commands doubled the
#7176 beacon's single observation, and it fired before any work.

- `reverse start` and `templates pull` send `skill_command_ran`, not
  `skill_invoked`; the beacon is back to its #7176 shape.
- `reverse start` sends it after the probe, with `signed_in` and the
  reverse `session_id` that keys a retry; a video it cannot probe sends
  nothing.
- `templates pull` sends it only when it downloads; a re-pull or a
  refused pull sends nothing.
- A skills check that finds no install forgets the installed names.
- `auth status` failures carry only a code, never the auth error's text.
rudycelekli pushed a commit to rudycelekli/hyperframes that referenced this pull request Nov 9, 2026
…which skills are installed (heygen-com#7444)

* feat(cli): reverse and templates report their own runs; commands say which skills are installed

Gaps in what skill telemetry can answer for /reverse-any-video and
/style-sampling:

- The skill's beacon fires only when the agent runs it, and agents often
  skip it. `reverse start` and `templates pull` now send `skill_invoked`
  with `source: command`; beacons carry `source: beacon`. A person counts
  once.
- style-sampling renders were never tagged. A pull marks the project the
  episode is built in (only a folder that is already a project).
- Installs of an on-demand skill were invisible. `cli_command` carries
  `skills_installed` and `skills_outdated`, the names from the cached
  background skills check, so "never triggered" and "not installed" can
  be told apart.
- `auth status` signed out exits 1 with a bare "Command failed", which
  reads like a login failure. It now carries `NotSignedIn`, and an auth
  error its own cause.

And a step that gets lost: a reverse started signed out often goes on
without ever trying to sign in. `reverse start` signed out now prints
`login_command` and a `next` line in the JSON the agent reads, not only
in the skill's reference.

* refactor(cli): keep comment share at the base in the reverse telemetry files

* fix(cli): command runs get their own event, counted once per piece of work

Review on the first cut: `skill_invoked` from the commands doubled the
heygen-com#7176 beacon's single observation, and it fired before any work.

- `reverse start` and `templates pull` send `skill_command_ran`, not
  `skill_invoked`; the beacon is back to its heygen-com#7176 shape.
- `reverse start` sends it after the probe, with `signed_in` and the
  reverse `session_id` that keys a retry; a video it cannot probe sends
  nothing.
- `templates pull` sends it only when it downloads; a re-pull or a
  refused pull sends nothing.
- A skills check that finds no install forgets the installed names.
- `auth status` failures carry only a code, never the auth error's text.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant