feat: agent-friendly setup + private API key naming - #55
Merged
scott-lowe-vapi merged 2 commits intoOct 1, 2026
Merged
Conversation
- npm run setup -- <org> [--region us|eu] [--resources all|none] runs setup without prompts; key comes from VAPI_TOKEN or an existing .env.<org> and is never accepted as a CLI flag - bare npm run setup without a TTY now explains the direct path instead of dying with 'User force closed the prompt' - direct setup refuses to overwrite an org that already exists locally - setup summary recommends validate/apply instead of push - add engines + .nvmrc (matches @inquirer/prompts' Node range) - README: remove leftover merge-conflict markers, document non-interactive setup, make 'How to Use This Repo' apply-first - AGENTS.md: add a first-time setup checklist for agents
steven-diaz-vapi
approved these changes
Oct 1, 2026
…l works) The dashboard calls these 'Private API Keys' (dashboard.vapi.ai/org/api-keys); 'VAPI_TOKEN' / 'API token' matched nothing a user could find. - add src/api-key.ts: one config-free resolver used by config, call, sim, rollback, promote, interactive and setup. Real env beats .env files regardless of name; within a source VAPI_PRIVATE_API_KEY beats VAPI_TOKEN - missing-key errors now link the dashboard page and name the right section - setup/updateEnvConnection write VAPI_PRIVATE_API_KEY and drop a legacy VAPI_TOKEN line, so re-running setup migrates old .env files - docs, .env.example and examples use 'private API key' + the new name
scott-lowe-vapi
changed the title
scott-lowe-vapi
added a commit
that referenced
this pull request
Oct 1, 2026
…CI (#56) ## Problem `npm test` isn't wired into any workflow (`.github/workflows/` only held `promotion.yml`), so nothing blocks a merge that breaks tests. Since the hash-store migration in #41, **20 of 357 tests have failed on `main`**, unnoticed. Bisecting first-parent `main`: 0 failures at #42, 20 failures from #41's merge onward, and no new failures since then (#55 included). ## Diagnosis: stale tests, not engine bugs #41 moved drift baselines out of the state file into `.vapi-state-hash/<org>/<uuid>`, made `upsertState` / `asResourceState` strip legacy fields, and made pull/push/apply refuse legacy-shaped state. The tests were never moved over. | Tests | Failures | Cause | |---|---|---| | `audit.test.ts` | 4 | Fixtures put `lastPulledHash` in state; `audit.ts` correctly reads the hash store now | | `state-migration.test.ts`, 2 in `drift.test.ts` | 4 | Asserted legacy fields survive state writes, which is exactly what the migration removed | | `reconcile-state-key.test.ts` | 4 | Asserted `lastPushedHash` lands in state; the shared push path records the baseline in the hash store instead | | `drift.test.ts` (`checkDriftForUpdate`) | 3 | Missing the new required `env` argument, so the hash-store path was `undefined` | | `drift.test.ts` (converged edge) | 1 | Pinned `both-diverged` for local == platform ≠ baseline, which `classifyDrift` now deliberately returns as `clean` (the documented invariant behind the phantom-drift fix, improvements.md #23) | | Pull/push spawn tests | 4 | Legacy-format state fixtures, refused by the migration gate | The last row matters most. Three of those were the regression guards #41 added for #22 (rename keeps the local filename; same-name clobber) and #23 (a stale baseline must not block a push). **They have never passed**, so those fixes had no working coverage. ## Changes - **CI**: `.github/workflows/ci.yml` runs `npm run build` + `npm test` on every PR and on pushes to `main`, on Node 20 and 22 (the `engines` range). It sets a job timeout and read-only permissions. - **Fixtures moved to the hash store**: - The spawn tests copy `src/` into a temp dir, so the engine's store resolves there; they seed `<tmp>/.vapi-state-hash/<env>/<uuid>`. - `drift.test.ts` seeds through `writeBaseline` under a throwaway `drift-test-<pid>` org and removes it afterwards. - `audit.ts` gains an optional `baselineReader` DI seam next to `stateLoader` / `listLocalIds`. That is the only production-code change, and the default is the existing `readBaseline(VAPI_ENV, uuid)`. - **`push-stale-baseline-noop` now tests what it says.** With empty `credentials`, `maybeBootstrapState` treats state as uninitialized and its bootstrap pull rewrote the stale baseline before the drift check ever ran. Migrating the fixture alone would have produced a green test that exercises nothing. It now seeds a dummy credential and asserts no bootstrap ran. - **Converged-edge test** rewritten to pin `clean`, with the rationale from `drift.ts`. - **Section J** (classifier short-circuit) rewritten around the hash store. The original bug (a rebuilt state section dropped the baseline) can't happen by construction now. The new test pins that separation, so moving the baseline back into the state entry fails it. - **`tool-assistant-cycle.test.ts`** deleted nothing it wrote: `updateToolAssistantRefs` records a baseline into the developer's *real* `.vapi-state-hash/test-fixture-org/` on every run. It now removes it. - `improvements.md` #32. ## Evidence - `npm test`: **355 / 355 pass** (was 337 / 357; two fewer tests because Section J's four tests became one and one `state-migration` test was split in two). `npm run build` is clean. - **Mutation check:** removing the agree-gate (`localHash === platformHash` in both `classifyDrift` and `checkDriftForUpdate`) fails `push-stale-baseline-noop` and the converged-edge test. Restoring it passes both. - **Isolation check:** after a full run, the real `.vapi-state-hash/` holds no test-written baseline. The `drift-test-<pid>` folder is removed, and the `tool-assistant-cycle` baseline file is deleted, leaving only an empty gitignored `test-fixture-org/` folder. ## Not in this PR `tsconfig.json` still includes only `src/**/*`, so the typecheck never sees `tests/`. That's why stale fixture shapes compiled silently. Including `tests/` surfaces **37 existing type errors**; that's its own change. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
feat: agent-friendly setup + private API key naming
Problem
Coding agents (Claude Code, Cursor, Codex) and CI run commands without a TTY. In that environment
npm run setup— the only documented way to bootstrap an org — dies with✗ Setup failed: User force closed the prompt with 0 nulland exits 0. There was no documented non-interactive path. Agents had to reverse-engineer.env.<org>+pull --bootstrapfrom the source, or ask the user to paste their private key into chat.Separately, the repo called the credential
VAPI_TOKEN/ "API token". That name doesn't appear anywhere in the dashboard, which calls it a Private API Key (https://dashboard.vapi.ai/org/api-keys), so new users didn't know where to get one.What this changes
Non-interactive setup.
npm run setup -- <org> [--region us|eu] [--resources all|none]VAPI_PRIVATE_API_KEY(or legacyVAPI_TOKEN), or from an existing.env.<org>. The recommended agent flow is: the human creates.env.<org>, so the key never passes through the agent.--token/--api-keyflags, since argv leaks into shell history and agent transcripts..env.exampleplaceholder as "no key."--region, elseVAPI_BASE_URL, else auto-detect (US, then EU), matching the wizard.--resources all(default) does a plain pull into the emptyresources/<org>/. It uses the engine's paginated list calls rather than the wizard's unpaginated snapshot fetch.--resources noneseeds state only (pull --bootstrap), which is the README's "template-safe first run."resources/<org>/or.vapi-state.<org>.jsonalready exists. The interactive wizard's "override" deletes local files, and that's not something an agent should do unprompted.VAPI_BASE_URLfor the child pulls, because process env beats.envfiles inconfig.ts.API key naming (commit 2).
VAPI_PRIVATE_API_KEYis now the canonical variable, andVAPI_TOKENkeeps working.src/api-key.tsresolves the key for every entry point:config,call,sim,rollback,promote,interactive, andsetup..envfiles, regardless of which name is used. Within a single source,VAPI_PRIVATE_API_KEYwins.setup(viaupdateEnvConnection) writesVAPI_PRIVATE_API_KEYand removes any legacyVAPI_TOKENline, so re-running setup migrates old.envfiles..env.example, and the cross-org examples now say "private API key" and use the new name.VAPI_TOKENto keep the diff small.TTY guard. Bare
npm run setupwithout a TTY now exits 1 and prints the direct-mode usage instead of the opaque prompt error.Apply-first guidance. The setup summary now suggests
validate/applyinstead ofpush. README "How to Use This Repo" no longer tells people topushafter setup, which contradicted the rest of the README.Node version. Adds
engines(^20.12.0 || ^22.13.0 || >=23.5.0, matching@inquirer/prompts) and an.nvmrcpinned to 22. Previously the only prerequisite was "Node.js installed."README fixes.
<<<<<<< HEAD/=======/>>>>>>>markers in Project Structure, keeping the newerlearnings/tree.AGENTS.md. Adds a "First-time setup (agents)" checklist and a Quick Reference row. It tells agents not to handle the key themselves, to ask whether to download existing resources, and not to delete local files to get around the "already set up" refusal.
.env.example. Mentions the direct-mode path.
Testing
tests/api-key.test.tscovers name precedence, legacy fallback, env-over-file ordering, and the error message.tests/setup-noninteractive.test.ts(12 tests, all passing) covers:--help.envvalue parsing and the placeholder checkVAPI_PRIVATE_API_KEYand legacyVAPI_TOKEN, that a bad key againsthttps://api.vapi.aiis picked up correctly, fails cleanly, and writes nothing.npm run buildpasses.npm run setup -- <test-org>andnpm run setup -- <test-org-2> --resources noneagainst a sandbox org before merging. Also run one existing command (e.g.npm run pull -- <org>) with an old.env.<org>that still usesVAPI_TOKEN.npm testshows 20 pre-existing failures that also fail onmainwithout this change (audit, drift, state-merge, pull-rename, and dep-dedup suites). Those are unrelated and probably worth their own issue.