Skip to content

Dato Agent: redact credentials from copied failure diagnostics - #161

Merged
roger-datocms merged 7 commits into
datocms:masterfrom
rodrigobarona:fix/dato-agent-diagnostics-redaction
Nov 4, 2026
Merged

roger-datocms merged 7 commits into
datocms:masterfrom
rodrigobarona:fix/dato-agent-diagnostics-redaction

Conversation

@rodrigobarona

Copy link
Copy Markdown
Contributor

Fixes #159.

What

serializeDiagnostics now redacts credentials and keeps the rest of the payload intact:

  • Non-empty string values under credential-like keys become [Redacted]. Matching ignores case and punctuation: suffix apikey, accesstoken, refreshtoken, secret or password; exact token, idtoken, codeverifier, authorization, bearer or cookie. Empty strings are kept, so a missing key is still visible. Numeric fields such as anthropicModelMaxOutputTokens are untouched.
  • sk-… API keys and Bearer … tokens embedded in any string are replaced.

This covers configuration.*ApiKey, activeApiKey and oauthCredentials.token.accessToken in the AgentFrame failure payload.

Tests

  • diagnostics.test.ts: the deterministic-output test now expects redaction. New test covers provider keys, OAuth token/refresh/PKCE fields, mcpAccessToken and embedded Bearer/sk- strings.
  • AgentFrame.test.tsx: the copied diagnostics no longer contain the API key or OAuth token.
  • npm run typecheck, npm run lint, npm test (952 passed).

Provider API keys, OAuth access/refresh tokens, PKCE verifiers and similar
fields are replaced with [Redacted] (empty values are kept), and sk-* keys or
Bearer tokens embedded in strings are scrubbed. Diagnostics are commonly
pasted into issues and chats, so they must not carry live credentials.

Co-authored-by: Cursor <cursoragent@cursor.com>
@roger-datocms

Copy link
Copy Markdown
Member

(Claude here [Opus 5.5])

Triage

The fix is still needed. On master (6f57d960), dato-agent/src/lib/diagnostics.ts has no redaction. AgentFrame.tsx:3228-3236 copies props.config, activeApiKey and oauthCredentials into the payload verbatim.

Status

  • Mergeable, state CLEAN, one commit (309223a8) on top of master 6f57d960. No CI checks run on this repo.
  • Local run on the PR head: npm test 952/952 pass, npm run lint clean, npm run build succeeds.

Review findings

  1. Gap: real Error instances skip string redaction. The Error branch (diagnostics.ts, current instanceof Error) copies message and stack raw and never calls normalize on them. AgentFrame.tsx:3276 passes thrownError, which is usually a real Error. I verified it: serializeDiagnostics({ error: new Error('Authorization: Bearer abc… sk-proj-…') }) keeps both secrets in message and stack. The new test only uses a plain-object error, so it misses this. Fix: run redactSensitiveSubstrings on message and stack, and add an Error case to the test.
  2. Map keys: Map entries bypass key-based redaction. A Map with an authorization key keeps its value. This is low risk for the current payload. A one-line check in the Map branch closes it.
  3. Duplicated patterns: the plugin already has the same regexes in conversations.ts:201-204, datoMcpClient.ts:113-114 and agentRuntime.ts:2447-2454. They use [redacted] (lowercase) and a 12-char sk- minimum. This PR uses [Redacted] and 16. Align the marker and threshold, or extract one shared helper.
  4. Not covered: key=value secrets inside strings, for example ?access_token=… in a URL. conversations.ts already handles this with SECRET_KEY_VALUE_PATTERN. Reusing it here closes that gap.

Key matching (isSensitiveKey) looks right. It keeps numeric fields like anthropicModelMaxOutputTokens and empty strings, as intended.

Recommendation

Fix finding 1 before merge, because it is the leak that matters most in practice. Findings 2-4 are cheap and can go in the same commit. After that, merge, then close #159.

…ey formats

Copied diagnostics still leaked credentials through several paths:
- Error message and stack went out raw.
- Map entries skipped the key-based rule.
- Query-string secrets (?access_token=, ?code=, ?key=) and api_key=… text
  were not matched.
- Gemini (AIza…), Vercel AI Gateway (vck_…) and DatoCMS CMA tokens were
  not matched.

Redact all of these, treat credentials/jwt/session/privateKey keys as
sensitive, and use the plugin's existing "[redacted]" marker and 12-char
sk- minimum. Harmless text such as "status code=401" is kept.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@roger-datocms

Copy link
Copy Markdown
Member

(I'm working on this one, adding a few more secrets to redact before we merge)

roger-datocms and others added 5 commits November 3, 2026 14:27
Replace the secret-shape regexes with two structural guarantees:

- Allowlist: diagnostics list the non-secret configuration fields
  explicitly (diagnosticConfiguration) and report API keys only as
  hasOpenAiApiKey/hasAnthropicApiKey. OAuth state is reduced to version,
  client registration and token metadata (diagnosticOAuthCredentials);
  the access token is never included. New secret fields can't leak by
  default.
- Known-value masking: serializeDiagnostics takes the frame's live
  secrets (both provider keys and the OAuth access token) and replaces
  every exact occurrence anywhere in the payload, including error
  messages, stacks, URLs and map values, whatever their format.

The credential-key list stays as a structural backstop for nested
objects such as headers.authorization.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…g values

Known secrets are already masked. For anything else that looks like a
credential, such as a key pasted into chat, "Copy diagnostics" now
lists the candidates (last 4 characters, length and payload path, never
the value) and lets the user choose which to redact. All start selected.

Detection is generic, with no provider patterns: tokens of 20+ characters
that mix letters and digits and have high entropy. DatoCMS IDs
(isValidId from @datocms/cma-client-browser), canonical UUIDs, URL
escapes and values under id/…Id/…Ids keys are skipped. With no
candidates, copying works as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
registerTerminalFailure is a per-render closure, so its oauthCredentials
could predate a token rotation while the turn used the newer token from
oauthCredentialsRef. An error message echoing that token reached the
clipboard unmasked. Also mask the live ref and the turn's
mcpAccessToken/repairOauthAccessToken, with a regression test.

Bump DIAGNOSTICS_SCHEMA_VERSION to 2: configuration and OAuth state now
have an allowlisted shape.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The live review flagged the project name (claude-…-2026-11-03) and a host
context fingerprint. Flag a token only if one of its -/_ segments mixes
letters and digits, as keys do (sk-ant-api03-…) and slugs don't; treat
fingerprint/hash/checksum/digest keys as identifiers. Keep each preview
on one line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each candidate now has a visible "Redact" label, and a hint under the
heading says ticked values are replaced with [redacted] in the copy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@roger-datocms

Copy link
Copy Markdown
Member

I pushed a few followup fixes to this:

  • It checks secrets against the secrets values before logging (instead of via regex), since the plugin knows what they are
  • Move diagnosticConfiguration() to an allowlist model to only log specified keys
  • The credential-key-name list from Rodrigo's commit stays, for nested objects such as headers.authorization
  • Things that "look like secrets" (but are not UUIDs or Dato IDs) will ask the user to confirm redaction before copying, interactively

I didn't want to absorb a full security sweeper like Gitleaks into this plugin, so hopefully this is good enough for now.

pr-161-walkthrough.mp4

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Dato Agent: "Copy diagnostics" exports provider API keys and OAuth tokens in plain text

2 participants