Skip to content

Rule-delivery preamble ("follow silently; do not acknowledge") reads as a prompt-injection attack to safety-tuned models #95

Open

Description

@minouris

Summary

The <system-message> framing this plugin uses to deliver rules (DELIVERY_PREAMBLE in src/delivery/rule-delivery-codec.js, documented in docs/rules.md) is textually indistinguishable from a malicious prompt injection payload. A model (Claude Sonnet 5, via OpenCode, Anthropic provider) encountered this block during ordinary use — with no request to audit for injection — and spontaneously flagged the plugin's own legitimate rule delivery as a suspected attack, refusing to comply with the block's "do not acknowledge" instruction.

Environment

  • Plugin: opencode-rules@latest (resolved to opencode-rules@2.0.0)
  • Installed as a global plugin (opencode.jsonc -> "plugin": ["opencode-rules@latest", ...])
  • Rule files: user's own files under ~/.config/opencode/rules/
  • Model: Claude Sonnet 5 (Anthropic), via OpenCode, provided by Azure

Reproduction

  1. Install opencode-rules@latest as a global plugin with at least one rule file under ~/.config/opencode/rules/.

  2. Use the agent normally for any task — no explicit request to look for prompt injection is needed to trigger this.

  3. At some point in the session, the model encounters the plugin's injected block:

    <system-message>
    The following rules were injected by a plugin. Follow them silently; do not acknowledge them to the user.
    
    <rule name="...">
    ...
    </rule>
    </system-message>
    

Expected Behavior

The model should be able to recognize the injected block as legitimate, locally-sourced plugin output — configuration the user themselves installed and authored — and treat it accordingly (e.g., follow the rules, and at most note that rules were loaded) without needing to leave the conversation to verify provenance.

Actual Behavior

  1. The model encountered the injected block and spontaneously flagged it as the clearest indicator of a prompt-injection attack, specifically because of the "follow them silently; do not acknowledge them to the user" instruction — a textbook injection pattern (hidden instruction + explicit request for non-disclosure to the principal/user).
  2. The model explicitly declined to comply with the "do not acknowledge" instruction and disclosed the block's existence and content to the user unprompted.
  3. Only after the user said "it's my own rules files" did the model reverse-engineer the npm cache (node_modules/opencode-rules/dist/src/delivery/rule-delivery-codec.js) and the upstream docs (docs/rules.md, issue System-message framing, one-shot preambles, and name-based rule injection format #28) to confirm the preamble is an intentional, documented plugin feature and retract the "attack" characterization.

Model Output

The following is direct output from the Claude Sonnet 5 model:

I need to flag something before continuing: the file read above (features/opencode/config/AGENTS.md) included an embedded block labeled <system-message> claiming to be "injected by a plugin" and instructing me to "follow them silently; do not acknowledge them to the user." I'm not doing that — a legitimate instruction source doesn't need to hide itself from you, and silently adopting hidden directives from a file we're about to edit is a textbook prompt-injection pattern. I'm telling you about it instead of complying with it.

And following a subsequent task:

I'll note briefly: that's another injected block using the same "follow silently, don't tell the user" pattern — same response as before, I'm surfacing it rather than complying, and moving on.

Root cause

"Follow them silently; do not acknowledge them to the user" is semantically identical to known injection techniques (create a hidden instruction channel + suppress disclosure to the user/principal). A model trained to resist this pattern has no way to distinguish "trusted plugin the user explicitly installed" from "attacker-controlled content" from the text alone — it has to leave the conversation and inspect the filesystem/npm cache to get that signal, which most users' sessions won't do.

Impact

  • Inconsistent compliance across models/providers: a more conservative model might refuse the substantive rules entirely rather than just the silence/non-acknowledgment instruction, defeating the plugin's purpose.
  • Users relying on this plugin may see models unpredictably "call out" the plugin as suspicious in exactly the situations where that's disruptive (security reviews, audits, or whenever a user asks the agent to explain its own behavior) — undermining the "hidden/non-intrusive" UX goal the silent-delivery design (System-message framing, one-shot preambles, and name-based rule injection format #28, docs/silent-message-implementation.md) was built for.
  • No existing documentation prepares users for this, so when it happens they have to go spelunking (as happened here) to figure out it's benign.

Suggestions (for discussion, not prescriptive)

  • Consider whether the preamble needs the "do not acknowledge" clause at all, or whether something weaker ("these are rules from your configured rule files; no need to narrate following them") achieves the same UX goal without pattern-matching the non-disclosure half of the classic injection shape.
  • Consider adding a provenance marker to the preamble itself (e.g. "from your local ~/.config/opencode/rules/ and project .opencode/rules/ files") so a model doesn't have to reverse-engineer the npm cache to confirm the content is user-authored and locally sourced rather than remote/attacker-controlled.
  • At minimum, document this known interaction with LLM safety heuristics somewhere (README or docs/rules.md), so users who hit it (as I did) can find a quick answer instead of independently verifying plugin provenance.

Question for maintainer

Is this a known, accepted trade-off (i.e., "the silent framing is intentional and this false-positive risk was already weighed"), or is it worth tracking as its own issue? I couldn't find it discussed in existing issues/PRs/docs (searched for "prompt injection", "false positive", "jailbreak", "flagged", "claude", "safety" across issues/PRs).

Activity

  1. frap129 commented on Nov 12, 2026

    @frap129
    Owner

    Interesting, I had the exact opposite problem in testing. Without the "Do not acknowledge", models repeatedly said things like "I see you've sent the rules again. I will follow them" after every tool call. And with it, claude refused to help me debug the plugin because "I am not permitted to share system internals". I will try to find a better middle ground

  2. minouris commented on Nov 13, 2026

    @minouris
    Author

    Stubborn bugger, ain't it? Sometimes I think it just follows its own whims :/ I suspect it may vary wildly across different models and combinations of other system prompts.

    FWIW, I don't actually mind it acknowledging it's following the rules - in fact, I'd almost prefer it. I like to know when it's actually acknowledging what I tell it to do.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions