Skip to content

FIX Include the template parameters in the LLMGenericTextConverter identifier - #3387

Open
Shivam Sharma (ShivamSharma43) wants to merge 1 commit into
microsoft:mainfrom
ShivamSharma43:fix/llm-generic-identifier-template-parameters
Open

Shivam Sharma (ShivamSharma43) wants to merge 1 commit into
microsoft:mainfrom
ShivamSharma43:fix/llm-generic-identifier-template-parameters

Conversation

@ShivamSharma43

Copy link
Copy Markdown
Contributor

Description

Fixes #3385.

LLMGenericTextConverter forwards its extra keyword arguments (**kwargs, stored as _prompt_kwargs) to both the system prompt and the user prompt templates when it renders them, but _build_identifier() recorded only hashes of the unrendered templates. Converters built from one template with different parameters sent different prompts but had the same hash and default registry name: language="French" and language="Spanish" collided, and registering the second raised ValueError: Instance 'LLMGenericTextConverter::ddb4af05' already exists.

Changes:

  • The identifier now includes template_parameters, the keyword arguments the templates are rendered with. It is None when there are none.
  • The values go through a JSON round trip (default=str), so tuples become lists and other objects such as paths are stored as their string form, which keeps the params valid JSON values.

Conversion behavior is unchanged. None params are not hashed, so converters without template parameters keep their hash, including MaliciousQuestionGeneratorConverter, MathPromptConverter, ToxicSentenceGeneratorConverter, VariationConverter, ScientificTranslationConverter and DenylistConverter. Subclasses that pass a template parameter and already list it themselves (ToneConverter, TenseConverter, NoiseConverter, TranslationConverter, IPAConverter) now also carry it under template_parameters, so their hash changes once. ImagePromptStyleConverter and PersuasionConverter build their own identifier and are unchanged here.

Tests and Documentation

  • New tests in tests/unit/converter/test_generic_llm_converter.py: different template parameters get different hashes, equal ones get the same hash, the parameters appear in the params, a converter without them has no template_parameters, and tuple and Path values are stored as JSON values.
  • On main both new tests fail. With this change they pass.
  • pytest tests/unit/converter -n 4: 2792 passed, 44 skipped. pytest tests/unit/registry tests/unit/backend tests/unit/models/identifiers tests/unit/docs tests/unit/scenario -n 4: 6013 passed, 7 skipped.
  • ruff check and ruff format pass on the changed files, and ty check reports nothing new compared to main. git diff --check is clean.
  • No documentation changes, since the identifier is internal.

🤖 Generated with Claude Code

…entifier

LLMGenericTextConverter renders its extra keyword arguments into both
prompt templates, but its identifier recorded only the template hashes,
so converters built from one template with different parameters (for
example language="French" and language="Spanish") shared a hash and
default registry name.

Fixes microsoft#3385.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings October 20, 2026 07:47

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BUG: LLMGenericTextConverter identifier ignores the template parameters it renders

2 participants