Skip to content

fix: stop dropping llm.system_message - #598

Merged
codelion merged 1 commit into
algorithmicsuperintelligence:mainfrom
fermibubble:fix-llm-system-message
Nov 16, 2026
Merged

codelion merged 1 commit into
algorithmicsuperintelligence:mainfrom
fermibubble:fix-llm-system-message

Conversation

@fermibubble

Copy link
Copy Markdown
Contributor

The prompt sampler only reads prompt.system_message, and load_config then copies that value to every model. So a system message set under llm: gets replaced by the default template, with no warning.

The Rust and R examples both set theirs under llm:, so their instructions never reach the model. On main, with examples/rust_adaptive_sort/config.yaml:

  • llm.system_message: "You are an expert Rust systems programmer specializing in high-performance algorithms..."
  • system prompt actually sent: "You are an expert software developer tasked with iteratively improving a codebase..."

I ran into this while using the Rust example as a template for a Java experiment.

Changes:

  • Config.from_dict uses llm.system_message as prompt.system_message when the prompt one is unset, and warns that the key belongs under prompt:. If both are set, prompt.system_message wins and the warning says llm.system_message is ignored.
  • The controller does the same for configs built in code, next to validate_evolution_mode(). A second call does nothing, so the warning shows once.
  • The Rust and R examples move their messages under prompt:, like the other 42 configs that set one.

Tests: new tests/test_system_message_config.py, 5 of its 6 tests fail on main. Full suite (python -m unittest discover -s tests -p "test_*.py"): 1082 OK, 7 skipped.

The prompt sampler only reads prompt.system_message, and load_config then copied that value to every model, so a system message set under llm was replaced by the default template without a warning. The Rust and R examples set theirs there, so their prompts never reached the model. Config loading (and the controller, for configs built in code) now uses llm.system_message for the prompt when prompt.system_message is unset, warns that the key belongs under prompt, and warns when both are set. The two examples move their messages under prompt.
@CLAassistant

CLAassistant commented Nov 12, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@codelion
codelion merged commit e4cadbb into algorithmicsuperintelligence:main Nov 16, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants