Skip to content

BUG - ContextFilterPlugin sends orphaned tool call message to LLM causing failed API requests. #4027

Closed

Description

@MahorShekhar

I'm using ContextFilterPlugin with num_invocations_to_keep = 2 but getting error

openai.BadRequestError: Error code: 400 - {'error': {'message': "Invalid parameter: messages with role 'tool' must be a response to a preceeding message with 'tool_calls'.", 'type': 'invalid_request_error', 'param': 'messages.[1].role', 'code': None}}

My Debugging:

In ContextFilterPlugin

Line 64-65: It counts ALL model turns in the conversation
 Line 66-78: It walks backwards from the end, counting model turns, and when it finds the Nth model turn (counting backwards), it includes "user messages immediately before it

  Here's the bug with my conversation:

 Request log shows this sequence:
  1. user: "For context: [system] said: Session Created."
  2. user: "Hello"
  3. model: "Hi there! 😊 How can I assist you with Instrumentation Cables today?"
  4. user: "I want to know about instrumentation RFP"
  5. model: function_call (knowledge_base)  <--- MODEL TURN 1 (counting backwards)
  6. user: function_response                 <--- Tool response
  7. model: "I found some information..."    <--- MODEL TURN 2 (counting backwards)
  8. user: "can you explain more about quality plan"
  9. model: function_call (knowledge_base)   <--- MODEL TURN 3 (counting backwards) 
  10. user: function_response                <--- Tool response (THIS IS WHERE WE ARE NOW)

  With num_invocations_to_keep=2, the plugin logic does:

  1. Count backwards from the end
  2. Find the 2nd model turn from the end → that's message #7 ("I found some information...")
  3. Set split_index at #6 (user: function_response)
  4. Include everything from index 6 onwards: [6, 7, 8, 9, 10]

  Result: The contents array becomes:
  - user: function_response (orphaned! no preceding function_call)
  - model: "I found some information..."
  - user: "can you explain more about quality plan"
  - model: function_call (knowledge_base)
  - user: function_response

  The bug is: The logic counts individual model messages as "invocations", NOT complete user→model interaction cycles with tool calls!

  When there's a tool call, I get:
  - model: function_call
  - user: function_response
  - model: actual_response

  That's 3 messages but should be treated as 1 invocation.

  Request Contents: [
    {'parts': [{'text': 'For context:'}, {'text': '[system] said: Session Created.'}], 'role': 'user'},  # 1
    {'parts': [{'text': 'Hello'}], 'role': 'user'},  # 2
    {'parts': [{'text': 'Hi there! 😊 How can I assist you with Instrumentation Cables today?'}], 'role': 'model'},  # 3 - MODEL TURN 1
    {'parts': [{'text': 'I want to know about instrumentation RFP'}], 'role': 'user'},  # 4
    {'parts': [{'function_call': {...}}], 'role': 'model'},  # 5 - MODEL TURN 2 (function call)
    {'parts': [{'function_response': {...}}], 'role': 'user'},  # 6 - tool response
    {'parts': [{'text': "I found some information..."}], 'role': 'model'},  # 7 - MODEL TURN 3 (final answer)
    {'parts': [{'text': 'can you explain more about quality plan'}], 'role': 'user'},  # 8
    {'parts': [{'function_call': {...}}], 'role': 'model'},  # 9 - MODEL TURN 4 (function call)
    {'parts': [{'function_response': {...}}], 'role': 'user'}  # 10 - tool response
  ]

  Total model turns = 4 (#3, #5, #7, #9)

  With num_invocations_to_keep=2:
  - Plugin walks backwards looking for 2nd model turn
  - 1st model turn (backwards): #9 (function_call)
  - 2nd model turn (backwards): #7 (text response)
  - Sets split_index to include user messages before #7
  - Goes back to include #6 (function_response)
  - Final contents = [#6, #7, #8, #9, #10]

  This creates an orphaned function_response at #6 without its corresponding function_call at #5!

  Root cause confirmed: The ContextFilterPlugin treats each role='model' message as an invocation, but when a model makes a tool call, it creates TWO model messages:
  1. model: function_call
  2. model: final_response (after tool execution)

  These should be treated as ONE invocation, not two separate ones!

@Jacksunwei @wyf7107 @DeanChensj please address this bug in the next release

Activity

  1. added
    core[Component] This issue is related to the core interface and implementation
    on Dec 26, 2025
  2. added theissue type on Dec 26, 2025
  3. bluemeaford commented on Dec 29, 2025

    @bluemeaford

    Thanks for the detailed analysis — this makes sense. The filter appears to count individual model messages rather than full tool-call cycles, which can leave orphaned tool messages. A safe invariant might be to ensure filtered history never includes a tool message unless the corresponding assistant tool_calls is retained. I can try to put together a PR with a test if that approach sounds reasonable.

  4. self-assigned this
    on Dec 29, 2025
  5. added a commit that references this issue on Jan 5, 2026
    f755dca
  6. MahorShekhar commented on Jan 5, 2026

    @MahorShekhar
    Author

    @bluemeaford @GWeale I've raised a PR proposing a solution for the issue. Please check once.

    #4074

  7. added a commit that references this issue on Jan 5, 2026
    e32f017
  8. added a commit that references this issue on Aug 11, 2026
    b8c2936
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

core[Component] This issue is related to the core interface and implementation

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions