Skip to content

Conversation

@ahmadafzal007
Copy link

Description

Fixes #34282

This PR fixes a bug in SummarizationMiddleware where ToolMessages could be retained while their parent AIMessages (tool calls) were summarized/removed, leading to 400 Bad Request errors from LLM providers.

Changes

  • Modified _find_safe_cutoff in summarization.py to backtrack and include the parent AIMessage if a orphaned ToolMessage is detected in the preserved message history.

Verification

  • Verified with a standalone reproduction script that simulated the split scenario.

@github-actions github-actions bot added langchain `langchain` package issues & PRs fix For PRs that implement a fix labels Dec 10, 2025
@ahmadafzal007 ahmadafzal007 changed the title fix(middleware): ensure parent AI messages are preserved during summa… fix(langchain): ensure parent AI messages are preserved during summarization Dec 10, 2025
@github-actions github-actions bot added fix For PRs that implement a fix and removed fix For PRs that implement a fix labels Dec 10, 2025
@ahmadafzal007
Copy link
Author

Issue Explanation: The "Orphaned Tool Result" Bug

What was the problem?

Imagine a conversation between a User, an AI, and a Tool (like a calculator).

  1. User: "Calculate 2 + 2"
  2. AI: "I need to use the calculator." (This is a Tool Call Request)
  3. Tool: "The answer is 4." (This is a Tool Call Result)

These last two messages (Request and Result) are a pair. They must always stay together. If you delete the Request but keep the Result, the AI gets confused because it sees an answer to a question it doesn't remember asking. This causes a "400 Bad Request" error.

The bug in LangChain's SummarizationMiddleware was that when the conversation got too long, it would try to "summarize" (delete and condense) old messages. Sometimes, it would cut the conversation right between the Request and the Result, deleting the Request but keeping the Result.

How did we reproduce it?

We created a script that simulated a conversation where this exact split would happen:

  1. User says something.
  2. AI asks these tools for help (The Request).
  3. User interrupts or adds more info.
  4. Tool gives the result (The Result).

We told the summarizer to "keep the last 2 messages".

  • The last 2 messages were the User interruption (3) and the Tool Result (4).
  • This meant the AI Request (2) was left behind in the "to be deleted/summarized" pile.
  • The script checked if we had a Result without a Request, and it confirmed the bug: "Orphaned ToolMessage found!"

How did we solve it?

We updated the SummarizationMiddleware code to be smarter.

Now, before it finishes deleting old messages, it checks the messages it plans to keep.

  1. It looks for any Tool Results.
  2. If it finds one, it checks if the matching AI Request is also being kept.
  3. If the AI Request is missing (because it was about to be deleted), the code changes its mind and moves the "cutoff point" back to include that AI Request.

This ensures that a Tool Result is never left alone without its parent AI Request.

Verification

We ran the reproduction script again with the fix.

  • Result: The script saw the Tool Result was being kept, so it automatically kept the AI Request too.
  • Output: "[SUCCESS] No orphaned ToolMessages found."

@ahmadafzal007 ahmadafzal007 force-pushed the ahmed/fix-summarization-middleware-orphan-tools branch from d40e9d5 to 58c584c Compare December 11, 2025 11:40
ahmadafzal007 added a commit to ahmadafzal007/langchain that referenced this pull request Dec 11, 2025
@ahmadafzal007 ahmadafzal007 force-pushed the ahmed/fix-summarization-middleware-orphan-tools branch 2 times, most recently from 67ab59d to bcd5a89 Compare December 11, 2025 12:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fix For PRs that implement a fix langchain `langchain` package issues & PRs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

After SummarizationMiddleware is triggered, agent can randomly triggers 400 - No tool call found

1 participant