Skip to content

fix(bedrock): degrade to empty args on malformed streamed tool JSON#2230

Open
anxkhn wants to merge 1 commit into
kagent-dev:mainfrom
anxkhn:patch-14
Open

fix(bedrock): degrade to empty args on malformed streamed tool JSON#2230
anxkhn wants to merge 1 commit into
kagent-dev:mainfrom
anxkhn:patch-14

Conversation

@anxkhn

@anxkhn anxkhn commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

What

In the Bedrock streaming path, tool-call arguments are accumulated across
contentBlockDelta chunks and then parsed with json.loads, but without the
json.JSONDecodeError guard that every other provider uses. When a streamed
tool-argument JSON is truncated or malformed, the json.loads raises inside the
response generator, gets caught by the broad except Exception that wraps the
whole method, and the entire turn is returned as an API_ERROR.

This PR wraps that single json.loads in try/except json.JSONDecodeError and
falls back to {}, so a recoverable-but-malformed streamed tool call degrades to
empty args instead of aborting the turn.

Why

Every sibling provider already handles this exact situation gracefully, so
Bedrock streaming is the odd one out:

  • OpenAI, non-streaming: _openai.py (try/except json.JSONDecodeError: args = {})
  • OpenAI, streaming: _openai.py (same guard on the accumulated tool call)
  • SAP AI Core, streaming: _sap_ai_core.py (same guard)
  • Bedrock, non-streaming: _bedrock.py reads an already-parsed dict
    (tool.get("input", {})), so it never raises here.

Only the Bedrock streaming branch parsed the accumulated input_json with a bare
json.loads. Streamed tool arguments are a realistic place for truncation (early
stream termination, or a model emitting slightly malformed JSON), so the
inconsistency is user-visible: the same partial tool call that recovers on
OpenAI/SAP fails the whole turn on Bedrock streaming. This change restores the
behavior the rest of the models package already standardized on.

How

python/packages/kagent-adk/src/kagent/adk/models/_bedrock.py:

for tool_id, tool in tool_uses.items():
    try:
        args = json.loads(tool["input_json"]) if tool["input_json"] else {}
    except json.JSONDecodeError:
        args = {}
    part = types.Part.from_function_call(name=tool["name"], args=args)

The try/except is copied verbatim from the sibling providers so the behavior is
identical across the package.

Testing

Added a colocated regression test,
test_streaming_malformed_tool_input_degrades_to_empty_args, in
python/packages/kagent-adk/tests/unittests/models/test_bedrock.py. It drives the
real generate_content_async(..., stream=True) path with a mocked
converse_stream that emits a toolUse whose accumulated input is truncated
({"city": "Paris"), and asserts the turn completes cleanly:
error_code is None, the function-call name/id are preserved, and
args == {}. Bedrock (AWS) is fully mocked, so no live credentials are needed.

  • Before the fix, this test fails: the turn comes back as API_ERROR
    ("Expecting ',' delimiter").
  • After the fix, it passes.
cd python
uv run pytest packages/kagent-adk/tests/unittests/models/test_bedrock.py   # 34 passed
uv run pytest packages/kagent-adk/tests/unittests/models/                  # 159 passed, 1 skipped
make lint                                                                   # clean

No CRD or Go API change, so no make generate and no E2E changes are required.

In the Bedrock streaming path, tool-call arguments are accumulated across
contentBlockDelta chunks and parsed with json.loads, but without the
JSONDecodeError guard that every sibling provider uses (OpenAI streaming and
non-streaming, SAP AI Core streaming). A truncated or malformed streamed
tool-argument JSON therefore raises inside the generator and aborts the whole
turn as an API_ERROR, instead of degrading to empty args like the other
providers and the Bedrock non-streaming path.

Wrap the json.loads call in try/except json.JSONDecodeError and fall back to
{}, matching the existing behavior across the models package. Add a regression
test that streams a truncated toolUse input and asserts the turn completes with
empty args rather than an error.

Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
@anxkhn anxkhn requested a review from a team as a code owner July 13, 2026 23:11
Copilot AI review requested due to automatic review settings July 13, 2026 23:11
@github-actions github-actions Bot added the bug Something isn't working label Jul 13, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR aligns the Bedrock streaming tool-call parsing behavior with the other model providers by preventing malformed/truncated streamed tool-argument JSON from aborting an entire turn. Instead, malformed tool args degrade to {} so the turn can complete successfully.

Changes:

  • Guard Bedrock streaming json.loads(...) with try/except json.JSONDecodeError and fall back to {}.
  • Add a regression test that simulates a truncated streamed tool input and asserts the turn completes with empty args.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
python/packages/kagent-adk/src/kagent/adk/models/_bedrock.py Wraps streamed tool-argument JSON parsing to degrade to {} on JSONDecodeError instead of failing the turn.
python/packages/kagent-adk/tests/unittests/models/test_bedrock.py Adds a streaming regression test for malformed tool input; includes a small cleanup opportunity (nit) around unused to_thread patching.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +324 to +336
async def fake_to_thread(fn, **kwargs):
return fn(**kwargs)

request = mock.MagicMock()
request.model = "us.anthropic.claude-sonnet-4-20250514-v1:0"
request.contents = []
request.config = None

with (
mock.patch("kagent.adk.models._bedrock._get_bedrock_client", return_value=mock_client),
mock.patch("kagent.adk.models._bedrock.asyncio.to_thread", side_effect=fake_to_thread),
):
responses = [r async for r in llm.generate_content_async(request, stream=True)]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants