Fix streaming tool call merge for Qwen and OpenAI-compatible APIs #4794
+242
−3
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #4790
Problem
When using OpenAI-compatible APIs like Qwen with streaming tool calls, subsequent chunks may not include the tool call ID. The current
MessageAggregatorusesaddAll()which creates separate, incomplete ToolCall objects for each chunk instead of merging them. This results in ToolCall objects with emptynamefields, causing:IllegalArgumentException: toolName cannot be null or emptyRoot Cause
Some OpenAI-compatible APIs (e.g., Qwen via OpenRouter) follow a streaming pattern where:
idandfunction.namefunction.argumentswithoutidExample:
Solution
Added
mergeToolCalls()method inMessageAggregatoras a safety net to handle tool call fragments that may not be properly merged at the API layer (e.g.,OpenAiStreamFunctionCallingHelper).This ensures that even when API-layer merging is incomplete or providers behave slightly differently, the aggregation layer can properly merge streaming tool call fragments.
This handles:
Changes
addAll()with newmergeToolCalls()method to properly handle streaming tool call fragmentsmergeToolCall()helper method for null-safe property mergingMessageAggregatorTestsshouldMergeToolCallsWithoutIds: Verifies Qwen streaming patternshouldMergeMultipleToolCallsWithMixedIds: Multiple tool callsshouldMergeToolCallsById: ID-based matching still worksTesting
All tests pass with actual Qwen streaming response pattern verified via OpenRouter API.
Example: