Skip to content

Fix for tool_calls finish reason when last streamed message is empty#4074

Open
dkalinowski wants to merge 1 commit intomainfrom
fix-empty-last-msg
Open

Fix for tool_calls finish reason when last streamed message is empty#4074
dkalinowski wants to merge 1 commit intomainfrom
fix-empty-last-msg

Conversation

@dkalinowski
Copy link
Collaborator

🛠 Summary

CVS-182445

Copilot AI review requested due to automatic review settings March 20, 2026 15:30
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes OpenAI-compatible streaming serialization so finish_reason is correctly reported as "tool_calls" even when the final streamed chunk is empty (a known behavior with some streamers where the last callback may produce "" but still carry STOP).

Changes:

  • Add per-stream state tracking (toolCallsDetectedInStream) to remember whether tool calls were emitted earlier in the stream.
  • Update serializeStreamingChunk() to map STOP to "tool_calls" if tool calls were detected in any prior chunk.
  • Add a regression test covering the “tool call ended with NONE, followed by empty STOP chunk” scenario.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
src/test/http_openai_handler_test.cpp Adds a regression test ensuring "finish_reason":"tool_calls" is emitted when an empty STOP chunk follows tool-call streaming.
src/llm/apis/openai_completions.hpp Introduces toolCallsDetectedInStream state to track tool-call presence across streamed chunks.
src/llm/apis/openai_completions.cpp Sets the state when tool-call deltas appear and uses it when mapping the final streamed finish_reason.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants