kmail.at
← learning

langchain · difficulty ◆◆

Consistent Tool-Call Chunks in the v1 Stream

Watch the function name type out, then stream clean JSON args.

Token-by-token tool calls should look exactly like the one-shot version. Now they do.

2026-06-13 · 6 min read

$ pip install -q langchain-openai==1.3.1

What it does

When you use ChatOpenAI with streaming enabled and call the model with a tool/function-calling schema, the model streams back partial chunks that represent the tool call in real time. This fix normalizes how those streamed tool-call chunks are formatted in API version v1, ensuring the output is consistent with non-streamed tool calls regardless of whether you receive the tool call all at once or token-by-token.

Why it matters

Tool calling is one of the most common production patterns, especially with gpt-4o and gpt-4o-mini via langchain-openai. If you build streaming UX (showing a function name being typed out live) or consume tool calls in an agent loop with streaming callbacks, you need reliable chunk shapes. This fix closes the compatibility gap between streamed and non-streamed output in the v1 API.

Example

$ Stream tool call chunks
Streaming tool call chunks:
  content chunk: ''
  tool_call_chunk: [{'name': 'get_weather', 'args': '{"city": ', 'index': 0}]
  tool_call_chunk: [{'name': '', 'args': '"Paris', 'index': 0}]
  tool_call_chunk: [{'name': '', 'args': '"}', 'index': 0}]

The name arrives in the first chunk and args stream token-by-token as the model generates JSON.

Common flags

AIMessageChunk.tool_call_chunks
Structured accessor for streamed tool-call fragments
--v1
OpenAI v1 API chunk normalization

History

Origin

Released in langchain-openai==1.3.1 and langchain-core==1.4.7 (2026-06-13), PR #35983.

Compatibility goal

The stream now yields clean, well-structured chunks that match the shape of a non-streamed AIMessage.tool_calls list.

Fun facts

Pros & cons

pros

  • + Streamed output matches non-streamed
  • + Clean incremental chunks
  • + Fewer edge-case bugs

cons

  • − Needs both core and openai updated
  • − Chunks still require accumulation logic

Takeaways

  1. 1Upgrade langchain-openai to 1.3.1 and core to 1.4.7.
  2. 2Inspect chunk.tool_call_chunks while streaming to see the normalized shape.
  3. 3Confirm parity by comparing astream() output with a plain invoke().

Related commands

← all learning