Skip to content

feat(google): add response_mime_type and response_schema params#5064

Open
Panmax wants to merge 2 commits intolivekit:mainfrom
Panmax:gemini-llm-json
Open

feat(google): add response_mime_type and response_schema params#5064
Panmax wants to merge 2 commits intolivekit:mainfrom
Panmax:gemini-llm-json

Conversation

@Panmax
Copy link
Contributor

@Panmax Panmax commented Mar 10, 2026

Add instance-level output format control to the Google GenAI LLM plugin. Previously, structured output could only be configured per-call via the chat() method's response_format parameter. In scenarios where the LLM is consistently used to produce JSON responses (e.g. extracting structured data, building tool-calling pipelines), having to pass response_format on every chat() call is repetitive and error-prone.

In our scenario, the agent's prompt requires the model to output JSON, which is then parsed in the tts_node to let the agent read out the content, and send an RPC to the frontend, so we need instance-level control over the JSON output.

With these two new init parameters:

  • response_mime_type: controls the output MIME type (e.g. "application/json") without requiring a schema, useful for free-form JSON output.
  • response_schema: constrains the JSON output to a specific structure (Pydantic model or Google SchemaUnion), automatically sets mime type to "application/json" if not explicitly provided.

The chat() response_format parameter still takes precedence when provided, preserving full backward compatibility.

…M.__init__

Add instance-level output format control to the Google GenAI LLM plugin.
Previously, structured output could only be configured per-call via the
chat() method's response_format parameter. In scenarios where the LLM is
consistently used to produce JSON responses (e.g. extracting structured
data, building tool-calling pipelines), having to pass response_format on
every chat() call is repetitive and error-prone.

With these two new __init__ parameters:
- response_mime_type: controls the output MIME type (e.g. "application/json")
  without requiring a schema, useful for free-form JSON output.
- response_schema: constrains the JSON output to a specific structure
  (Pydantic model or Google SchemaUnion), automatically sets mime type to
  "application/json" if not explicitly provided.

The chat() response_format parameter still takes precedence when provided,
preserving full backward compatibility.
devin-ai-integration[bot]

This comment was marked as resolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant