-
Couldn't load subscription status.
- Fork 840
Enable JSON-schema based structured output by default #6159
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
src/Libraries/Microsoft.Extensions.AI.Abstractions/ChatCompletion/ChatModelMetadata.cs
Outdated
Show resolved
Hide resolved
shyamnamboodiripad
approved these changes
Mar 19, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Approving for the change under the Eval project 👍🏾
peterwald
approved these changes
Mar 19, 2025
stephentoub
reviewed
Mar 20, 2025
src/Libraries/Microsoft.Extensions.AI.Abstractions/ChatCompletion/ChatModelMetadata.cs
Outdated
Show resolved
Hide resolved
src/Libraries/Microsoft.Extensions.AI.Abstractions/Embeddings/EmbeddingGeneratorMetadata.cs
Outdated
Show resolved
Hide resolved
src/Libraries/Microsoft.Extensions.AI.OpenAI/OpenAIChatClientMetadata.cs
Outdated
Show resolved
Hide resolved
src/Libraries/Microsoft.Extensions.AI.OpenAI/OpenAIChatClientMetadata.cs
Outdated
Show resolved
Hide resolved
src/Libraries/Microsoft.Extensions.AI/ChatCompletion/ChatClientStructuredOutputExtensions.cs
Outdated
Show resolved
Hide resolved
stephentoub
reviewed
Mar 20, 2025
src/Libraries/Microsoft.Extensions.AI.Ollama/OllamaChatClient.cs
Outdated
Show resolved
Hide resolved
stephentoub
reviewed
Mar 20, 2025
src/Libraries/Microsoft.Extensions.AI.OpenAI/OpenAIChatClientMetadata.cs
Outdated
Show resolved
Hide resolved
stephentoub
reviewed
Mar 20, 2025
src/Libraries/Microsoft.Extensions.AI.OpenAI/OpenAIEmbeddingGenerator.cs
Outdated
Show resolved
Hide resolved
stephentoub
reviewed
Mar 20, 2025
src/Libraries/Microsoft.Extensions.AI.OpenAI/OpenAIEmbeddingGenerator.cs
Outdated
Show resolved
Hide resolved
src/Libraries/Microsoft.Extensions.AI/ChatCompletion/ChatClientStructuredOutputExtensions.cs
Show resolved
Hide resolved
test/Libraries/Microsoft.Extensions.AI.AzureAIInference.Tests/IntegrationTestHelpers.cs
Show resolved
Hide resolved
test/Libraries/Microsoft.Extensions.AI.Integration.Tests/ChatClientIntegrationTests.cs
Show resolved
Hide resolved
…ema is on by default
a3edd1d to
3832a27
Compare
stephentoub
reviewed
Mar 21, 2025
src/Libraries/Microsoft.Extensions.AI/ChatCompletion/ChatClientStructuredOutputExtensions.cs
Outdated
Show resolved
Hide resolved
stephentoub
approved these changes
Mar 21, 2025
…tStructuredOutputExtensions.cs
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Update Following discussion, this change has been reduced to:
useJsonSchemato trueModelId/DimensionstoDefaultModelId/DefaultModelDimensionsPlus other minor related renames and integration test fixes.
Old description
Fixes #6126 based on the design discussed there.
This introduces a new concept of per-model metadata. Usage:
There's also the equivalent for embedding generators, and that's now how you can find the
Dimensionsvalue.Structured output
GetResponse<T>still allows the developer to specifyuseNativeJsonSchema, but if it's unspecified, the default is now taken from the model metadata.Providers
This PR bakes in knowledge about which model IDs currently support structured output. I'm on the fence about how much we really want to get into this business, but there's no way to get the info programmatically. Neither OpenAI or Azure AI Inference expose this metadata on the endpoints they have for describing models.
I've tried to make the logic as forgiving as possible if the external reality changes and new models appear, but having correct metadata here is a best-effort thing that developers would still have to override on a per-call basis if our default doesn't match their requirement.
The main alternative would be to remove the built-in knowledge of specific models and just default to "unknown" for the OpenAI/AIInference providers across all models. In effect that would retain the same behavior we have before this PR. However it doesn't move us forwards and cements an old-fashioned view of the world.
Yet another alternative would be defaulting to "true" for all models, since most OpenAI models do support it. However even some new-ish OpenAI models (e.g., o1-mini) don't support it, and of course many Azure AI Inference ones don't since it includes lots of small models.
Microsoft Reviewers: Open in CodeFlow