Skip to content

MEAI: Not possible to set strict mode for OpenAI chat clients. #6535

@NiceWaffel

Description

@NiceWaffel

Description

With pull request #6285 the default value for jsonSchemaIsStrict was set to null (left at default). This reverted the changes from #6064. @SteveSandersonMS and @eiriktsarpalis had a discussion on this and decided to remove the flag altogether.

While I do agree that turning on strict should not be the default, developers should have the option to set it for the ChatCompletionOptions, using an additional property on ChatOptions or ChatResponseFormat.

I think my comment on #6266 was misunderstood, as I don't see an option to turn strict mode on for structured outputs, only for tools. I also don't see an option how "Developers can still control this manually", as Steve stated in the abovementioned discussion. Please correct me if this is somehow user error.

Reproduction Steps

Expected behavior

The following chat options should set the strict flag in the request to OpenAI.

var options = new ChatOptions()
{
  AdditionalProperties = new AdditionalPropertiesDictionary
  {
    { "Strict", true }
  }
}

Actual behavior

The flag is left at the default value in the API call, with no obvious way to override this.

Regression?

No response

Known Workarounds

None, other than re-implementing the OpenAIChatClient.

Configuration

No response

Other information

This code in ToOpenAIOptions(ChatOptions? options) in OpenAIChatClient.cs:

[... 477 lines of other code ...]
else if (options.ResponseFormat is ChatResponseFormatJson jsonFormat)
{
    result.ResponseFormat = jsonFormat.Schema is string jsonSchema ?
        OpenAI.Chat.ChatResponseFormat.CreateJsonSchemaFormat(jsonFormat.SchemaName ?? "json_schema", BinaryData.FromString(jsonSchema), jsonFormat.SchemaDescription) :
        OpenAI.Chat.ChatResponseFormat.CreateJsonObjectFormat();
}
[...]

should be changed to (something like):

[...]
else if (options.ResponseFormat is ChatResponseFormatJson jsonFormat)
{
    bool? strict = null;
    if (additionalProperties.TryGetConvertedValue("Strict", out bool? strictSetByDeveloper))
    {
        strict = strictSetByDeveloper;
    }

    result.ResponseFormat = jsonFormat.Schema is string jsonSchema ?
        OpenAI.Chat.ChatResponseFormat.CreateJsonSchemaFormat(jsonFormat.SchemaName ?? "json_schema", BinaryData.FromString(jsonSchema), jsonFormat.SchemaDescription, jsonSchemaIsStrict: strict) :
        OpenAI.Chat.ChatResponseFormat.CreateJsonObjectFormat();
}
[...]

Metadata

Metadata

Assignees

Labels

area-aiMicrosoft.Extensions.AI libraries

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions