[BUG] InternalMCPCallItemResource.Error - type mismatch

[gophph]

Describe the bug

I'm getting following exception when trying to deserialize OpenAI response that contains error object in the output:

System.InvalidOperationException: The requested operation requires an element of type 'String', but the target element has type 'Object'.

Looks like effectively OpenAPI returns error object, while SDK expects string

Callstack:

System.InvalidOperationException: The requested operation requires an element of type 'String', but the target element has type 'Object'.
   at System.Text.Json.ThrowHelper.ThrowJsonElementWrongTypeException(JsonTokenType expectedType, JsonTokenType actualType)
   at System.Text.Json.JsonDocument.CheckExpectedType(JsonTokenType expected, JsonTokenType actual)
   at System.Text.Json.JsonDocument.GetString(Int32 index, JsonTokenType expectedType)
   at System.Text.Json.JsonElement.GetString()
   at OpenAI.Responses.InternalMCPCallItemResource.DeserializeInternalMCPCallItemResource(JsonElement element, ModelReaderWriterOptions options)
   at OpenAI.Responses.ResponseItem.DeserializeResponseItem(JsonElement element, ModelReaderWriterOptions options)
   at OpenAI.Responses.OpenAIResponse.DeserializeOpenAIResponse(JsonElement element, ModelReaderWriterOptions options)
   at OpenAI.Responses.OpenAIResponse.PersistableModelCreateCore(BinaryData data, ModelReaderWriterOptions options)
   at OpenAI.Responses.OpenAIResponse.System.ClientModel.Primitives.IPersistableModel<OpenAI.Responses.OpenAIResponse>.Create(BinaryData data, ModelReaderWriterOptions options)
   at System.ClientModel.Primitives.ModelReaderWriter.ReadInternal(BinaryData data, Type returnType, ModelReaderWriterOptions options, ModelReaderWriterContext context)
   at System.ClientModel.Primitives.ModelReaderWriter.ReadInternal[T](BinaryData data, ModelReaderWriterOptions options, ModelReaderWriterContext context)
   at System.ClientModel.Primitives.ModelReaderWriter.Read[T](BinaryData data, ModelReaderWriterOptions options)
...

Steps to reproduce

1. Use Response API to make call with MCP tool configured
2. Provide erroneous or incomplete prompt that causes MCP tool to fail
3. Observe following object returned in Output array:

{
    "id" : "mcp_689762d15184819080bf25e0c01a09300ce824501528abbb",
    "type" : "mcp_call",
    "approval_request_id" : null,
    "arguments" : <redacted>,
    "error" : {
      "type" : "mcp_tool_execution_error",
      "content" : [ {
        "type" : "text",
        "text" : "An error occurred invoking 'create_custom_asset'.",
        "annotations" : null,
        "meta" : null
      } ]
    },
    "name" : "create_custom_asset",
    "output" : null,
    "server_label" : "MyMCP"
  }

Code snippets

        var azureClient = new AzureOpenAIClient(
            new Uri(_configuration.OpenAiEndpointUrl),
            new AzureKeyCredential(_configuration.OpenAiApiKey));
        
        var client = azureClient.GetOpenAIResponseClient(_configuration.Model);

        var rawRequest = new
        {
            model = _configuration.Model,
            instructions =
                "<redacted>",
            previous_response_id = string.IsNullOrEmpty(previousResponseId) ? null : previousResponseId,
            input,
            tools = new[]
            {
                new
                {
                    type = "mcp",
                    server_label = "MyMCP",
                    server_url = _configuration.McpServerUrl,
                    headers = new { Authorization = bearerToken },
                    require_approval = "never"
                }
            }

        };
        var binRequest = BinaryContent.Create(BinaryData.FromObjectAsJson(rawRequest));
        var rawResponse = await client.CreateResponseAsync(binRequest);            
            
        var binResponse = rawResponse.GetRawResponse().Content;
        var outputText = var response = ModelReaderWriter.Read<OpenAIResponse>(binResponse).GetOutputText();

OS

macOs

.NET version

.Net 8

Library version

Azure.AI.OpenAI/2.2.0-beta5

[jsquire]

Hi @gophph. Thanks for teaching out and we regret that you're experiencing difficulties. Would you please be so kind as to provide an example of the call that you made with the client library?

[gophph]

@jsquire thanks for your answer. I've added code snippet to original issue description

[jsquire]

Thanks, @gophph. This looks to be a case where the OpenAI REST API spec is not accurate for the behavior of the service. The spec says that MCPToolCall has an error field that looks like:

/** The error from the tool call, if any. */
error?: string | null;

The MCP tool guide disagrees saying:

Failed tool calls will populate the error field of this item with MCP protocol errors, MCP tool execution errors, or general connectivity errors. The MCP errors are documented in the MCP spec here.

Which, in turn, doesn't strictly define the structure at all, just shows one example of what it may look like. (see: Error Handling)

{
  "jsonrpc": "2.0",
  "id": 4,
  "result": {
    "content": [
      {
        "type": "text",
        "text": "Failed to fetch weather data: API rate limit exceeded"
      }
    ],
    "isError": true
  }
}

The end result being that there is no official definition of what the error object structure is. We'll need to coordinate with the OpenAI service team on an updated spec definition to resolve this.

[jsquire]

@trrwilson: Would appreciate your thoughts.