Skip to content

[bug] Exporting spans to collector via OTLP gRPC endpoint with status descriptions that are 128 chars or longer causes exception and dropping of spans #6141

@mjatkin

Description

@mjatkin

Package

OpenTelemetry.Exporter.OpenTelemetryProtocol

Package Version

Package Name Version
OpenTelemetry.Api 1.11.1
OpenTelemetry 1.11.1
OpenTelemetry.Api.ProviderBuilderExtensions 1.11.1
OpenTelemetry.Exporter.OpenTelemetryProtocol 1.11.1

Runtime Version

net8.0

Description

When exporting spans to the Open Telemetry Collector (v0.119.0) via the OTLP gRPC endpoint, if the status description of a given Activity is set to a string with a length of 128 or larger, the export fails.

The client diagnostics log file produces entries similar to below:

2025-02-18T00:08:14.9648356Z:Export failed for {0}. Message: {1}{http://otel-collector:4317/opentelemetry.proto.collector.trace.v1.TraceService/Export}{Export failed due to unexpected status code.}

The Collector internal spans for traces give the following error:

grpc: error unmarshalling request: unexpected EOF

This issue was not present in v1.10.0 of the OpenTelemetry.Exporter.OpenTelemetryProtocol package.

Steps to Reproduce

The below code will produce this issue if configured to send to a Collector OTLP gRPC endpoint. I've also linked an entire solution that will spin up a collector and configure everything automatically to highlight the issue.

  1. Open example.sln in Visual Studio
  2. Run the Docker Compose launch option

OR

  1. Open a terminal in the root dir
  2. Run docker compose up

otel-dotnet-export-issue.zip

using System.Diagnostics;
using OpenTelemetry;
using OpenTelemetry.Trace;

namespace Example
{
    class Program
    {
        private static readonly ActivitySource Source = new("Example");

        static async Task Main(string[] args)
        {
            using var tracerProvider = Sdk.CreateTracerProviderBuilder()
                .AddSource("Example")
                .AddOtlpExporter()
                .Build();


            while (true)
            {
                // Set the length of the string to set as status description
                string testString = new string('A', 128);

                using var activity = Source.StartActivity("START")
                    ?.SetStatus(ActivityStatusCode.Error, testString);

                Console.WriteLine($"Running at: {DateTimeOffset.Now}");

                await Task.Delay(1000);

            }
        }
    }
}

Expected Result

I expect the spans to export successfully and be processed by the collector. Or at the very least, a clear error that this is occurring if there is some sort of limit on the length of this status description.

Actual Result

The spans are almost silently dropped. Unless you have the collector internal spans on or are looking at the otel client diagnostic log, you will just see missing spans in the destination.

Additional Context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions