Move towards Log API/SDK stabilization

[alanwest]

OpenTelemetry's vision for supporting logs is clear:

We embrace existing logging solutions and make sure OpenTelemetry works nicely with existing logging libraries, log collection and processing solutions.

A number of prototypes exist demonstrating the Log API's utility in implementing log appenders for existing logging libraries.

• Java Log4j2 appender
• Java Logback appender
• .NET Serilog appender
• Possibly others in Python or Javascript maybe? (share links if anyone knows)

We have also received end user feedback suggesting a demand for a stable release of these log appenders.

I believe the API and SDK components as specified are nearly sufficient and we should consider making them stable. The LoggerProvider and Logger APIs are necessary for developing log appenders. The SDK components (i.e., log processors, exporters, etc) nicely mirrors the trace specification.

Below I've attempted to capture a list of remaining issues. If folks have additional things to add, please share!

Remaining issues

• Java: Log API and SDK prototype implementation
• Java: Log4J appender implementation. An example using Java agent is here.
• .Net: Log API and SDK prototype implementation
• .Net: Serilog appender implementation. .Net maintainers to link to example also showing trace context injection
• Python: Log API and SDK prototype implementation
• Python: logging handler implementation and usage example.
• Split logging functionality into API and SDK like other signals opentelemetry-python#3134
• Split out Event API from Log API #2941
• Define LogRecord batching processor defaults #3002
• Reorganize specification related to OTLP file exporter #3031
• Make context available in log record processor onEmit #2927
• Define data ownership for LogRecordProcessor #2969
• Add log attribute limit configuration #2861
• Settle on a name (Logs API vs SPI vs Bride API, etc)
• Temporarily remove "Events" from logs README #3247
• Present to Specification SIG the plans to stabilize logs API #3022
• Present in Maintainers meeting the plans to stabilize logs API #3115
• Proofread Logs Bridge API and SDK in preparation to mark it Stable #3268
• Make sure the current Logs Bridge API/SDK design doesn’t prevent future addition of end-user facing log API #3301
• Should "Emit a LogRecord" API set Timestamp if unspecified? #3386
• Drop log API include_trace_context parameter #3397
• Clarify how ObservedTimestamp field is set if unspecified #3385
• Consider removing include_trace_context from "Get a Logger" API #3394
• Clarify "Emit a LogRecord" API #3383
• Mark Logs API and SDK stable

[tigrannajaryan]

I support this. Let's discuss in the SIG meeting today.

The event_domain parameter of the Get a Logger API should remain experimental.

I think this means the entire Get a Logger needs to be marked experimental for now. At least in some languages adding/removing a parameter is a breaking change.

SDK spec

I think there is less pressure for stabilizing the SDK spec. Do need to decide on for example batching processor defaults right now? Seems like a low priority decision.

I would rather spend our efforts on figuring out the Events API first since this is where most of the unknowns are.

[tigrannajaryan]

One thing we need to decide now: in a hypothetical scenario where we eventually come to a decision that we don't need an "Events API" at all would the current structure of LoggerProvider/Logger API work fine on its own even if we delete everything related to "Events" from the API?

We may decide to support "Events" using some other form of helpers instead of making it a sibling API of Logger so we need to be prepared for such deletion.

[alanwest]

I think this means the entire Get a Logger needs to be marked experimental for now. At least in some languages adding/removing a parameter is a breaking change.

Marking Get a Logger stable is core to the premise of decoupling logs from events. I think this topic warrants a separate issue/PR, but my initial thought is the event_domain parameter just needs to be removed from the Get a Logger API. That would effectively decouple the Get a Logger API from anything event related. A new experimental API would need to be proposed in support of events.

would the current structure of LoggerProvider/Logger API work fine on its own even if we delete everything related to "Events" from the API?

Yes, this is my hope.

I think there is less pressure for stabilizing the SDK spec. Do need to decide on for example batching processor defaults right now? Seems like a low priority decision.

Ideally we move towards stabilizing the SDK to the extent possible as well. A stable SDK spec allows for a stable release of OTLP log exporters.

[jack-berg]

I've opened #2955 to discuss issue 5 in this list:

Consider the mutability of the ReadWriteLogRecord interface: Is everything mutable or only some things?