sdk/log: self-observability: log created metric #7121

mahendrabishnoi2 · 2025-08-03T14:45:47Z

Adds experimental otel.sdk.log.created metric to logger. Since this is experimental, metric is behind the OTEL_GO_X_SELF_OBSERVABILITY feature gate.

Metric defined in https://github.com/open-telemetry/semantic-conventions/blob/v1.36.0/docs/otel/sdk-metrics.md.

Given the feature is experimental, it always uses the global meterprovider when enabled.

sdk/log/internal/x/x.go

sdk/log/logger.go

pellared

Overall LGTM

sdk/log/internal/x/README.md

sdk/log/logger_test.go

Copilot

Pull Request Overview

This PR adds experimental self-observability metrics to the OpenTelemetry Go SDK's log package, specifically implementing the otel.sdk.log.created metric that tracks the number of log records created. The feature is controlled by the OTEL_GO_X_SELF_OBSERVABILITY environment variable and uses the global meter provider when enabled.

Implements the otel.sdk.log.created metric as defined in semantic conventions v1.36.0
Adds experimental feature flag system for controlling self-observability features
Updates dependencies to include metric SDK packages

Reviewed Changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
sdk/log/logger.go	Adds self-observability initialization and metric recording to the logger
sdk/log/logger_test.go	Adds comprehensive tests for the self-observability feature
sdk/log/internal/x/x.go	Implements the experimental feature flag system
sdk/log/internal/x/x_test.go	Tests for the feature flag system
sdk/log/internal/x/README.md	Documentation for experimental features
sdk/log/go.mod	Updates module dependencies
sdk/log/doc.go	Adds reference to experimental features
CHANGELOG.md	Documents the new feature

sdk/log/logger.go

codecov · 2025-08-04T15:32:39Z

Codecov Report

❌ Patch coverage is 93.33333% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.9%. Comparing base (97c22e3) to head (d9bde60).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
sdk/log/logger.go	85.0%	2 Missing and 1 partial ⚠️

Additional details and impacted files

@@          Coverage Diff          @@
##            main   #7121   +/-   ##
=====================================
  Coverage   82.9%   82.9%           
=====================================
  Files        262     263    +1     
  Lines      24461   24503   +42     
=====================================
+ Hits       20291   20330   +39     
- Misses      3794    3796    +2     
- Partials     376     377    +1

Files with missing lines	Coverage Δ
sdk/log/internal/x/x.go	`100.0% <100.0%> (ø)`
sdk/log/logger.go	`96.2% <85.0%> (-3.8%)`	⬇️

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

mahendrabishnoi2 · 2025-08-04T15:34:11Z

@pellared This is ready for re-review.

One question: As per the processes of otel, who is supposed to mark comments as resolved? Reviewer or Author?

…n this in PR review if it can be moved to sdk/intenal

…w log.Record is created before it is passed to log processors defined in LogProvider - Keep all of above behind SelfObservability feature flag

…ounter is nil when feature flag is off (also a bug - good tests) - go.mod updated by make precommit

pellared · 2025-08-04T15:37:35Z

who is supposed to mark comments as resolved? Reviewer or Author?

We have no strong rules around that 😉

pellared · 2025-08-04T15:38:24Z

From https://github.com/open-telemetry/opentelemetry-go/blob/main/CONTRIBUTING.md#how-to-send-pull-requests

Avoid rebasing and force-pushing to your branch to facilitate reviewing the pull request. Rewriting Git history makes it difficult to keep track of iterations during code review. All pull requests are squashed to a single commit upon merge to main.

sdk/log/logger_test.go

mahendrabishnoi2 · 2025-08-04T15:40:01Z

From https://github.com/open-telemetry/opentelemetry-go/blob/main/CONTRIBUTING.md#how-to-send-pull-requests

Avoid rebasing and force-pushing to your branch to facilitate reviewing the pull request. Rewriting Git history makes it difficult to keep track of iterations during code review. All pull requests are squashed to a single commit upon merge to main.

Will keep this in mind. Thanks

pellared

Only one nit comment needs to be addressed (https://github.com/open-telemetry/opentelemetry-go/pull/7121/files#r2251864158)

…as it's always the same

sdk/log/logger.go

flc1125

Overall, there are no issues.

sdk/log/internal/x/README.md

pellared · 2025-08-05T14:49:22Z

@open-telemetry/go-approvers, I am planning to merge this tomorrow.

pellared · 2025-08-07T08:14:51Z

@mahendrabishnoi2, thank you for your solid contribution! 🏅

This release is the last to support [Go 1.23]. The next release will require at least [Go 1.24]. ### Added - Add native histogram exemplar support in `go.opentelemetry.io/otel/exporters/prometheus`. (#6772) - Add template attribute functions to the `go.opentelmetry.io/otel/semconv/v1.34.0` package. (#6939) - `ContainerLabel` - `DBOperationParameter` - `DBSystemParameter` - `HTTPRequestHeader` - `HTTPResponseHeader` - `K8SCronJobAnnotation` - `K8SCronJobLabel` - `K8SDaemonSetAnnotation` - `K8SDaemonSetLabel` - `K8SDeploymentAnnotation` - `K8SDeploymentLabel` - `K8SJobAnnotation` - `K8SJobLabel` - `K8SNamespaceAnnotation` - `K8SNamespaceLabel` - `K8SNodeAnnotation` - `K8SNodeLabel` - `K8SPodAnnotation` - `K8SPodLabel` - `K8SReplicaSetAnnotation` - `K8SReplicaSetLabel` - `K8SStatefulSetAnnotation` - `K8SStatefulSetLabel` - `ProcessEnvironmentVariable` - `RPCConnectRPCRequestMetadata` - `RPCConnectRPCResponseMetadata` - `RPCGRPCRequestMetadata` - `RPCGRPCResponseMetadata` - Add `ErrorType` attribute helper function to the `go.opentelmetry.io/otel/semconv/v1.34.0` package. (#6962) - Add `WithAllowKeyDuplication` in `go.opentelemetry.io/otel/sdk/log` which can be used to disable deduplication for log records. (#6968) - Add `WithCardinalityLimit` option to configure the cardinality limit in `go.opentelemetry.io/otel/sdk/metric`. (#6996, #7065, #7081, #7164, #7165, #7179) - Add `Clone` method to `Record` in `go.opentelemetry.io/otel/log` that returns a copy of the record with no shared state. (#7001) - Add experimental self-observability span and batch span processor metrics in `go.opentelemetry.io/otel/sdk/trace`. Check the `go.opentelemetry.io/otel/sdk/trace/internal/x` package documentation for more information. (#7027, #6393, #7209) - The `go.opentelemetry.io/otel/semconv/v1.36.0` package. The package contains semantic conventions from the `v1.36.0` version of the OpenTelemetry Semantic Conventions. See the [migration documentation](./semconv/v1.36.0/MIGRATION.md) for information on how to upgrade from `go.opentelemetry.io/otel/semconv/v1.34.0.`(#7032, #7041) - Add support for configuring Prometheus name translation using `WithTranslationStrategy` option in `go.opentelemetry.io/otel/exporters/prometheus`. The current default translation strategy when UTF-8 mode is enabled is `NoUTF8EscapingWithSuffixes`, but a future release will change the default strategy to `UnderscoreEscapingWithSuffixes` for compliance with the specification. (#7111) - Add experimental self-observability log metrics in `go.opentelemetry.io/otel/sdk/log`. Check the `go.opentelemetry.io/otel/sdk/log/internal/x` package documentation for more information. (#7121) - Add experimental self-observability trace exporter metrics in `go.opentelemetry.io/otel/exporters/stdout/stdouttrace`. Check the `go.opentelemetry.io/otel/exporters/stdout/stdouttrace/internal/x` package documentation for more information. (#7133) - Support testing of [Go 1.25]. (#7187) - The `go.opentelemetry.io/otel/semconv/v1.37.0` package. The package contains semantic conventions from the `v1.37.0` version of the OpenTelemetry Semantic Conventions. See the [migration documentation](./semconv/v1.37.0/MIGRATION.md) for information on how to upgrade from `go.opentelemetry.io/otel/semconv/v1.36.0.`(#7254) ### Changed - Optimize `TraceIDFromHex` and `SpanIDFromHex` in `go.opentelemetry.io/otel/sdk/trace`. (#6791) - Change `AssertEqual` in `go.opentelemetry.io/otel/log/logtest` to accept `TestingT` in order to support benchmarks and fuzz tests. (#6908) - Change `DefaultExemplarReservoirProviderSelector` in `go.opentelemetry.io/otel/sdk/metric` to use `runtime.GOMAXPROCS(0)` instead of `runtime.NumCPU()` for the `FixedSizeReservoirProvider` default size. (#7094) ### Fixed - `SetBody` method of `Record` in `go.opentelemetry.io/otel/sdk/log` now deduplicates key-value collections (`log.Value` of `log.KindMap` from `go.opentelemetry.io/otel/log`). (#7002) - Fix `go.opentelemetry.io/otel/exporters/prometheus` to not append a suffix if it's already present in metric name. (#7088) - Fix the `go.opentelemetry.io/otel/exporters/stdout/stdouttrace` self-observability component type and name. (#7195) - Fix partial export count metric in `go.opentelemetry.io/otel/exporters/stdout/stdouttrace`. (#7199) ### Deprecated - Deprecate `WithoutUnits` and `WithoutCounterSuffixes` options, preferring `WithTranslationStrategy` instead. (#7111) - Deprecate support for `OTEL_GO_X_CARDINALITY_LIMIT` environment variable in `go.opentelemetry.io/otel/sdk/metric`. Use `WithCardinalityLimit` option instead. (#7166)

mahendrabishnoi2 requested review from MrAlias, XSAM, dashpole, dmathieu, flc1125 and pellared as code owners August 3, 2025 14:45

mahendrabishnoi2 commented Aug 3, 2025

View reviewed changes

sdk/log/internal/x/x.go Show resolved Hide resolved

sdk/log/logger.go Show resolved Hide resolved

sdk/log/logger.go Show resolved Hide resolved

mahendrabishnoi2 changed the title ~~sdk/log: self-observability: log created metric~~ [WIP] sdk/log: self-observability: log created metric Aug 3, 2025

mahendrabishnoi2 changed the title ~~[WIP] sdk/log: self-observability: log created metric~~ sdk/log: self-observability: log created metric Aug 3, 2025

mahendrabishnoi2 mentioned this pull request Aug 4, 2025

[WIP] sdk/log: self observability: batch log processor metrics #7124

Draft

46 tasks

pellared reviewed Aug 4, 2025

View reviewed changes

pellared mentioned this pull request Aug 4, 2025

feat: add self-observability metrics to otlpmetricgrpc metric exporters #7120

Open

pellared requested a review from Copilot August 4, 2025 15:31

Copilot AI reviewed Aug 4, 2025

View reviewed changes

sdk/log/logger.go Show resolved Hide resolved

mahendrabishnoi2 added 8 commits August 4, 2025 21:05

copy feature flag code from sdk/trace/internal -> will discuss more o…

0d400cc

…n this in PR review if it can be moved to sdk/intenal

add doc comment about experimental metric (self observability)

75c8a0c

- add support for otel.sdk.log.created metric, increment it when ne…

c2c73b6

…w log.Record is created before it is passed to log processors defined in LogProvider - Keep all of above behind SelfObservability feature flag

- Fix the issue reported by a failed test case - SDKLogCreated.Int64C…

f25ec73

…ounter is nil when feature flag is off (also a bug - good tests) - go.mod updated by make precommit

test cases for SDKLogCreated metric

7e4644b

go mod tidy

379105c

update CHANGELOG.md

60a3a7f

Addressed review comments

e13e4c0

mahendrabishnoi2 force-pushed the logs-created-metrics branch from dee55c8 to e13e4c0 Compare August 4, 2025 15:35

Addressed review comments

f30fea7

pellared reviewed Aug 4, 2025

View reviewed changes

sdk/log/logger_test.go Outdated Show resolved Hide resolved

pellared approved these changes Aug 4, 2025

View reviewed changes

mahendrabishnoi2 and others added 2 commits August 4, 2025 21:11

Addressed review comments - removed context from the test case setup …

ba25fe6

…as it's always the same

Merge branch 'main' into logs-created-metrics

440f8a0

flc1125 reviewed Aug 5, 2025

View reviewed changes

sdk/log/logger.go Show resolved Hide resolved

flc1125 reviewed Aug 5, 2025

View reviewed changes

sdk/log/internal/x/README.md Outdated Show resolved Hide resolved

flc1125 approved these changes Aug 5, 2025

View reviewed changes

addressed review comment

1f201e3

Merge branch 'main' into logs-created-metrics

d9bde60

pellared merged commit ffa3b4a into open-telemetry:main Aug 7, 2025
33 checks passed

mahendrabishnoi2 deleted the logs-created-metrics branch August 9, 2025 12:58

MrAlias added this to the v1.38.0 milestone Aug 20, 2025

mahendrabishnoi2 mentioned this pull request Aug 28, 2025

REQUEST: New membership for mahendrabishnoi2 open-telemetry/community#2955

Closed

6 tasks

MrAlias mentioned this pull request Aug 29, 2025

Release v1.38.0 #7271

Merged

sdk/log: self-observability: log created metric #7121

sdk/log: self-observability: log created metric #7121

Uh oh!

Conversation

mahendrabishnoi2 commented Aug 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pellared left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

codecov bot commented Aug 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

mahendrabishnoi2 commented Aug 4, 2025

Uh oh!

pellared commented Aug 4, 2025

Uh oh!

pellared commented Aug 4, 2025

Uh oh!

Uh oh!

mahendrabishnoi2 commented Aug 4, 2025

Uh oh!

pellared left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

flc1125 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

pellared commented Aug 5, 2025

Uh oh!

Uh oh!

pellared commented Aug 7, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

mahendrabishnoi2 commented Aug 3, 2025 •

edited

Loading

codecov bot commented Aug 4, 2025 •

edited

Loading