2554 Enhance REST OTEL instrumentation with custom metrics and traces #2617

tommasodotNET · 2025-03-14T18:01:24Z

Why make this change?

Closes 🥕⭐ [Enhancement]: Custom DAB OTEL Traces & Counters #2554
- Enhances OTEL instrumentation with custom traces and metrics for the REST APIs

What is this change?

This PR enhances the OTEL instrumentation for the REST APIs by adding custom traces and metrics.

I have removed ASP NET Core standard instrumentation since it does not provide great value given the custom nature of the webservice. I have written two main Helper classes: TelemetryMetricsHelper and TelemetryTracesHelper to provide a single point of management for custom traces and metrics.

Metrics can be filtered for status_code, api_type, endpoint and method.

I have also fixed the loggings which are now sent to the configured OTEL endpoint.

Logs

Metrics

Traces

How was this tested?

Integration Tests
Unit Tests

Sample Request(s)

To test everything locally I recommend using this repo that allows to run the local build of the dab cli and send metrics to the .NET Aspire OTEL endoint.

JerryNixon · 2025-03-14T19:10:35Z

@Aniruddh25 is March realistic?

src/Service/Controllers/RestController.cs

Co-authored-by: Aaron Powell <[email protected]>

tommasodotNET · 2025-03-19T10:37:18Z

/azp run

azure-pipelines · 2025-03-19T10:37:24Z

Commenter does not have sufficient privileges for PR 2617 in repo Azure/data-api-builder

sezal98 · 2025-03-27T09:40:51Z

/azp run

azure-pipelines · 2025-03-27T09:41:15Z

Azure Pipelines successfully started running 6 pipeline(s).

src/Service/Controllers/RestController.cs

…a-api-builder into features/2554-enh-otel

sezal98 · 2025-04-01T16:10:01Z

/azp run

azure-pipelines · 2025-04-01T16:10:25Z

Azure Pipelines successfully started running 6 pipeline(s).

src/Service/Controllers/RestController.cs

src/Service/Telemetry/TelemetryTracesHelper.cs

src/Service/Controllers/RestController.cs

Co-authored-by: RubenCerna2079 <[email protected]>

RubenCerna2079 · 2025-04-21T19:14:24Z

/azp run

azure-pipelines · 2025-04-21T19:14:48Z

Azure Pipelines successfully started running 6 pipeline(s).

RubenCerna2079 · 2025-04-21T19:16:09Z

@tommasodotNET I made some changes to your PR, it was mostly changing methods to use more specific classes instead of string where I found it possible. I also fixed small formatting issues that were causing the unit tests to fail.

RubenCerna2079 · 2025-04-21T20:06:56Z

/azp run

azure-pipelines · 2025-04-21T20:07:20Z

Azure Pipelines successfully started running 6 pipeline(s).

RubenCerna2079 · 2025-04-21T23:29:03Z

@tommasodotNET I also found out a new issue with the tests related to a format issue in the dab-config.CosmosDb_NoSql.json, so I fixed it as it seems that was the only test failing.

RubenCerna2079 · 2025-04-21T23:29:09Z

/azp run

azure-pipelines · 2025-04-21T23:29:33Z

Azure Pipelines successfully started running 6 pipeline(s).

tommasodotNET · 2025-04-22T18:27:01Z

Thanks @RubenCerna2079 for your support

src/Service/Controllers/RestController.cs

Aniruddh25 · 2025-04-22T23:00:06Z

/azp run

azure-pipelines · 2025-04-22T23:00:35Z

Azure Pipelines successfully started running 6 pipeline(s).

Aniruddh25

LGTM, given all tests are passing. This has come a long way!

ship it!!

RubenCerna2079 · 2025-04-23T02:14:12Z

/azp run

azure-pipelines · 2025-04-23T02:14:34Z

Azure Pipelines successfully started running 6 pipeline(s).

…#2617) ## Why make this change? - Closes #2554 - Enhances OTEL instrumentation with custom traces and metrics for the REST APIs ## What is this change? This PR enhances the OTEL instrumentation for the REST APIs by adding custom traces and metrics. I have removed ASP NET Core standard instrumentation since it does not provide great value given the custom nature of the webservice. I have written two main Helper classes: `TelemetryMetricsHelper` and `TelemetryTracesHelper` to provide a single point of management for custom traces and metrics. Metrics can be filtered for `status_code`, `api_type`, `endpoint` and `method`. I have also fixed the loggings which are now sent to the configured OTEL endpoint. ### Logs ![image](https://github.com/user-attachments/assets/2b2b21ab-b16e-4678-8a3f-2c6bc3ab7168) ### Metrics ![Screenshot 2025-03-14 190730](https://github.com/user-attachments/assets/322f6111-8580-49c2-b6ff-5cddc191178c) ### Traces ![image (2)](https://github.com/user-attachments/assets/1e9f320a-7445-44d6-a1c6-b0d5eb21fa8f) ![image (3)](https://github.com/user-attachments/assets/efc754c1-a646-4500-9ebf-4f5cfba34b85) ![image (4)](https://github.com/user-attachments/assets/cadc9665-934f-48bd-a1bc-afedbf5fe395) ## How was this tested? - [ ] Integration Tests - [ ] Unit Tests ## Sample Request(s) To test everything locally I recommend using [this repo](https://github.com/tommasodotNET/dab-workbench) that allows to run the local build of the dab cli and send metrics to the .NET Aspire OTEL endoint. --------- Co-authored-by: Aaron Powell <[email protected]> Co-authored-by: RubenCerna2079 <[email protected]> Co-authored-by: aaronburtle <[email protected]> Co-authored-by: Ruben Cerna <[email protected]> Co-authored-by: Aniruddh Munde <[email protected]>

## Why make this change? This change is made in order to add all of the commits for milestone 1.5 into its respective branch. ## What is this change? This change cherry-picks all of the commits that were added after the first release candidate. Cherry-picked commits: - #2648 #2657 #2617 #2659 #2655 #2633 #2667 #2673 #2650 #2695 #2702 #2688 ## How was this tested? - [ ] Integration Tests - [ ] Unit Tests ## Sample Request(s) --------- Co-authored-by: Sezal Chug <[email protected]> Co-authored-by: sezalchug <[email protected]> Co-authored-by: Tommaso Stocchi <[email protected]> Co-authored-by: Aaron Powell <[email protected]> Co-authored-by: aaronburtle <[email protected]> Co-authored-by: Aniruddh Munde <[email protected]> Co-authored-by: Jerry Nixon <[email protected]> Co-authored-by: Copilot <[email protected]> Co-authored-by: Michael Staib <[email protected]> Co-authored-by: souvikghosh04 <[email protected]> Co-authored-by: Souvik Ghosh <[email protected]>

…#2617) - Closes #2554 - Enhances OTEL instrumentation with custom traces and metrics for the REST APIs This PR enhances the OTEL instrumentation for the REST APIs by adding custom traces and metrics. I have removed ASP NET Core standard instrumentation since it does not provide great value given the custom nature of the webservice. I have written two main Helper classes: `TelemetryMetricsHelper` and `TelemetryTracesHelper` to provide a single point of management for custom traces and metrics. Metrics can be filtered for `status_code`, `api_type`, `endpoint` and `method`. I have also fixed the loggings which are now sent to the configured OTEL endpoint. ![image](https://github.com/user-attachments/assets/2b2b21ab-b16e-4678-8a3f-2c6bc3ab7168) ![Screenshot 2025-03-14 190730](https://github.com/user-attachments/assets/322f6111-8580-49c2-b6ff-5cddc191178c) ![image (2)](https://github.com/user-attachments/assets/1e9f320a-7445-44d6-a1c6-b0d5eb21fa8f) ![image (3)](https://github.com/user-attachments/assets/efc754c1-a646-4500-9ebf-4f5cfba34b85) ![image (4)](https://github.com/user-attachments/assets/cadc9665-934f-48bd-a1bc-afedbf5fe395) - [ ] Integration Tests - [ ] Unit Tests To test everything locally I recommend using [this repo](https://github.com/tommasodotNET/dab-workbench) that allows to run the local build of the dab cli and send metrics to the .NET Aspire OTEL endoint. --------- Co-authored-by: Aaron Powell <[email protected]> Co-authored-by: RubenCerna2079 <[email protected]> Co-authored-by: aaronburtle <[email protected]> Co-authored-by: Ruben Cerna <[email protected]> Co-authored-by: Aniruddh Munde <[email protected]>

tommasodotNET added 4 commits February 27, 2025 19:39

initialize custom metrics/traces for REST endpoint

343e62c

initialize custom traces on sql db

f807727

traces refactor

ef47e17

fixes otel logging

6b2d1f7

tommasodotNET requested review from Aniruddh25, RubenCerna2079, aaronburtle, neeraj-sharma2592, ravishetye, rohkhann, sezal98 and sourabh1007 as code owners March 14, 2025 18:01

tommasodotNET added 2 commits March 14, 2025 19:11

fixes otel logging resource name

970360c

cleans up pr

2199d68

aaronpowell reviewed Mar 17, 2025

View reviewed changes

src/Service/Controllers/RestController.cs Outdated Show resolved Hide resolved

src/Service/Controllers/RestController.cs Outdated Show resolved Hide resolved

tommasodotNET and others added 2 commits March 17, 2025 19:33

Update src/Service/Controllers/RestController.cs

0176cfd

Co-authored-by: Aaron Powell <[email protected]>

Update src/Service/Controllers/RestController.cs

1393def

Co-authored-by: Aaron Powell <[email protected]>

sezal98 reviewed Mar 31, 2025

View reviewed changes

src/Service/Controllers/RestController.cs Outdated Show resolved Hide resolved

src/Service/Controllers/RestController.cs Outdated Show resolved Hide resolved

tommasodotNET added 2 commits March 31, 2025 16:44

adding null check on route

5db6e92

Merge branch 'features/2554-enh-otel' of github.com:tommasodotNET/dat…

5a4848b

…a-api-builder into features/2554-enh-otel

RubenCerna2079 mentioned this pull request Apr 2, 2025

OTEL Implementation for GraphQL #2642

Closed

RubenCerna2079 reviewed Apr 2, 2025

View reviewed changes

Update src/Service/Controllers/RestController.cs

fc44a21

Co-authored-by: RubenCerna2079 <[email protected]>

Changes to Tomaso Branch

0a21b84

RubenCerna2079 and others added 2 commits April 21, 2025 13:05

Fix Unit Test errors

3d09aca

Merge branch 'main' into features/2554-enh-otel

d674495

Fixed Unit Test Failure

de36de5

Aniruddh25 reviewed Apr 22, 2025

View reviewed changes

src/Service/Controllers/RestController.cs Outdated Show resolved Hide resolved

Merge branch 'main' into features/2554-enh-otel

be44074

Aniruddh25 approved these changes Apr 22, 2025

View reviewed changes

Changes based on comments

8780c39

Aniruddh25 approved these changes Apr 23, 2025

View reviewed changes

RubenCerna2079 approved these changes Apr 23, 2025

View reviewed changes

RubenCerna2079 merged commit c9e732e into Azure:main Apr 23, 2025
11 checks passed

Aniruddh25 added this to the 1.5 milestone May 11, 2025

RubenCerna2079 mentioned this pull request May 29, 2025

Cherry-Picking Commits for 1.5 #2689

Merged

2 tasks

2554 Enhance REST OTEL instrumentation with custom metrics and traces #2617

2554 Enhance REST OTEL instrumentation with custom metrics and traces #2617

Uh oh!

Conversation

tommasodotNET commented Mar 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why make this change?

What is this change?

Logs

Metrics

Traces

How was this tested?

Sample Request(s)

Uh oh!

JerryNixon commented Mar 14, 2025

Uh oh!

Uh oh!

Uh oh!

tommasodotNET commented Mar 19, 2025

Uh oh!

azure-pipelines bot commented Mar 19, 2025

Uh oh!

sezal98 commented Mar 27, 2025

Uh oh!

azure-pipelines bot commented Mar 27, 2025

Uh oh!

Uh oh!

Uh oh!

sezal98 commented Apr 1, 2025

Uh oh!

azure-pipelines bot commented Apr 1, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

RubenCerna2079 commented Apr 21, 2025

Uh oh!

azure-pipelines bot commented Apr 21, 2025

Uh oh!

RubenCerna2079 commented Apr 21, 2025

Uh oh!

RubenCerna2079 commented Apr 21, 2025

Uh oh!

azure-pipelines bot commented Apr 21, 2025

Uh oh!

RubenCerna2079 commented Apr 21, 2025

Uh oh!

RubenCerna2079 commented Apr 21, 2025

Uh oh!

azure-pipelines bot commented Apr 21, 2025

Uh oh!

tommasodotNET commented Apr 22, 2025

Uh oh!

Uh oh!

Aniruddh25 commented Apr 22, 2025

Uh oh!

azure-pipelines bot commented Apr 22, 2025

Uh oh!

Aniruddh25 left a comment

Choose a reason for hiding this comment

Uh oh!

RubenCerna2079 commented Apr 23, 2025

Uh oh!

azure-pipelines bot commented Apr 23, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

tommasodotNET commented Mar 14, 2025 •

edited

Loading