Skip to content

ci(codecov): simplify codecov configuration #1335

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

ci(codecov): simplify codecov configuration #1335

wants to merge 2 commits into from
+228 −269

Conversation

@bantonsson
Copy link
Contributor

@bantonsson bantonsson commented Nov 11, 2025

What does this PR do?

A brief description of the change being made with this pull request.

Motivation

What inspired you to submit this pull request?

Additional Notes

Anything else we should know when reviewing?

How to test the change?

Describe here in detail how the change can be validated.

@codecov-commenter
Copy link

codecov-commenter commented Nov 11, 2025 •
edited
Loading

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 71.94%. Comparing base (35314d9) to head (971d183).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files 
@@            Coverage Diff             @@
##             main    #1335      +/-   ##
==========================================
+ Coverage   71.08%   71.94%   +0.85%     
==========================================
  Files         379      362      -17     
  Lines       59814    58769    -1045     
==========================================
- Hits        42520    42280     -240     
+ Misses      17294    16489     -805
Components Coverage Δ
datadog-crashtracker ∅ <ø> (∅)
datadog-crashtracker-ffi ∅ <ø> (∅)
datadog-alloc ∅ <ø> (∅)
data-pipeline ∅ <ø> (∅)
data-pipeline-ffi ∅ <ø> (∅)
ddcommon ∅ <ø> (∅)
ddcommon-ffi ∅ <ø> (∅)
ddtelemetry ∅ <ø> (∅)
ddtelemetry-ffi ∅ <ø> (∅)
dogstatsd-client ∅ <ø> (∅)
datadog-ipc 82.61% <ø> (+0.02%) ⬆️
datadog-profiling 0.00% <ø> (ø)
datadog-profiling-ffi ∅ <ø> (∅)
datadog-sidecar 36.37% <ø> (-1.20%) ⬇️
datdog-sidecar-ffi 9.78% <ø> (-5.86%) ⬇️
spawn-worker 55.18% <ø> (ø)
tinybytes ∅ <ø> (∅)
datadog-trace-normalization ∅ <ø> (∅)
datadog-trace-obfuscation 94.17% <ø> (ø)
datadog-trace-protobuf ∅ <ø> (∅)
datadog-trace-utils ∅ <ø> (∅)
datadog-tracer-flare 59.91% <ø> (-2.35%) ⬇️
datadog-log ∅ <ø> (∅)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@pr-commenter
Copy link

pr-commenter bot commented Nov 11, 2025 •
edited
Loading

Benchmarks

Comparison

Benchmark execution time: 2025-11-12 11:17:49

Comparing candidate commit 971d183 in PR branch ban/simplify-codecov with baseline commit 1b7b2da in branch main.

Found 11 performance improvements and 4 performance regressions! Performance is the same for 40 metrics, 2 unstable metrics.

scenario:concentrator/add_spans_to_concentrator

  • 🟥 execution_time [+510.233µs; +515.682µs] or [+4.770%; +4.821%]

scenario:credit_card/is_card_number/ 3782-8224-6310-005

  • 🟩 execution_time [-4.668µs; -4.223µs] or [-5.699%; -5.155%]
  • 🟩 throughput [+663800.575op/s; +731022.070op/s] or [+5.436%; +5.987%]

scenario:credit_card/is_card_number/x371413321323331

  • 🟩 execution_time [-393.502ns; -382.966ns] or [-6.116%; -5.953%]
  • 🟩 throughput [+9848410.050op/s; +10128750.658op/s] or [+6.336%; +6.516%]

scenario:credit_card/is_card_number_no_luhn/ 3782-8224-6310-005

  • 🟥 execution_time [+3.169µs; +3.296µs] or [+5.068%; +5.270%]
  • 🟥 throughput [-802736.821op/s; -770561.619op/s] or [-5.020%; -4.819%]

scenario:credit_card/is_card_number_no_luhn/x371413321323331

  • 🟩 execution_time [-399.910ns; -390.757ns] or [-6.211%; -6.069%]
  • 🟩 throughput [+10042313.931op/s; +10284558.541op/s] or [+6.466%; +6.622%]

scenario:normalization/normalize_service/normalize_service/[empty string]

  • 🟩 execution_time [-1.601µs; -1.584µs] or [-4.231%; -4.184%]
  • 🟩 throughput [+1153784.782op/s; +1166717.078op/s] or [+4.367%; +4.416%]

scenario:sdk_test_data/rules-based

  • 🟩 execution_time [-22.936µs; -22.302µs] or [-15.637%; -15.205%]

scenario:single_flag_killswitch/rules-based

  • 🟩 execution_time [-9.579ns; -8.736ns] or [-5.069%; -4.622%]

scenario:sql/obfuscate_sql_string

  • 🟥 execution_time [+4.698µs; +4.767µs] or [+5.556%; +5.637%]

scenario:tags/replace_trace_tags

  • 🟩 execution_time [-137.888ns; -129.968ns] or [-5.527%; -5.210%]

Candidate

Candidate benchmark details

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 971d183 1762945482 ban/simplify-codecov
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
sdk_test_data/rules-based execution_time 122.023µs 124.054µs ± 1.371µs 123.847µs ± 0.494µs 124.436µs 125.481µs 128.773µs 137.444µs 10.98% 5.527 47.025 1.10% 0.097µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
sdk_test_data/rules-based execution_time [123.864µs; 124.243µs] or [-0.153%; +0.153%] None None None

Group 2

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 971d183 1762945482 ban/simplify-codecov
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
ip_address/quantize_peer_ip_address_benchmark execution_time 5.008µs 5.072µs ± 0.038µs 5.076µs ± 0.034µs 5.097µs 5.131µs 5.133µs 5.137µs 1.19% 0.072 -1.229 0.75% 0.003µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
ip_address/quantize_peer_ip_address_benchmark execution_time [5.067µs; 5.078µs] or [-0.104%; +0.104%] None None None

Group 3

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 971d183 1762945482 ban/simplify-codecov
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching deserializing traces from msgpack to their internal representation execution_time 60.350ms 60.980ms ± 2.348ms 60.695ms ± 0.061ms 60.751ms 60.934ms 78.283ms 80.782ms 33.10% 7.827 60.305 3.84% 0.166ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching deserializing traces from msgpack to their internal representation execution_time [60.654ms; 61.305ms] or [-0.534%; +0.534%] None None None

Group 4

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 971d183 1762945482 ban/simplify-codecov
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching string interning on wordpress profile execution_time 158.938µs 159.798µs ± 0.342µs 159.743µs ± 0.119µs 159.869µs 160.219µs 160.785µs 162.989µs 2.03% 4.557 37.608 0.21% 0.024µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching string interning on wordpress profile execution_time [159.751µs; 159.845µs] or [-0.030%; +0.030%] None None None

Group 5

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 971d183 1762945482 ban/simplify-codecov
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
sql/obfuscate_sql_string execution_time 88.998µs 89.300µs ± 0.192µs 89.257µs ± 0.048µs 89.320µs 89.526µs 90.067µs 91.109µs 2.07% 5.907 46.291 0.21% 0.014µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
sql/obfuscate_sql_string execution_time [89.273µs; 89.326µs] or [-0.030%; +0.030%] None None None

Group 6

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 971d183 1762945482 ban/simplify-codecov
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
tags/replace_trace_tags execution_time 2.285µs 2.361µs ± 0.019µs 2.362µs ± 0.005µs 2.368µs 2.383µs 2.389µs 2.394µs 1.35% -2.122 5.640 0.79% 0.001µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
tags/replace_trace_tags execution_time [2.358µs; 2.363µs] or [-0.110%; +0.110%] None None None

Group 7

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 971d183 1762945482 ban/simplify-codecov
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
redis/obfuscate_redis_string execution_time 34.594µs 35.151µs ± 0.969µs 34.703µs ± 0.041µs 34.779µs 37.197µs 37.275µs 38.904µs 12.10% 1.774 1.462 2.75% 0.069µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
redis/obfuscate_redis_string execution_time [35.017µs; 35.285µs] or [-0.382%; +0.382%] None None None

Group 8

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 971d183 1762945482 ban/simplify-codecov
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
receiver_entry_point/report/2597 execution_time 6.116ms 6.368ms ± 0.081ms 6.392ms ± 0.020ms 6.406ms 6.444ms 6.471ms 6.539ms 2.31% -1.692 2.362 1.26% 0.006ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
receiver_entry_point/report/2597 execution_time [6.356ms; 6.379ms] or [-0.175%; +0.175%] None None None

Group 9

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 971d183 1762945482 ban/simplify-codecov
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time 204.299µs 204.851µs ± 0.324µs 204.794µs ± 0.212µs 205.042µs 205.399µs 205.606µs 206.748µs 0.95% 1.406 5.250 0.16% 0.023µs 1 200
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput 4836804.263op/s 4881611.790op/s ± 7692.702op/s 4882959.422op/s ± 5049.361op/s 4886568.063op/s 4892087.430op/s 4893971.868op/s 4894793.686op/s 0.24% -1.382 5.090 0.16% 543.956op/s 1 200
normalization/normalize_name/normalize_name/bad-name execution_time 18.205µs 18.294µs ± 0.038µs 18.297µs ± 0.024µs 18.317µs 18.355µs 18.377µs 18.396µs 0.54% -0.083 -0.022 0.20% 0.003µs 1 200
normalization/normalize_name/normalize_name/bad-name throughput 54358600.490op/s 54662863.086op/s ± 112126.912op/s 54654270.408op/s ± 70619.805op/s 54737993.046op/s 54854838.778op/s 54921730.580op/s 54929026.159op/s 0.50% 0.095 -0.024 0.20% 7928.570op/s 1 200
normalization/normalize_name/normalize_name/good execution_time 10.415µs 10.494µs ± 0.038µs 10.488µs ± 0.021µs 10.512µs 10.557µs 10.586µs 10.743µs 2.43% 1.849 8.748 0.36% 0.003µs 1 200
normalization/normalize_name/normalize_name/good throughput 93079635.668op/s 95296360.649op/s ± 340241.394op/s 95343524.290op/s ± 190068.954op/s 95521186.714op/s 95721539.474op/s 95847803.110op/s 96018740.513op/s 0.71% -1.772 8.146 0.36% 24058.700op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time [204.806µs; 204.896µs] or [-0.022%; +0.022%] None None None
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput [4880545.656op/s; 4882677.925op/s] or [-0.022%; +0.022%] None None None
normalization/normalize_name/normalize_name/bad-name execution_time [18.289µs; 18.299µs] or [-0.028%; +0.028%] None None None
normalization/normalize_name/normalize_name/bad-name throughput [54647323.375op/s; 54678402.798op/s] or [-0.028%; +0.028%] None None None
normalization/normalize_name/normalize_name/good execution_time [10.488µs; 10.499µs] or [-0.050%; +0.050%] None None None
normalization/normalize_name/normalize_name/good throughput [95249206.464op/s; 95343514.834op/s] or [-0.049%; +0.049%] None None None

Group 10

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 971d183 1762945482 ban/simplify-codecov
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
write only interface execution_time 1.205µs 3.236µs ± 1.426µs 3.019µs ± 0.031µs 3.049µs 3.698µs 14.015µs 14.742µs 388.30% 7.286 54.519 43.96% 0.101µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
write only interface execution_time [3.038µs; 3.433µs] or [-6.108%; +6.108%] None None None

Group 11

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 971d183 1762945482 ban/simplify-codecov
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
concentrator/add_spans_to_concentrator execution_time 11.179ms 11.209ms ± 0.013ms 11.208ms ± 0.008ms 11.215ms 11.233ms 11.250ms 11.284ms 0.68% 1.531 5.510 0.12% 0.001ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
concentrator/add_spans_to_concentrator execution_time [11.207ms; 11.211ms] or [-0.016%; +0.016%] None None None

Group 12

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 971d183 1762945482 ban/simplify-codecov
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_trace/test_trace execution_time 246.040ns 262.148ns ± 20.925ns 252.651ns ± 4.767ns 265.057ns 308.978ns 334.109ns 335.210ns 32.68% 1.958 3.116 7.96% 1.480ns 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_trace/test_trace execution_time [259.248ns; 265.048ns] or [-1.106%; +1.106%] None None None

Group 13

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 971d183 1762945482 ban/simplify-codecov
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
credit_card/is_card_number/ execution_time 3.896µs 3.912µs ± 0.003µs 3.913µs ± 0.002µs 3.914µs 3.917µs 3.919µs 3.921µs 0.20% -0.595 4.195 0.07% 0.000µs 1 200
credit_card/is_card_number/ throughput 255068399.629op/s 255595284.053op/s ± 191948.405op/s 255581346.372op/s ± 134329.029op/s 255734658.445op/s 255879927.088op/s 255955047.178op/s 256679566.229op/s 0.43% 0.608 4.260 0.07% 13572.802op/s 1 200
credit_card/is_card_number/ 3782-8224-6310-005 execution_time 77.029µs 77.476µs ± 0.637µs 77.166µs ± 0.073µs 77.624µs 78.943µs 79.853µs 79.919µs 3.57% 2.205 4.447 0.82% 0.045µs 1 200
credit_card/is_card_number/ 3782-8224-6310-005 throughput 12512593.203op/s 12908035.283op/s ± 104339.204op/s 12959038.424op/s ± 12284.294op/s 12967246.033op/s 12974271.825op/s 12978567.129op/s 12982040.645op/s 0.18% -2.167 4.248 0.81% 7377.896op/s 1 200
credit_card/is_card_number/ 378282246310005 execution_time 70.618µs 70.954µs ± 0.586µs 70.727µs ± 0.054µs 70.834µs 72.273µs 73.521µs 73.567µs 4.01% 2.949 8.570 0.82% 0.041µs 1 200
credit_card/is_card_number/ 378282246310005 throughput 13593129.859op/s 14094616.139op/s ± 113610.492op/s 14138854.279op/s ± 10699.224op/s 14147106.683op/s 14154017.037op/s 14158072.406op/s 14160725.439op/s 0.15% -2.904 8.264 0.80% 8033.475op/s 1 200
credit_card/is_card_number/37828224631 execution_time 3.891µs 3.911µs ± 0.003µs 3.911µs ± 0.001µs 3.912µs 3.915µs 3.916µs 3.921µs 0.26% -1.443 15.587 0.07% 0.000µs 1 200
credit_card/is_card_number/37828224631 throughput 255049653.857op/s 255693665.990op/s ± 176624.330op/s 255713434.483op/s ± 87983.569op/s 255790397.583op/s 255889483.935op/s 255930128.384op/s 257026936.443op/s 0.51% 1.475 15.829 0.07% 12489.226op/s 1 200
credit_card/is_card_number/378282246310005 execution_time 67.395µs 67.615µs ± 0.410µs 67.486µs ± 0.034µs 67.531µs 68.505µs 69.540µs 70.000µs 3.73% 3.661 14.304 0.61% 0.029µs 1 200
credit_card/is_card_number/378282246310005 throughput 14285624.656op/s 14790071.572op/s ± 87857.261op/s 14817928.954op/s ± 7483.922op/s 14824356.049op/s 14830330.547op/s 14834628.119op/s 14837898.962op/s 0.13% -3.609 13.833 0.59% 6212.446op/s 1 200
credit_card/is_card_number/37828224631000521389798 execution_time 44.750µs 45.209µs ± 0.168µs 45.208µs ± 0.122µs 45.334µs 45.491µs 45.512µs 45.557µs 0.77% -0.166 -0.410 0.37% 0.012µs 1 200
credit_card/is_card_number/37828224631000521389798 throughput 21950560.417op/s 22119975.009op/s ± 82092.379op/s 22120137.457op/s ± 59493.439op/s 22171919.267op/s 22244000.092op/s 22323061.071op/s 22346173.022op/s 1.02% 0.184 -0.391 0.37% 5804.808op/s 1 200
credit_card/is_card_number/x371413321323331 execution_time 6.027µs 6.045µs ± 0.038µs 6.034µs ± 0.003µs 6.037µs 6.108µs 6.176µs 6.351µs 5.26% 4.528 25.921 0.63% 0.003µs 1 200
credit_card/is_card_number/x371413321323331 throughput 157447275.153op/s 165425125.747op/s ± 1008917.121op/s 165726115.851op/s ± 79851.785op/s 165796203.187op/s 165872671.598op/s 165919534.549op/s 165928066.070op/s 0.12% -4.395 24.247 0.61% 71341.214op/s 1 200
credit_card/is_card_number_no_luhn/ execution_time 3.892µs 3.912µs ± 0.003µs 3.912µs ± 0.001µs 3.914µs 3.916µs 3.918µs 3.920µs 0.20% -1.384 11.816 0.07% 0.000µs 1 200
credit_card/is_card_number_no_luhn/ throughput 255122122.879op/s 255619946.083op/s ± 182431.648op/s 255640423.000op/s ± 93708.791op/s 255721263.224op/s 255850144.429op/s 255907266.110op/s 256924811.964op/s 0.50% 1.409 12.013 0.07% 12899.866op/s 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time 65.504µs 65.770µs ± 0.083µs 65.770µs ± 0.063µs 65.834µs 65.894µs 65.958µs 65.981µs 0.32% -0.151 -0.070 0.13% 0.006µs 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput 15155851.184op/s 15204476.535op/s ± 19301.656op/s 15204408.369op/s ± 14623.926op/s 15218857.997op/s 15235696.600op/s 15249445.012op/s 15266336.202op/s 0.41% 0.159 -0.065 0.13% 1364.833op/s 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time 52.940µs 53.001µs ± 0.032µs 52.999µs ± 0.019µs 53.017µs 53.064µs 53.102µs 53.120µs 0.23% 0.924 1.408 0.06% 0.002µs 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 throughput 18825417.461op/s 18867417.365op/s ± 11412.847op/s 18868429.344op/s ± 6640.935op/s 18875124.946op/s 18883332.815op/s 18886090.829op/s 18889382.278op/s 0.11% -0.919 1.394 0.06% 807.010op/s 1 200
credit_card/is_card_number_no_luhn/37828224631 execution_time 3.895µs 3.912µs ± 0.003µs 3.912µs ± 0.002µs 3.913µs 3.918µs 3.921µs 3.923µs 0.30% 0.142 4.395 0.08% 0.000µs 1 200
credit_card/is_card_number_no_luhn/37828224631 throughput 254883455.362op/s 255636007.364op/s ± 210100.520op/s 255653138.780op/s ± 109216.083op/s 255766537.667op/s 255901236.901op/s 255961524.850op/s 256750468.868op/s 0.43% -0.126 4.441 0.08% 14856.350op/s 1 200
credit_card/is_card_number_no_luhn/378282246310005 execution_time 49.742µs 49.804µs ± 0.036µs 49.802µs ± 0.019µs 49.819µs 49.884µs 49.911µs 49.925µs 0.25% 1.007 1.423 0.07% 0.003µs 1 200
credit_card/is_card_number_no_luhn/378282246310005 throughput 20030224.712op/s 20078753.598op/s ± 14426.692op/s 20079567.034op/s ± 7630.610op/s 20087877.971op/s 20098663.037op/s 20102863.472op/s 20103685.005op/s 0.12% -1.002 1.410 0.07% 1020.121op/s 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time 44.817µs 45.166µs ± 0.143µs 45.178µs ± 0.091µs 45.251µs 45.410µs 45.500µs 45.549µs 0.82% 0.125 -0.272 0.31% 0.010µs 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput 21954376.630op/s 22140550.011op/s ± 69837.664op/s 22134629.801op/s ± 44546.819op/s 22187492.911op/s 22250599.812op/s 22286711.374op/s 22313055.791op/s 0.81% -0.108 -0.278 0.31% 4938.269op/s 1 200
credit_card/is_card_number_no_luhn/x371413321323331 execution_time 6.027µs 6.043µs ± 0.030µs 6.034µs ± 0.003µs 6.037µs 6.104µs 6.175µs 6.198µs 2.72% 3.278 10.602 0.50% 0.002µs 1 200
credit_card/is_card_number_no_luhn/x371413321323331 throughput 161346318.948op/s 165476915.879op/s ± 811322.771op/s 165730806.980op/s ± 75817.185op/s 165797711.406op/s 165891492.533op/s 165923842.311op/s 165926651.191op/s 0.12% -3.250 10.393 0.49% 57369.183op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
credit_card/is_card_number/ execution_time [3.912µs; 3.913µs] or [-0.010%; +0.010%] None None None
credit_card/is_card_number/ throughput [255568681.850op/s; 255621886.256op/s] or [-0.010%; +0.010%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 execution_time [77.388µs; 77.565µs] or [-0.114%; +0.114%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 throughput [12893574.873op/s; 12922495.693op/s] or [-0.112%; +0.112%] None None None
credit_card/is_card_number/ 378282246310005 execution_time [70.873µs; 71.035µs] or [-0.114%; +0.114%] None None None
credit_card/is_card_number/ 378282246310005 throughput [14078870.818op/s; 14110361.461op/s] or [-0.112%; +0.112%] None None None
credit_card/is_card_number/37828224631 execution_time [3.911µs; 3.911µs] or [-0.010%; +0.010%] None None None
credit_card/is_card_number/37828224631 throughput [255669187.556op/s; 255718144.423op/s] or [-0.010%; +0.010%] None None None
credit_card/is_card_number/378282246310005 execution_time [67.558µs; 67.672µs] or [-0.084%; +0.084%] None None None
credit_card/is_card_number/378282246310005 throughput [14777895.401op/s; 14802247.743op/s] or [-0.082%; +0.082%] None None None
credit_card/is_card_number/37828224631000521389798 execution_time [45.185µs; 45.232µs] or [-0.051%; +0.051%] None None None
credit_card/is_card_number/37828224631000521389798 throughput [22108597.795op/s; 22131352.223op/s] or [-0.051%; +0.051%] None None None
credit_card/is_card_number/x371413321323331 execution_time [6.040µs; 6.051µs] or [-0.087%; +0.087%] None None None
credit_card/is_card_number/x371413321323331 throughput [165285299.537op/s; 165564951.956op/s] or [-0.085%; +0.085%] None None None
credit_card/is_card_number_no_luhn/ execution_time [3.912µs; 3.912µs] or [-0.010%; +0.010%] None None None
credit_card/is_card_number_no_luhn/ throughput [255594662.811op/s; 255645229.355op/s] or [-0.010%; +0.010%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time [65.759µs; 65.782µs] or [-0.018%; +0.018%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput [15201801.511op/s; 15207151.558op/s] or [-0.018%; +0.018%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time [52.997µs; 53.006µs] or [-0.008%; +0.008%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 throughput [18865835.654op/s; 18868999.075op/s] or [-0.008%; +0.008%] None None None
credit_card/is_card_number_no_luhn/37828224631 execution_time [3.911µs; 3.912µs] or [-0.011%; +0.011%] None None None
credit_card/is_card_number_no_luhn/37828224631 throughput [255606889.453op/s; 255665125.276op/s] or [-0.011%; +0.011%] None None None
credit_card/is_card_number_no_luhn/378282246310005 execution_time [49.799µs; 49.809µs] or [-0.010%; +0.010%] None None None
credit_card/is_card_number_no_luhn/378282246310005 throughput [20076754.197op/s; 20080752.999op/s] or [-0.010%; +0.010%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time [45.147µs; 45.186µs] or [-0.044%; +0.044%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput [22130871.183op/s; 22150228.840op/s] or [-0.044%; +0.044%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 execution_time [6.039µs; 6.047µs] or [-0.069%; +0.069%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 throughput [165364474.346op/s; 165589357.412op/s] or [-0.068%; +0.068%] None None None

Group 14

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 971d183 1762945482 ban/simplify-codecov
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
two way interface execution_time 17.764µs 25.896µs ± 9.735µs 18.051µs ± 0.165µs 34.647µs 44.224µs 44.693µs 67.961µs 276.49% 0.902 0.412 37.50% 0.688µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
two way interface execution_time [24.547µs; 27.246µs] or [-5.210%; +5.210%] None None None

Group 15

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 971d183 1762945482 ban/simplify-codecov
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
single_flag_killswitch/rules-based execution_time 176.849ns 179.829ns ± 2.375ns 179.475ns ± 1.681ns 181.053ns 183.909ns 186.265ns 189.871ns 5.79% 0.958 1.031 1.32% 0.168ns 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
single_flag_killswitch/rules-based execution_time [179.499ns; 180.158ns] or [-0.183%; +0.183%] None None None

Group 16

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 971d183 1762945482 ban/simplify-codecov
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time 533.440µs 534.219µs ± 0.369µs 534.187µs ± 0.237µs 534.421µs 534.904µs 535.144µs 535.204µs 0.19% 0.517 -0.101 0.07% 0.026µs 1 200
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput 1868446.424op/s 1871893.373op/s ± 1291.772op/s 1872003.315op/s ± 831.051op/s 1872834.535op/s 1873742.708op/s 1874295.927op/s 1874625.695op/s 0.14% -0.513 -0.105 0.07% 91.342op/s 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time 381.075µs 382.521µs ± 0.433µs 382.507µs ± 0.228µs 382.762µs 383.187µs 383.489µs 385.062µs 0.67% 0.920 5.548 0.11% 0.031µs 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput 2596981.449op/s 2614236.464op/s ± 2953.794op/s 2614330.647op/s ± 1559.834op/s 2615824.367op/s 2618216.785op/s 2621581.549op/s 2624156.847op/s 0.38% -0.898 5.437 0.11% 208.865op/s 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time 194.861µs 195.242µs ± 0.306µs 195.217µs ± 0.082µs 195.303µs 195.462µs 195.512µs 199.140µs 2.01% 10.384 130.019 0.16% 0.022µs 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput 5021599.485op/s 5121870.287op/s ± 7900.982op/s 5122505.390op/s ± 2158.923op/s 5124491.602op/s 5127866.661op/s 5129957.326op/s 5131863.226op/s 0.18% -10.270 128.109 0.15% 558.684op/s 1 200
normalization/normalize_service/normalize_service/[empty string] execution_time 36.168µs 36.261µs ± 0.047µs 36.256µs ± 0.028µs 36.285µs 36.344µs 36.429µs 36.438µs 0.50% 0.979 1.757 0.13% 0.003µs 1 200
normalization/normalize_service/normalize_service/[empty string] throughput 27443775.968op/s 27577815.215op/s ± 35456.277op/s 27581321.349op/s ± 21481.740op/s 27602400.974op/s 27626675.659op/s 27638269.482op/s 27648996.155op/s 0.25% -0.968 1.722 0.13% 2507.137op/s 1 200
normalization/normalize_service/normalize_service/test_ASCII execution_time 44.920µs 45.110µs ± 0.114µs 45.108µs ± 0.086µs 45.185µs 45.309µs 45.369µs 45.401µs 0.65% 0.202 -0.716 0.25% 0.008µs 1 200
normalization/normalize_service/normalize_service/test_ASCII throughput 22026075.937op/s 22168179.984op/s ± 55766.654op/s 22168901.912op/s ± 42122.597op/s 22213409.611op/s 22254383.928op/s 22260627.852op/s 22261892.365op/s 0.42% -0.193 -0.725 0.25% 3943.298op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time [534.168µs; 534.270µs] or [-0.010%; +0.010%] None None None
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput [1871714.346op/s; 1872072.400op/s] or [-0.010%; +0.010%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time [382.461µs; 382.581µs] or [-0.016%; +0.016%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput [2613827.096op/s; 2614645.831op/s] or [-0.016%; +0.016%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time [195.199µs; 195.284µs] or [-0.022%; +0.022%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput [5120775.287op/s; 5122965.287op/s] or [-0.021%; +0.021%] None None None
normalization/normalize_service/normalize_service/[empty string] execution_time [36.255µs; 36.268µs] or [-0.018%; +0.018%] None None None
normalization/normalize_service/normalize_service/[empty string] throughput [27572901.316op/s; 27582729.114op/s] or [-0.018%; +0.018%] None None None
normalization/normalize_service/normalize_service/test_ASCII execution_time [45.094µs; 45.126µs] or [-0.035%; +0.035%] None None None
normalization/normalize_service/normalize_service/test_ASCII throughput [22160451.263op/s; 22175908.706op/s] or [-0.035%; +0.035%] None None None

Group 17

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 971d183 1762945482 ban/simplify-codecov
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching serializing traces from their internal representation to msgpack execution_time 14.082ms 14.130ms ± 0.031ms 14.125ms ± 0.014ms 14.140ms 14.174ms 14.258ms 14.283ms 1.12% 2.275 7.519 0.22% 0.002ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching serializing traces from their internal representation to msgpack execution_time [14.126ms; 14.135ms] or [-0.030%; +0.030%] None None None

Baseline

Omitted due to size.

@bantonsson bantonsson force-pushed the ban/simplify-codecov branch 7 times, most recently from 4621ad0 to abedef5 Compare November 12, 2025 08:44
@dd-octo-sts
Copy link

dd-octo-sts bot commented Nov 12, 2025 •
edited
Loading

Artifact Size Benchmark Report

aarch64-alpine-linux-musl
Artifact Baseline Commit Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so 7.57 MB 7.50 MB --.82% (-64.00 KB) 💪
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a 83.05 MB 83.00 MB --.05% (-49.22 KB) 💪
aarch64-unknown-linux-gnu
Artifact Baseline Commit Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so 9.55 MB 9.55 MB +.01% (+1.43 KB) 🔍
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a 96.25 MB 96.28 MB +.03% (+35.14 KB) 🔍
libdatadog-x64-windows
Artifact Baseline Commit Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll 19.07 MB 19.07 MB +0% (+512 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib 65.49 KB 65.49 KB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb 132.16 MB 132.16 MB -0% (-8.00 KB) 👌
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib 696.66 MB 701.00 MB +.62% (+4.33 MB) 🔍
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll 6.34 MB 6.34 MB +.05% (+3.50 KB) 🔍
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib 65.49 KB 65.49 KB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb 19.07 MB 19.05 MB --.08% (-16.00 KB) 💪
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib 36.91 MB 37.27 MB +.96% (+365.31 KB) 🔍
libdatadog-x86-windows
Artifact Baseline Commit Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll 16.16 MB 16.17 MB +.05% (+9.50 KB) 🔍
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib 66.50 KB 66.50 KB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb 134.40 MB 134.30 MB --.07% (-104.00 KB) 💪
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib 684.68 MB 688.20 MB +.51% (+3.51 MB) 🔍
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll 4.86 MB 4.86 MB --.06% (-3.00 KB) 💪
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib 66.50 KB 66.50 KB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb 20.23 MB 20.21 MB --.11% (-24.00 KB) 💪
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib 34.65 MB 34.99 MB +.96% (+343.27 KB) 🔍
x86_64-alpine-linux-musl
Artifact Baseline Commit Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a 71.98 MB 71.93 MB --.06% (-50.02 KB) 💪
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so 8.92 MB 8.93 MB +.08% (+8.00 KB) 🔍
x86_64-unknown-linux-gnu
Artifact Baseline Commit Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a 90.98 MB 90.95 MB --.03% (-31.48 KB) 💪
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so 10.27 MB 10.28 MB +.01% (+1.39 KB) 🔍

@bantonsson bantonsson force-pushed the ban/simplify-codecov branch 2 times, most recently from 23f03a8 to 2cdfe59 Compare November 12, 2025 10:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

ci-build

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@bantonsson @codecov-commenter