Skip to content

Conversation

@emdneto
Copy link
Member

@emdneto emdneto commented Mar 10, 2025

To help show outcomes from #4466

$ tox -f benchmark-sdk -- -k 'baggage'
opentelemetry-sdk/benchmarks/test_baggage.py::test_set_baggage[10] PASSED                                                                                                                      [  6%]
opentelemetry-sdk/benchmarks/test_baggage.py::test_set_baggage[100] PASSED                                                                                                                     [ 12%]
opentelemetry-sdk/benchmarks/test_baggage.py::test_set_baggage[1000] PASSED                                                                                                                    [ 18%]
opentelemetry-sdk/benchmarks/test_baggage.py::test_set_baggage[10000] PASSED                                                                                                                   [ 25%]
opentelemetry-sdk/benchmarks/test_baggage.py::test_get_baggage[10] PASSED                                                                                                                      [ 31%]
opentelemetry-sdk/benchmarks/test_baggage.py::test_get_baggage[100] PASSED                                                                                                                     [ 37%]
opentelemetry-sdk/benchmarks/test_baggage.py::test_get_baggage[1000] PASSED                                                                                                                    [ 43%]
opentelemetry-sdk/benchmarks/test_baggage.py::test_get_baggage[10000] PASSED                                                                                                                   [ 50%]
opentelemetry-sdk/benchmarks/test_baggage.py::test_remove_baggage[10] PASSED                                                                                                                   [ 56%]
opentelemetry-sdk/benchmarks/test_baggage.py::test_remove_baggage[100] PASSED                                                                                                                  [ 62%]
opentelemetry-sdk/benchmarks/test_baggage.py::test_remove_baggage[1000] PASSED                                                                                                                 [ 68%]
opentelemetry-sdk/benchmarks/test_baggage.py::test_remove_baggage[10000] PASSED                                                                                                                [ 75%]
opentelemetry-sdk/benchmarks/test_baggage.py::test_clear_baggage[10] PASSED                                                                                                                    [ 81%]
opentelemetry-sdk/benchmarks/test_baggage.py::test_clear_baggage[100] PASSED                                                                                                                   [ 87%]
opentelemetry-sdk/benchmarks/test_baggage.py::test_clear_baggage[1000] PASSED                                                                                                                  [ 93%]
opentelemetry-sdk/benchmarks/test_baggage.py::test_clear_baggage[10000] PASSED                                                                                                                 [100%]

I'm adding the test under opentelemetry-sdk benchmarks because we have only one job to run benchmarks and it's tailored to SDK. For now, I think it's more important to have the benchmark published.

@emdneto emdneto requested a review from a team as a code owner March 10, 2025 14:49
@emdneto emdneto added the Skip Changelog PRs that do not require a CHANGELOG.md entry label Mar 10, 2025
@emdneto emdneto mentioned this pull request Mar 10, 2025
4 tasks
Signed-off-by: emdneto <[email protected]>
Copy link
Contributor

@tammy-baylis-swi tammy-baylis-swi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great! Is there a way to run that would also show the times each took?

@emdneto
Copy link
Member Author

emdneto commented Mar 10, 2025

This is great! Is there a way to run that would also show the times each took?

Benchmarks are published here you can control+f for the specific benchmark you want. Most recent benchmarks are at the very bottom of the page.

Locally you can have results like this: tox -f benchmark-sdk -- -k 'baggage'

------------------------------------------------------------------------------------------------------------------------ benchmark: 16 tests ------------------------------------------------------------------------------------------------------------------------
Name (time in ns)                             Min                           Max                          Mean                      StdDev                        Median                         IQR            Outliers             OPS            Rounds  Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_clear_baggage[10]                   166.2411 (1.0)            328,541.2677 (127.61)             294.9771 (1.37)           1,357.0462 (62.34)              250.0601 (1.19)              41.9095 (12.13)     80;8539  3,390,093.7895 (0.73)     133335           1
test_clear_baggage[100]                  203.4118 (1.22)             8,029.3976 (3.12)               217.3468 (1.01)              43.1534 (1.98)               213.2181 (1.01)               4.8757 (1.41)   7856;17200  4,600,942.2890 (0.99)     196693          17
test_clear_baggage[10000]                204.8716 (1.23)             2,574.6607 (1.0)                215.8921 (1.0)               21.7683 (1.0)                211.8371 (1.00)               3.4537 (1.0)    7847;16647  4,631,942.3873 (1.0)      198327          24
test_clear_baggage[1000]                 207.0796 (1.25)           127,797.4993 (49.64)              230.9329 (1.07)             664.0449 (30.51)              210.8281 (1.0)                6.6590 (1.93)     140;6143  4,330,262.0387 (0.93)      46785         100
test_get_baggage[10]                   2,520.3917 (15.16)       10,833,708.1037 (>1000.0)          3,061.5884 (14.18)         34,318.6225 (>1000.0)          2,645.8874 (12.55)             62.6314 (18.13)   163;12093    326,627.8431 (0.07)     177787           2
test_remove_baggage[10]                6,957.9110 (41.85)          125,250.3134 (48.65)            7,398.8153 (34.27)          1,221.1860 (56.10)            7,207.9711 (34.19)            125.2629 (36.27)   4902;9242    135,156.7735 (0.03)      86648           1
test_set_baggage[10]                   7,874.7980 (47.37)          278,374.6459 (108.12)           9,565.3656 (44.31)          6,335.4588 (291.04)           8,833.1290 (41.90)            208.1506 (60.27)    324;1934    104,543.8343 (0.02)      13575           1
test_get_baggage[100]                 24,457.9278 (147.12)       4,219,542.2575 (>1000.0)         26,085.4991 (120.83)        29,431.0285 (>1000.0)         25,167.1299 (119.37)           291.0383 (84.27)    193;3925     38,335.4751 (0.01)      36091           1
test_set_baggage[100]                179,375.0562 (>1000.0)      1,242,415.9795 (482.56)         190,795.0193 (883.75)        22,146.5954 (>1000.0)        185,916.6659 (881.84)         7,458.0312 (>1000.0)   105;279      5,241.2270 (0.00)       4362           1
test_remove_baggage[100]             199,040.8637 (>1000.0)        602,959.1896 (234.19)         203,913.0065 (944.51)        10,649.5800 (489.22)         201,249.9608 (954.57)         3,542.7511 (>1000.0)   259;425      4,904.0521 (0.00)       4402           1
test_get_baggage[1000]               251,374.6731 (>1000.0)      2,801,582.6829 (>1000.0)        266,384.3175 (>1000.0)       91,680.8942 (>1000.0)        254,790.7643 (>1000.0)        4,249.8577 (>1000.0)    41;637      3,753.9747 (0.00)       3861           1
test_get_baggage[10000]            2,591,624.8560 (>1000.0)      2,729,250.1181 (>1000.0)      2,627,058.8946 (>1000.0)       19,690.7752 (904.56)       2,623,104.0247 (>1000.0)       24,249.7772 (>1000.0)      96;8        380.6538 (0.00)        378           1
test_set_baggage[1000]            13,605,582.9935 (>1000.0)     14,368,624.8921 (>1000.0)     13,724,878.6769 (>1000.0)      126,854.0544 (>1000.0)     13,685,083.4824 (>1000.0)       91,875.4376 (>1000.0)       6;4         72.8604 (0.00)         68           1
test_remove_baggage[1000]         14,673,875.1791 (>1000.0)     15,584,084.2053 (>1000.0)     14,778,549.0365 (>1000.0)      123,663.6792 (>1000.0)     14,758,416.4515 (>1000.0)       89,457.9571 (>1000.0)       4;2         67.6656 (0.00)         68           1
test_set_baggage[10000]        1,650,506,999.8093 (>1000.0)  1,667,188,624.8514 (>1000.0)  1,658,910,741.4708 (>1000.0)    6,128,545.5021 (>1000.0)  1,659,008,624.9635 (>1000.0)    7,700,999.7331 (>1000.0)       2;0          0.6028 (0.00)          5           1
test_remove_baggage[10000]     1,676,516,375.0388 (>1000.0)  1,893,300,541.6766 (>1000.0)  1,789,521,425.1243 (>1000.0)  105,281,425.0826 (>1000.0)  1,805,116,917.0067 (>1000.0)  206,181,510.7008 (>1000.0)       1;0          0.5588 (0.00)          5           1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

@xrmx xrmx merged commit 6754bd2 into open-telemetry:main Mar 11, 2025
385 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Skip Changelog PRs that do not require a CHANGELOG.md entry

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants