[Perf] Linux/arm64: 91 Improvements on 7/6/2023 3:11:59 PM

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	f11edcdc155c810f0f114f8b2ccb02d8e96ab82c
Compare	5d54b08d5fc40d0b1c156f430a487a94c1e34f79
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Threading.Tasks.ValueTaskPerfTest

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
Await_FromCompletedTask - Duration of single invocation	35.62 ns	28.61 ns	0.80	0.27	False
CreateAndAwait_FromCompletedTask_ConfigureAwait - Duration of single invocation	25.14 ns	19.92 ns	0.79	0.08	False
CreateAndAwait_FromDelayedTCS - Duration of single invocation	222.81 ns	211.47 ns	0.95	0.42	False
CreateAndAwait_FromResult_ConfigureAwait - Duration of single invocation	16.25 ns	12.61 ns	0.78	0.33	False
Await_FromResult - Duration of single invocation	16.12 ns	11.91 ns	0.74	0.35	False
CreateAndAwait_FromCompletedTask - Duration of single invocation	25.34 ns	19.62 ns	0.77	0.11	False
CreateAndAwait_FromResult - Duration of single invocation	16.30 ns	12.19 ns	0.75	0.34	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tasks.ValueTaskPerfTest*'

Payloads

Baseline
Compare

System.Threading.Tasks.ValueTaskPerfTest.Await_FromCompletedTask

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 28.61487390899056 < 33.34832279729494.
IsChangePoint: Marked as a change because one of 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 44.19649391917961 (T) = (0 -28.32453875511059) / Math.Sqrt((1.3402980768082071 / (299)) + (0.23652569379052643 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19026524492290722 = (34.980021022333275 - 28.32453875511059) / 34.980021022333275 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Threading.Tasks.ValueTaskPerfTest.CreateAndAwait_FromCompletedTask_ConfigureAwait

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 19.92428416614487 < 23.95357382251005.
IsChangePoint: Marked as a change because one of 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 31.241158145501796 (T) = (0 -19.955999671050797) / Math.Sqrt((0.31571572244260665 / (299)) + (0.3866985364468786 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21553441115456992 = (25.438973939471165 - 19.955999671050797) / 25.438973939471165 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Threading.Tasks.ValueTaskPerfTest.CreateAndAwait_FromDelayedTCS

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 211.46507138644685 < 220.31627455221306.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 16.783999421544404 (T) = (0 -205.64069899862946) / Math.Sqrt((401.8422417571078 / (299)) + (122.76542870863013 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21139905644619017 = (260.7664886525686 - 205.64069899862946) / 260.7664886525686 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Threading.Tasks.ValueTaskPerfTest.CreateAndAwait_FromResult_ConfigureAwait

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.605457328978128 < 15.172555552515997.
IsChangePoint: Marked as a change because one of 4/27/2023 10:53:57 PM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 20.75613035005599 (T) = (0 -11.902770511159709) / Math.Sqrt((0.7265589252259097 / (299)) + (0.6371686130458514 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2834178394042537 = (16.610475623987178 - 11.902770511159709) / 16.610475623987178 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Threading.Tasks.ValueTaskPerfTest.Await_FromResult

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.911558067903377 < 15.482179970424674.
IsChangePoint: Marked as a change because one of 4/26/2023 2:24:59 PM, 7/6/2023 3:11:59 PM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 24.361620929467236 (T) = (0 -11.811223363708569) / Math.Sqrt((0.7063935548926421 / (299)) + (0.4217720312694984 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.28544460092106877 = (16.529471863110054 - 11.811223363708569) / 16.529471863110054 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Threading.Tasks.ValueTaskPerfTest.CreateAndAwait_FromCompletedTask

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 19.621969693199542 < 23.92709083614278.
IsChangePoint: Marked as a change because one of 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 47.5834908052072 (T) = (0 -19.63970524257827) / Math.Sqrt((0.4157620584244874 / (299)) + (0.1773791566997335 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22903772890066643 = (25.474275433185003 - 19.63970524257827) / 25.474275433185003 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Threading.Tasks.ValueTaskPerfTest.CreateAndAwait_FromResult

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.18794280495896 < 14.857679884447988.
IsChangePoint: Marked as a change because one of 4/26/2023 12:55:36 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 18.94918227330156 (T) = (0 -12.025091340092189) / Math.Sqrt((0.8187101230440123 / (299)) + (0.7171015338077503 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.27492930727599096 = (16.58471575359814 - 12.025091340092189) / 16.58471575359814 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	f11edcdc155c810f0f114f8b2ccb02d8e96ab82c
Compare	5d54b08d5fc40d0b1c156f430a487a94c1e34f79
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_String

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
ToUpper - Duration of single invocation	16.68 ns	11.17 ns	0.67	0.25	False
ToLower - Duration of single invocation	32.34 ns	27.51 ns	0.85	0.28	False
ToLower - Duration of single invocation	32.03 ns	27.06 ns	0.84	0.30	False
ToUpper - Duration of single invocation	32.32 ns	27.12 ns	0.84	0.34	False
ToUpper - Duration of single invocation	32.03 ns	27.02 ns	0.84	0.31	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

System.Tests.Perf_String.ToUpper(s: "TEST")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.173341397436122 < 15.640955084950953.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 23.648928681558807 (T) = (0 -11.798848664146071) / Math.Sqrt((8.685046794585997 / (299)) + (0.7044760600327841 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3663951441390164 = (18.621777524278954 - 11.798848664146071) / 18.621777524278954 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.ToLower(s: "TeSt")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 27.508859473210574 < 30.698693552724677.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 6/22/2023 12:06:43 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 45.37365214793177 (T) = (0 -26.859124503868696) / Math.Sqrt((5.853930262406842 / (299)) + (0.315940277568661 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2613805025478663 = (36.36395274768563 - 26.859124503868696) / 36.36395274768563 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.ToLower(s: "TEST")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 27.056416604588595 < 30.77977791741454.
IsChangePoint: Marked as a change because one of 4/25/2023 1:02:54 PM, 6/22/2023 12:06:43 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 43.025941345937746 (T) = (0 -27.23199572013145) / Math.Sqrt((4.672958749224636 / (299)) + (0.36404854280507326 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.24813804897669597 = (36.2194092719654 - 27.23199572013145) / 36.2194092719654 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.ToUpper(s: "TeSt")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 27.11879204879119 < 30.236438170933877.
IsChangePoint: Marked as a change because one of 4/25/2023 1:02:54 PM, 6/22/2023 12:06:43 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 45.95882696750559 (T) = (0 -26.566791047323022) / Math.Sqrt((6.229309952122278 / (299)) + (0.3174116356301178 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.26900211298182225 = (36.343184459385974 - 26.566791047323022) / 36.343184459385974 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.ToUpper(s: "test")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 27.020194936049585 < 29.84573311864761.
IsChangePoint: Marked as a change because one of 6/22/2023 12:06:43 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 44.96770790275354 (T) = (0 -26.44897786388619) / Math.Sqrt((5.623792628427194 / (299)) + (0.3796516637425127 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.27142231031783565 = (36.302206667108244 - 26.44897786388619) / 36.302206667108244 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	f11edcdc155c810f0f114f8b2ccb02d8e96ab82c
Compare	5d54b08d5fc40d0b1c156f430a487a94c1e34f79
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Int32

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
TryParseSpan - Duration of single invocation	21.74 ns	16.85 ns	0.77	0.05	False
ParseHex - Duration of single invocation	22.52 ns	17.24 ns	0.77	0.03	False
TryParse - Duration of single invocation	23.42 ns	18.74 ns	0.80	0.04	False
TryFormat - Duration of single invocation	21.15 ns	16.24 ns	0.77	0.06	False
ParseHex - Duration of single invocation	11.08 ns	7.60 ns	0.69	0.12	False
TryParse - Duration of single invocation	14.96 ns	10.79 ns	0.72	0.03	False
ParseSpan - Duration of single invocation	11.12 ns	6.63 ns	0.60	0.08	False
TryParseSpan - Duration of single invocation	14.94 ns	10.31 ns	0.69	0.03	False
ParseHex - Duration of single invocation	16.15 ns	11.75 ns	0.73	0.05	False
TryParse - Duration of single invocation	21.93 ns	17.39 ns	0.79	0.03	False
ParseSpan - Duration of single invocation	24.74 ns	19.97 ns	0.81	0.02	False
TryParseSpan - Duration of single invocation	23.33 ns	18.87 ns	0.81	0.02	False
ParseSpan - Duration of single invocation	16.31 ns	12.02 ns	0.74	0.04	False
TryParse - Duration of single invocation	9.78 ns	5.57 ns	0.57	0.20	False
ToString - Duration of single invocation	27.69 ns	23.95 ns	0.86	0.21	False
TryParseSpan - Duration of single invocation	9.30 ns	5.51 ns	0.59	0.20	False
ParseSpan - Duration of single invocation	23.19 ns	18.29 ns	0.79	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int32*'

Payloads

Baseline
Compare

System.Tests.Perf_Int32.TryParseSpan(value: "2147483647")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 16.84987065546916 < 20.60136037324674.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 66.1236681130891 (T) = (0 -17.243627828275677) / Math.Sqrt((0.6770376447039987 / (299)) + (0.035691908334412714 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21347774223312568 = (21.923890465903 - 17.243627828275677) / 21.923890465903 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.ParseHex(value: "7FFFFFFF")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 17.23967105990274 < 21.07817084561693.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 51.20094544818758 (T) = (0 -17.476401486931664) / Math.Sqrt((0.9601437980234496 / (299)) + (0.04073874160248626 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18921133261806325 = (21.554817167540758 - 17.476401486931664) / 21.554817167540758 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.TryParse(value: "-2147483648")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 18.74400622911569 < 22.243793363704775.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 52.75747227481684 (T) = (0 -18.896376152224537) / Math.Sqrt((0.6102231207809043 / (299)) + (0.057926153537002804 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18369784872243505 = (23.14875211666477 - 18.896376152224537) / 23.14875211666477 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.TryFormat(value: -2147483648)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 16.243269101363254 < 20.093509956920716.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 17.54852172408188 (T) = (0 -16.495668786628418) / Math.Sqrt((3.398119277664962 / (299)) + (0.5795295229922168 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20103647827809065 = (20.64633533090134 - 16.495668786628418) / 20.64633533090134 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.ParseHex(value: "4")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 7.601611130004492 < 10.448304175368428.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 16.286003330132026 (T) = (0 -7.271710187013647) / Math.Sqrt((0.531811206062007 / (299)) + (0.5625442647600404 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3222037704230038 = (10.728460663689184 - 7.271710187013647) / 10.728460663689184 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.TryParse(value: "12345")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 10.788170535984998 < 14.165397075875257.
IsChangePoint: Marked as a change because one of 4/25/2023 6:51:20 AM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 70.8629712375499 (T) = (0 -10.808849494854524) / Math.Sqrt((0.4017549848952728 / (299)) + (0.022269493347146077 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2660362940198176 = (14.726681178900652 - 10.808849494854524) / 14.726681178900652 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.ParseSpan(value: "4")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 6.633279566825317 < 10.52486528819595.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 70.3455356579415 (T) = (0 -6.701678536543645) / Math.Sqrt((0.5177124024009376 / (299)) + (0.0310925352471612 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4026344949460245 = (11.21872367896119 - 6.701678536543645) / 11.21872367896119 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.TryParseSpan(value: "12345")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 10.309823093008529 < 14.13775627792588.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 66.39407422583571 (T) = (0 -10.616045202869318) / Math.Sqrt((0.3334669394598074 / (299)) + (0.03574284117204753 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2799536899652914 = (14.74355892797727 - 10.616045202869318) / 14.74355892797727 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.ParseHex(value: "3039")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.751972192990966 < 15.191575896607185.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 31.622174073739778 (T) = (0 -11.781315497071574) / Math.Sqrt((0.6736322053737943 / (299)) + (0.13724635194827398 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23300744257487815 = (15.360403934847481 - 11.781315497071574) / 15.360403934847481 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.TryParse(value: "2147483647")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 17.39267292558486 < 20.788458181788087.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 65.48222620288897 (T) = (0 -17.300731427928522) / Math.Sqrt((0.6305028923459868 / (299)) + (0.03705392532497072 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21044504609015738 = (21.912004151523632 - 17.300731427928522) / 21.912004151523632 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.ParseSpan(value: "-2147483648")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 19.96924713120833 < 23.51335565623138.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 59.2745368089882 (T) = (0 -20.063924744460728) / Math.Sqrt((0.5724230431432178 / (299)) + (0.04995825609395835 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1831158688685881 = (24.56153079711748 - 20.063924744460728) / 24.56153079711748 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.TryParseSpan(value: "-2147483648")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 18.867861903014088 < 22.118816236782216.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 60.91259056659071 (T) = (0 -18.70210902468866) / Math.Sqrt((0.5888049652438775 / (299)) + (0.041395727431686534 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18950455072917222 = (23.074909355153373 - 18.70210902468866) / 23.074909355153373 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.ParseSpan(value: "12345")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.024042387573301 < 15.497773187121304.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 100.57861677156835 (T) = (0 -11.960428344980196) / Math.Sqrt((0.3629316926546544 / (299)) + (0.007060494332515196 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2606184572261962 = (16.176260364994267 - 11.960428344980196) / 16.176260364994267 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.TryParse(value: "4")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.572285340877665 < 9.203816623951669.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/20/2023 7:49:01 PM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 68.78212598051799 (T) = (0 -5.349368127892622) / Math.Sqrt((0.5929375137619891 / (299)) + (0.03173280324947315 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.46098348399893074 = (9.924312092659532 - 5.349368127892622) / 9.924312092659532 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.ToString(value: -2147483648)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 23.9454208891537 < 26.881767515817696.
IsChangePoint: Marked as a change because one of 4/17/2023 2:42:45 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 34.14310484399721 (T) = (0 -23.42877767006951) / Math.Sqrt((3.4310349883049978 / (299)) + (0.3379923388025918 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22003670994053778 = (30.03830817253382 - 23.42877767006951) / 30.03830817253382 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.TryParseSpan(value: "4")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.511916215134723 < 9.191956726743102.
IsChangePoint: Marked as a change because one of 4/25/2023 6:51:20 AM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 21.83190966692694 (T) = (0 -5.572007371461912) / Math.Sqrt((0.49986263660302216 / (299)) + (0.4855494053203044 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.43629782340179485 = (9.88466534773294 - 5.572007371461912) / 9.88466534773294 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.ParseSpan(value: "2147483647")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 18.288706729453946 < 21.982701291269244.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 94.98355976951461 (T) = (0 -18.295261173325603) / Math.Sqrt((0.6033106204025 / (299)) + (0.011359691126822878 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21824660072253593 = (23.40285464730311 - 18.295261173325603) / 23.40285464730311 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	f11edcdc155c810f0f114f8b2ccb02d8e96ab82c
Compare	5d54b08d5fc40d0b1c156f430a487a94c1e34f79
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsKeyFalse<String, String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
SortedDictionary - Duration of single invocation	392.24 μs	347.57 μs	0.89	0.20	False
ImmutableSortedDictionary - Duration of single invocation	364.68 μs	332.34 μs	0.91	0.16	False
SortedList - Duration of single invocation	357.21 μs	337.98 μs	0.95	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyFalse<String, String>*'

Payloads

Baseline
Compare

System.Collections.ContainsKeyFalse<String, String>.SortedDictionary(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 347.5737635185186 < 371.74670777248474.
IsChangePoint: Marked as a change because one of 3/30/2023 4:24:24 AM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 25.5258802514531 (T) = (0 -350781.54514631035) / Math.Sqrt((154506577.0531915 / (299)) + (30945302.345224816 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1102091352643323 = (394229.2049160539 - 350781.54514631035) / 394229.2049160539 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsKeyFalse<String, String>.ImmutableSortedDictionary(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 332.3405547429078 < 346.5233693886032.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 27.273035656262856 (T) = (0 -338936.7636223128) / Math.Sqrt((135785003.84199327 / (299)) + (25452638.355543565 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11108740636125321 = (381293.6907945939 - 338936.7636223128) / 381293.6907945939 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsKeyFalse<String, String>.SortedList(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 337.98088705673763 < 341.43202344508927.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 30.247293557918383 (T) = (0 -329947.9326761464) / Math.Sqrt((95306116.30540818 / (299)) + (24393882.654520307 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1195823542549473 = (374762.9710407817 - 329947.9326761464) / 374762.9710407817 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	f11edcdc155c810f0f114f8b2ccb02d8e96ab82c
Compare	5d54b08d5fc40d0b1c156f430a487a94c1e34f79
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_UInt16

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
Parse - Duration of single invocation	10.13 ns	5.64 ns	0.56	0.22	False
Parse - Duration of single invocation	16.03 ns	12.46 ns	0.78	0.06	False
TryParse - Duration of single invocation	15.91 ns	11.61 ns	0.73	0.06	False
Parse - Duration of single invocation	15.72 ns	12.48 ns	0.79	0.04	False
TryParse - Duration of single invocation	9.60 ns	5.20 ns	0.54	0.22	False
TryParse - Duration of single invocation	15.81 ns	11.46 ns	0.72	0.06	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt16*'

Payloads

Baseline
Compare

System.Tests.Perf_UInt16.Parse(value: "0")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.638527868404994 < 9.634403010632353.
IsChangePoint: Marked as a change because one of 5/2/2023 4:25:00 AM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 51.66955684587208 (T) = (0 -5.708535153261609) / Math.Sqrt((0.8670322167172971 / (299)) + (0.057975449495815984 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.43709002315292955 = (10.141115610058703 - 5.708535153261609) / 10.141115610058703 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt16.Parse(value: "12345")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.456941693986263 < 15.2067860193202.
IsChangePoint: Marked as a change because one of 4/11/2023 12:48:48 AM, 4/25/2023 1:02:54 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 37.46337266147128 (T) = (0 -12.224062554563496) / Math.Sqrt((1.6309350457873482 / (299)) + (0.07278462807502982 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.24369100373771602 = (16.162788774132544 - 12.224062554563496) / 16.162788774132544 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt16.TryParse(value: "65535")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.61007077402278 < 15.16206434484124.
IsChangePoint: Marked as a change because one of 4/25/2023 6:51:20 AM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 24.937013017865723 (T) = (0 -11.189075057423697) / Math.Sqrt((1.361853043508897 / (299)) + (0.44520858562082366 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.30507785857817926 = (16.101192335778347 - 11.189075057423697) / 16.101192335778347 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt16.Parse(value: "65535")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.483959139473713 < 15.154389380221302.
IsChangePoint: Marked as a change because one of 4/12/2023 2:59:13 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 39.05343608661847 (T) = (0 -12.278064418714466) / Math.Sqrt((1.6274729770885903 / (299)) + (0.06278814835716497 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.24379111849933338 = (16.23633987787757 - 12.278064418714466) / 16.23633987787757 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt16.TryParse(value: "0")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.202710781640431 < 9.129709123679332.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 49.126826139306694 (T) = (0 -5.371718923038833) / Math.Sqrt((0.7838914754256827 / (299)) + (0.08042541615948925 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.46188056453866805 = (9.98239158270438 - 5.371718923038833) / 9.98239158270438 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt16.TryParse(value: "12345")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.459861059768267 < 15.014188907843156.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 53.58887104842987 (T) = (0 -11.463812290498463) / Math.Sqrt((1.3777791509971504 / (299)) + (0.03477333691913206 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2851674353509999 = (16.037059386245883 - 11.463812290498463) / 16.037059386245883 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	f11edcdc155c810f0f114f8b2ccb02d8e96ab82c
Compare	5d54b08d5fc40d0b1c156f430a487a94c1e34f79
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Int16

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
TryParse - Duration of single invocation	16.50 ns	12.70 ns	0.77	0.05	False
Parse - Duration of single invocation	17.91 ns	14.11 ns	0.79	0.03	False
ToString - Duration of single invocation	25.70 ns	17.55 ns	0.68	0.22	False
TryParse - Duration of single invocation	9.78 ns	8.23 ns	0.84	0.23	False
Parse - Duration of single invocation	10.10 ns	5.92 ns	0.59	0.23	False
TryParse - Duration of single invocation	17.61 ns	13.53 ns	0.77	0.05	False
Parse - Duration of single invocation	16.72 ns	12.99 ns	0.78	0.04	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int16*'

Payloads

Baseline
Compare

System.Tests.Perf_Int16.TryParse(value: "32767")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.702795803430211 < 15.692413954925414.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 42.04262564748341 (T) = (0 -12.50383394905417) / Math.Sqrt((1.5336309119946245 / (299)) + (0.03453213394293883 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2288005691425509 = (16.21348959652619 - 12.50383394905417) / 16.21348959652619 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int16.Parse(value: "-32768")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 14.110928524204184 < 16.931567804110585.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 34.19496808667929 (T) = (0 -13.861762009668285) / Math.Sqrt((1.9567904953713997 / (299)) + (0.05827754310958975 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20574867638346406 = (17.452614301667484 - 13.861762009668285) / 17.452614301667484 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int16.ToString(value: -32768)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 17.550953666711337 < 22.06038251071374.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 42.150607091816006 (T) = (0 -18.288720624542183) / Math.Sqrt((3.084203359464021 / (299)) + (0.213459460438036 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.27370092046579064 = (25.18070191727507 - 18.288720624542183) / 25.18070191727507 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int16.TryParse(value: "0")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 8.22809459164159 < 9.206845428577456.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 19.185844288506797 (T) = (0 -5.590141233460596) / Math.Sqrt((0.5650409038363848 / (299)) + (0.6490357943388433 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4385955655267308 = (9.9574226532526 - 5.590141233460596) / 9.9574226532526 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int16.Parse(value: "0")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.9195772953048555 < 9.625571387675166.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 46.01795748747514 (T) = (0 -5.718954897859498) / Math.Sqrt((0.3519191380210114 / (299)) + (0.10512597492581754 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4364491691617358 = (10.148072871000354 - 5.718954897859498) / 10.148072871000354 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int16.TryParse(value: "-32768")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 13.530078785354458 < 16.534470960833342.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 40.99352306564425 (T) = (0 -13.199631493477186) / Math.Sqrt((1.8721346725313208 / (299)) + (0.04736624512221127 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23610768706179297 = (17.27944013823966 - 13.199631493477186) / 17.27944013823966 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int16.Parse(value: "32767")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.993211410371636 < 15.881037048214298.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 35.05932793388623 (T) = (0 -13.024255090917418) / Math.Sqrt((1.7007078701779068 / (299)) + (0.04519517364160106 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20489466438127252 = (16.38054042334193 - 13.024255090917418) / 16.38054042334193 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	f11edcdc155c810f0f114f8b2ccb02d8e96ab82c
Compare	5d54b08d5fc40d0b1c156f430a487a94c1e34f79
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Numerics.Tests.Perf_BigInteger

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
ToStringX - Duration of single invocation	67.72 ns	62.05 ns	0.92	0.01	False
ToStringX - Duration of single invocation	58.29 ns	51.26 ns	0.88	0.01	False
ToStringD - Duration of single invocation	28.71 ns	23.21 ns	0.81	0.10	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_BigInteger*'

Payloads

Baseline
Compare

System.Numerics.Tests.Perf_BigInteger.ToStringX(numberString: -2147483648)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 62.04550550667206 < 63.853718662442674.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 31.885907115873078 (T) = (0 -61.84013037015586) / Math.Sqrt((1.7386114987605963 / (299)) + (0.32238363765469064 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08275079676169171 = (67.4191159303622 - 61.84013037015586) / 67.4191159303622 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_BigInteger.ToStringX(numberString: 123)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 51.256709230433266 < 53.45764812858196.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 27.251751272595637 (T) = (0 -51.70912875664284) / Math.Sqrt((1.5140714070997094 / (299)) + (0.43393042917455027 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09365474134217476 = (57.052352028869294 - 51.70912875664284) / 57.052352028869294 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_BigInteger.ToStringD(numberString: 123)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 23.20757641535791 < 27.269673964249474.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 37.566332045049485 (T) = (0 -23.630829404730246) / Math.Sqrt((6.93067804181449 / (299)) + (0.23667179409894068 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.24437121680335125 = (31.2730667891729 - 23.630829404730246) / 31.2730667891729 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	f11edcdc155c810f0f114f8b2ccb02d8e96ab82c
Compare	5d54b08d5fc40d0b1c156f430a487a94c1e34f79
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Tests.Perf_PriorityQueue<String, String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
K_Max_Elements - Duration of single invocation	11.58 μs	10.32 μs	0.89	0.08	False
K_Max_Elements - Duration of single invocation	76.83 μs	69.15 μs	0.90	0.13	False
HeapSort - Duration of single invocation	3.39 μs	3.18 μs	0.94	0.05	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_PriorityQueue<String, String>*'

Payloads

Baseline
Compare

System.Collections.Tests.Perf_PriorityQueue<String, String>.K_Max_Elements(Size: 100)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 10.31814597359736 < 10.766768898007783.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 38.90800766789742 (T) = (0 -10335.32454331592) / Math.Sqrt((155471.43400925168 / (299)) + (14074.830091791222 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1309692765503495 = (11892.933430810659 - 10335.32454331592) / 11892.933430810659 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_PriorityQueue<String, String>.K_Max_Elements(Size: 1000)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 69.15483497807017 < 72.23501652005953.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 38.37883445221983 (T) = (0 -69099.20997625704) / Math.Sqrt((6856415.189056899 / (299)) + (595999.2488038884 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1271408915811905 = (79164.2194138648 - 69099.20997625704) / 79164.2194138648 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_PriorityQueue<String, String>.HeapSort(Size: 10)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.179064682277765 < 3.2426921870365693.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 29.4180869877506 (T) = (0 -3124.3561228430835) / Math.Sqrt((12811.26560147744 / (299)) + (2760.6881404230844 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1307509838200528 = (3594.316547602852 - 3124.3561228430835) / 3594.316547602852 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	f11edcdc155c810f0f114f8b2ccb02d8e96ab82c
Compare	5d54b08d5fc40d0b1c156f430a487a94c1e34f79
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsFalse<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
ImmutableSortedSet - Duration of single invocation	365.81 μs	335.90 μs	0.92	0.10	False
SortedSet - Duration of single invocation	359.12 μs	320.12 μs	0.89	0.06	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse<String>*'

Payloads

Baseline
Compare

System.Collections.ContainsFalse<String>.ImmutableSortedSet(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 335.9032078267477 < 346.8870510319767.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 46.44396093154716 (T) = (0 -335038.52788507845) / Math.Sqrt((117125261.77578864 / (299)) + (6573100.76213622 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1160729425682686 = (379034.1353035848 - 335038.52788507845) / 379034.1353035848 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsFalse<String>.SortedSet(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 320.11657772108845 < 333.0187418263888.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 54.459044783698815 (T) = (0 -318148.60421941837) / Math.Sqrt((102164145.38484377 / (299)) + (4512489.5575132305 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12439301651132319 = (363346.35312274506 - 318148.60421941837) / 363346.35312274506 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	f11edcdc155c810f0f114f8b2ccb02d8e96ab82c
Compare	5d54b08d5fc40d0b1c156f430a487a94c1e34f79
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_UInt32

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
ParseSpan - Duration of single invocation	22.58 ns	17.72 ns	0.79	0.05	False
ParseSpan - Duration of single invocation	10.67 ns	6.45 ns	0.60	0.10	False
ParseSpan - Duration of single invocation	16.13 ns	12.01 ns	0.74	0.04	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt32*'

Payloads

Baseline
Compare

System.Tests.Perf_UInt32.ParseSpan(value: "4294967295")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 17.72351248706837 < 21.578563051177618.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 68.63708342239937 (T) = (0 -17.891331852038753) / Math.Sqrt((0.6618233251558375 / (299)) + (0.05604798444577966 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2365744138473105 = (23.435593693162367 - 17.891331852038753) / 23.435593693162367 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt32.ParseSpan(value: "0")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 6.449213055276183 < 10.295480492821083.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 87.9293723546258 (T) = (0 -6.47153884785188) / Math.Sqrt((0.5983261271160107 / (299)) + (0.00898233696702879 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4134774653391194 = (11.033742892067457 - 6.47153884785188) / 11.033742892067457 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt32.ParseSpan(value: "12345")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.01281510908901 < 15.336207203865388.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 71.78129487324202 (T) = (0 -12.027524028642828) / Math.Sqrt((0.20227198160980425 / (299)) + (0.03603827764000917 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2595214969278364 = (16.242907766723757 - 12.027524028642828) / 16.242907766723757 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	f11edcdc155c810f0f114f8b2ccb02d8e96ab82c
Compare	5d54b08d5fc40d0b1c156f430a487a94c1e34f79
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Int64

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
TryParseSpan - Duration of single invocation	37.67 ns	33.17 ns	0.88	0.01	False
ParseSpan - Duration of single invocation	16.58 ns	12.45 ns	0.75	0.05	True
ParseSpan - Duration of single invocation	39.26 ns	35.87 ns	0.91	0.01	False
ParseSpan - Duration of single invocation	37.83 ns	34.75 ns	0.92	0.02	False
TryParseSpan - Duration of single invocation	15.49 ns	10.69 ns	0.69	0.04	True
TryParseSpan - Duration of single invocation	36.62 ns	33.45 ns	0.91	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int64*'

Payloads

Baseline
Compare

System.Tests.Perf_Int64.TryParseSpan(value: "-9223372036854775808")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 33.16645023547373 < 36.06396914905866.
IsChangePoint: Marked as a change because one of 4/18/2023 7:36:21 AM, 5/2/2023 12:51:17 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 38.64752302033375 (T) = (0 -33.97955935545417) / Math.Sqrt((0.7763370587806439 / (299)) + (0.13713748196927042 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11536082384237323 = (38.41064274707141 - 33.97955935545417) / 38.41064274707141 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int64.ParseSpan(value: "12345")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.446509354871969 < 15.996814234160668.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 68.86631367715937 (T) = (0 -12.594924551155987) / Math.Sqrt((0.3609515464385343 / (299)) + (0.03746372345625307 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2590615910896017 = (16.998612029949566 - 12.594924551155987) / 16.998612029949566 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int64.ParseSpan(value: "-9223372036854775808")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 35.86840224826575 < 37.27900993857815.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 5/2/2023 12:51:17 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 50.285699315813986 (T) = (0 -35.452699998611486) / Math.Sqrt((0.9420777684544153 / (299)) + (0.05818073614171102 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11021358523216195 = (39.84405629283715 - 35.452699998611486) / 39.84405629283715 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int64.ParseSpan(value: "9223372036854775807")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 34.75344979600631 < 35.9100930202465.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 39.443029898785724 (T) = (0 -35.01634879005225) / Math.Sqrt((1.323276860599064 / (299)) + (0.05838826549614228 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09614155950412151 = (38.740965643736686 - 35.01634879005225) / 38.740965643736686 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int64.TryParseSpan(value: "12345")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 10.686932740037543 < 14.73625492187884.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 26.201865162410126 (T) = (0 -10.742078383910119) / Math.Sqrt((0.2870556309358095 / (299)) + (0.40227891786743325 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3034673222043668 = (15.422217401064861 - 10.742078383910119) / 15.422217401064861 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int64.TryParseSpan(value: "9223372036854775807")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 33.44627285051766 < 34.741873953992396.
IsChangePoint: Marked as a change because one of 4/18/2023 7:36:21 AM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 40.459138226405834 (T) = (0 -33.64805989646108) / Math.Sqrt((0.8545352670948614 / (299)) + (0.06486945491576777 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09626640236058845 = (37.23227728210079 - 33.64805989646108) / 37.23227728210079 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	f11edcdc155c810f0f114f8b2ccb02d8e96ab82c
Compare	5d54b08d5fc40d0b1c156f430a487a94c1e34f79
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Sort<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
Array_Comparison - Duration of single invocation	350.36 μs	321.82 μs	0.92	0.02	False
Array_ComparerClass - Duration of single invocation	352.89 μs	323.20 μs	0.92	0.02	False
List - Duration of single invocation	334.31 μs	302.09 μs	0.90	0.04	False
Array - Duration of single invocation	332.16 μs	302.98 μs	0.91	0.07	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort<String>*'

Payloads

Baseline
Compare

System.Collections.Sort<String>.Array_Comparison(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 321.8178198933333 < 332.8023684490513.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 47.80009407970637 (T) = (0 -322907.2339014652) / Math.Sqrt((110475003.4398215 / (299)) + (6082218.483953184 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11929773454883674 = (366647.44326057495 - 322907.2339014652) / 366647.44326057495 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Sort<String>.Array_ComparerClass(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 323.1953612153846 < 334.1543579348718.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 59.55237044591608 (T) = (0 -321933.06248048745) / Math.Sqrt((114531096.44885051 / (299)) + (2502353.555351849 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12306562664974979 = (367111.9211014282 - 321933.06248048745) / 367111.9211014282 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Sort<String>.List(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 302.09486882666675 < 317.80507461866665.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 44.095395978118745 (T) = (0 -305400.6935761792) / Math.Sqrt((93180863.28363694 / (299)) + (9306721.46452601 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12767391104030484 = (350099.23174530885 - 305400.6935761792) / 350099.23174530885 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Sort<String>.Array(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 302.97519130769234 < 315.60691175433334.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 47.66942807524628 (T) = (0 -305460.7069378248) / Math.Sqrt((103590283.27470136 / (299)) + (7234669.75485666 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12914221032093667 = (350758.42526527325 - 305460.7069378248) / 350758.42526527325 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	f11edcdc155c810f0f114f8b2ccb02d8e96ab82c
Compare	5d54b08d5fc40d0b1c156f430a487a94c1e34f79
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_UInt64

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
Parse - Duration of single invocation	17.02 ns	12.24 ns	0.72	0.04	False
TryParse - Duration of single invocation	15.44 ns	11.12 ns	0.72	0.04	False
ParseSpan - Duration of single invocation	16.82 ns	12.52 ns	0.74	0.04	False
Parse - Duration of single invocation	10.88 ns	8.74 ns	0.80	0.14	False
TryParse - Duration of single invocation	9.11 ns	5.25 ns	0.58	0.30	False
ParseSpan - Duration of single invocation	11.00 ns	6.49 ns	0.59	0.14	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt64*'

Payloads

Baseline
Compare

System.Tests.Perf_UInt64.Parse(value: "12345")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.236407177974966 < 16.112466055633643.
IsChangePoint: Marked as a change because one of 5/2/2023 12:51:17 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 78.20181979134337 (T) = (0 -12.395641237723781) / Math.Sqrt((1.1236833123391863 / (299)) + (0.0043056065727980755 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.287462937968126 = (17.396486299781113 - 12.395641237723781) / 17.396486299781113 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt64.TryParse(value: "12345")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.121835596376375 < 14.697640719167683.
IsChangePoint: Marked as a change because one of 5/2/2023 12:51:17 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 56.55055371109409 (T) = (0 -11.037694185482742) / Math.Sqrt((1.0310992720115335 / (299)) + (0.05199634629606748 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3065981355738316 = (15.918177829846327 - 11.037694185482742) / 15.918177829846327 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt64.ParseSpan(value: "12345")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.523716904849456 < 15.88388382433309.
IsChangePoint: Marked as a change because one of 5/2/2023 4:25:00 AM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 74.72041764157373 (T) = (0 -12.42511927644144) / Math.Sqrt((1.0210934248191137 / (299)) + (0.01331754865443159 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.28606382493274335 = (17.403683564950224 - 12.42511927644144) / 17.403683564950224 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt64.Parse(value: "0")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 8.739854867900728 < 10.373824891816072.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 17.049122846602522 (T) = (0 -6.696615924869132) / Math.Sqrt((0.6234789190244716 / (299)) + (0.7996832971230418 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.39100734606638454 = (10.996217904459503 - 6.696615924869132) / 10.996217904459503 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt64.TryParse(value: "0")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.2461038808862615 < 8.666697953203926.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 22.395537461508813 (T) = (0 -5.299626539596006) / Math.Sqrt((0.6639966674399068 / (299)) + (0.4730838011209222 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.45366495351961944 = (9.700323224251221 - 5.299626539596006) / 9.700323224251221 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt64.ParseSpan(value: "0")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 6.485588634981008 < 10.440397235547751.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 23.86229198349156 (T) = (0 -6.630645113701022) / Math.Sqrt((0.5445965167056505 / (299)) + (0.41591206181885343 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.39823289211726615 = (11.018623362500454 - 6.630645113701022) / 11.018623362500454 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	f11edcdc155c810f0f114f8b2ccb02d8e96ab82c
Compare	5d54b08d5fc40d0b1c156f430a487a94c1e34f79
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Diagnostics.Perf_Activity

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
ActivityAllocations - Duration of single invocation	237.67 ns	204.06 ns	0.86	0.32	False
ActivityAllocations - Duration of single invocation	319.17 ns	262.44 ns	0.82	0.19	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Diagnostics.Perf_Activity*'

Payloads

Baseline
Compare

System.Diagnostics.Perf_Activity.ActivityAllocations(idFormat: Hierarchical)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 204.0591868645913 < 229.28088731363707.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 14.972156807917635 (T) = (0 -207.8286217502345) / Math.Sqrt((204.84232852774826 / (299)) + (93.79356124294588 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16838852689252182 = (249.9107196941889 - 207.8286217502345) / 249.9107196941889 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Diagnostics.Perf_Activity.ActivityAllocations(idFormat: W3C)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 262.4352250346211 < 298.75304433662706.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/5/2023 5:05:14 PM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 20.98254089690038 (T) = (0 -273.8793813497546) / Math.Sqrt((494.3551069712964 / (299)) + (114.22694822350837 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.19353967833038044 = (339.60676550427246 - 273.8793813497546) / 339.60676550427246 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	f11edcdc155c810f0f114f8b2ccb02d8e96ab82c
Compare	5d54b08d5fc40d0b1c156f430a487a94c1e34f79
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.CtorFromCollection<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
ImmutableSortedDictionary - Duration of single invocation	399.44 μs	377.69 μs	0.95	0.26	False
SortedList - Duration of single invocation	408.24 μs	381.88 μs	0.94	0.01	False
ImmutableSortedSet - Duration of single invocation	386.36 μs	359.76 μs	0.93	0.10	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollection<String>*'

Payloads

Baseline
Compare

System.Collections.CtorFromCollection<String>.ImmutableSortedDictionary(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 377.6948480998169 < 392.0538303936298.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 23.949667241040057 (T) = (0 -375711.2429942356) / Math.Sqrt((250471634.4619478 / (299)) + (34782793.09766708 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10672977587504762 = (420602.0002091555 - 375711.2429942356) / 420602.0002091555 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollection<String>.SortedList(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 381.8820230691057 < 388.53024340258713.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 39.96753509676113 (T) = (0 -379613.363781812) / Math.Sqrt((187604937.65231732 / (299)) + (14974856.964686 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12314698175024462 = (432927.0195585803 - 379613.363781812) / 432927.0195585803 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollection<String>.ImmutableSortedSet(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 359.75563003130594 < 371.5012620913461.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 23.56761358455997 (T) = (0 -364588.39113737573) / Math.Sqrt((156757578.08011457 / (299)) + (46274015.256685555 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11553800032180829 = (412214.87330154364 - 364588.39113737573) / 412214.87330154364 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	f11edcdc155c810f0f114f8b2ccb02d8e96ab82c
Compare	5d54b08d5fc40d0b1c156f430a487a94c1e34f79
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Threading.Tasks.Tests.Perf_AsyncMethods

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
EmptyAsyncMethodInvocation - Duration of single invocation	23.77 ns	15.19 ns	0.64	0.43	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tasks.Tests.Perf_AsyncMethods*'

Payloads

Baseline
Compare

System.Threading.Tasks.Tests.Perf_AsyncMethods.EmptyAsyncMethodInvocation

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 15.191584560606062 < 20.81423261074617.
IsChangePoint: Marked as a change because one of 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 18.11028687961891 (T) = (0 -15.451901977018174) / Math.Sqrt((2.7100963451007245 / (299)) + (1.0565759084151358 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2605050233557268 = (20.89520884528085 - 15.451901977018174) / 20.89520884528085 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	f11edcdc155c810f0f114f8b2ccb02d8e96ab82c
Compare	5d54b08d5fc40d0b1c156f430a487a94c1e34f79
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_SByte

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
TryParse - Duration of single invocation	14.11 ns	10.23 ns	0.73	0.06	False
Parse - Duration of single invocation	13.74 ns	9.79 ns	0.71	0.07	False
Parse - Duration of single invocation	14.65 ns	10.95 ns	0.75	0.07	False
TryParse - Duration of single invocation	13.60 ns	9.73 ns	0.72	0.08	False
ToString - Duration of single invocation	21.27 ns	14.94 ns	0.70	0.25	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_SByte*'

Payloads

Baseline
Compare

System.Tests.Perf_SByte.TryParse(value: "-128")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 10.231122516358955 < 13.790925482284976.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 39.22143247242087 (T) = (0 -10.615374452996301) / Math.Sqrt((1.3834640868610044 / (299)) + (0.04744670153553023 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.25157412268512236 = (14.183601575991743 - 10.615374452996301) / 14.183601575991743 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_SByte.Parse(value: "127")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 9.793182467278756 < 12.869143004169677.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 40.97760192463516 (T) = (0 -9.993831112882692) / Math.Sqrt((0.9492218287837877 / (299)) + (0.04623994038258464 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2517283453841657 = (13.355886263009074 - 9.993831112882692) / 13.355886263009074 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_SByte.Parse(value: "-128")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 10.94911284654608 < 13.904852669618053.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 42.99186501257132 (T) = (0 -10.923097483458635) / Math.Sqrt((1.150319727495743 / (299)) + (0.02912514552447724 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23494113351974746 = (14.27746015638207 - 10.923097483458635) / 14.27746015638207 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_SByte.TryParse(value: "127")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 9.729717126190774 < 12.723308656867781.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 6.297972439162292 (T) = (0 -10.44637565046634) / Math.Sqrt((0.9971746079151446 / (299)) + (2.608129010823222 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21400648889715237 = (13.29066398501021 - 10.44637565046634) / 13.29066398501021 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_SByte.ToString(value: -128)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 14.941902176122106 < 20.242689050158585.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 44.66091837906721 (T) = (0 -15.553809128320134) / Math.Sqrt((2.871262623204019 / (299)) + (0.25269768559440253 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3285551479488345 = (23.16468594673938 - 15.553809128320134) / 23.16468594673938 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	f11edcdc155c810f0f114f8b2ccb02d8e96ab82c
Compare	5d54b08d5fc40d0b1c156f430a487a94c1e34f79
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.AddGivenSize<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
SortedList - Duration of single invocation	374.27 μs	336.48 μs	0.90	0.00	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.AddGivenSize<String>*'

Payloads

Baseline
Compare

System.Collections.AddGivenSize<String>.SortedList(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 336.48237526595744 < 346.0079923510175.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 33.52549546206648 (T) = (0 -339607.37367074715) / Math.Sqrt((48236345.07259646 / (299)) + (12197694.322855445 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.0938075558975887 = (374762.97212688654 - 339607.37367074715) / 374762.97212688654 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	f11edcdc155c810f0f114f8b2ccb02d8e96ab82c
Compare	5d54b08d5fc40d0b1c156f430a487a94c1e34f79
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.TryGetValueTrue<String, String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
ImmutableSortedDictionary - Duration of single invocation	280.97 μs	260.58 μs	0.93	0.16	False
SortedList - Duration of single invocation	279.65 μs	256.36 μs	0.92	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueTrue<String, String>*'

Payloads

Baseline
Compare

System.Collections.TryGetValueTrue<String, String>.ImmutableSortedDictionary(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 260.57917984972676 < 268.6267456702517.
IsChangePoint: Marked as a change because one of 5/18/2023 7:10:13 PM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 33.810034423488325 (T) = (0 -263307.7523440067) / Math.Sqrt((98328653.88310239 / (299)) + (7494458.939382344 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10887543132464969 = (295478.0527883005 - 263307.7523440067) / 295478.0527883005 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.TryGetValueTrue<String, String>.SortedList(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 256.3629157786885 < 275.35465916347505.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 38.95422578315311 (T) = (0 -260207.9303060304) / Math.Sqrt((94829089.70478712 / (299)) + (7264053.077457451 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.122891399251726 = (296665.5783377831 - 260207.9303060304) / 296665.5783377831 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	f11edcdc155c810f0f114f8b2ccb02d8e96ab82c
Compare	5d54b08d5fc40d0b1c156f430a487a94c1e34f79
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.IndexerSet<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
SortedList - Duration of single invocation	284.83 μs	261.73 μs	0.92	0.02	False
SortedDictionary - Duration of single invocation	306.07 μs	279.43 μs	0.91	0.24	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IndexerSet<String>*'

Payloads

Baseline
Compare

System.Collections.IndexerSet<String>.SortedList(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 261.7332675 < 268.8044599758185.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 36.355915317980816 (T) = (0 -263224.6276906664) / Math.Sqrt((66454875.29189907 / (299)) + (8148931.650729608 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1129014091511133 = (296725.336288473 - 263224.6276906664) / 296725.336288473 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.IndexerSet<String>.SortedDictionary(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 279.4256898251488 < 285.6226029753266.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 40.626145610197966 (T) = (0 -278780.8244898497) / Math.Sqrt((122475464.29916428 / (299)) + (4470414.303447313 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11229296756208088 = (314045.97947616904 - 278780.8244898497) / 314045.97947616904 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	f11edcdc155c810f0f114f8b2ccb02d8e96ab82c
Compare	5d54b08d5fc40d0b1c156f430a487a94c1e34f79
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsKeyTrue<String, String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
ImmutableSortedDictionary - Duration of single invocation	283.26 μs	258.42 μs	0.91	0.16	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyTrue<String, String>*'

Payloads

Baseline
Compare

System.Collections.ContainsKeyTrue<String, String>.ImmutableSortedDictionary(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 258.4186089480874 < 268.8563875130208.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 35.87519147243245 (T) = (0 -261626.6176481055) / Math.Sqrt((70939065.43034287 / (299)) + (7284392.513437367 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1091015625337526 = (293666.0416558621 - 261626.6176481055) / 293666.0416558621 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	f11edcdc155c810f0f114f8b2ccb02d8e96ab82c
Compare	5d54b08d5fc40d0b1c156f430a487a94c1e34f79
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Threading.Tests.Perf_Lock

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
ReaderWriterLockSlimPerf - Duration of single invocation	30.61 ns	21.63 ns	0.71	0.47	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tests.Perf_Lock*'

Payloads

Baseline
Compare

System.Threading.Tests.Perf_Lock.ReaderWriterLockSlimPerf

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 21.6268137893095 < 30.33695162566048.
IsChangePoint: Marked as a change because one of 4/26/2023 12:55:36 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 37.191704548918274 (T) = (0 -21.975731975713035) / Math.Sqrt((8.91631689776597 / (299)) + (0.9168510683571477 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.34900601300283063 = (33.757196555808726 - 21.975731975713035) / 33.757196555808726 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	f11edcdc155c810f0f114f8b2ccb02d8e96ab82c
Compare	5d54b08d5fc40d0b1c156f430a487a94c1e34f79
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.TryGetValueFalse<String, String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
ImmutableSortedDictionary - Duration of single invocation	368.25 μs	333.50 μs	0.91	0.16	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueFalse<String, String>*'

Payloads

Baseline
Compare

System.Collections.TryGetValueFalse<String, String>.ImmutableSortedDictionary(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 333.49628941869304 < 349.86283245881776.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/12/2023 7:14:29 PM falls between 7/3/2023 9:53:34 PM and 7/12/2023 7:14:29 PM.
IsImprovementStdDev: Marked as improvement because 42.072538493569226 (T) = (0 -336685.89544624713) / Math.Sqrt((131861512.98745264 / (299)) + (8565241.871740082 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11586769218996723 = (380809.4020228796 - 336685.89544624713) / 380809.4020228796 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[EgorBo]

TLS improvements: dotnet/runtime#87082