Skip to content

[Perf] Changes at 5/30/2022 5:18:40 PM #5868

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 9380113a892ec86d147e6e7506a10eda407a82be
Compare 842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff Diff

Regressions in System.Collections.TryAddGiventSize<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Dictionary - Duration of single invocation 9.87 μs 11.00 μs 1.11 0.02 True

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryAddGiventSize&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryAddGiventSize<String>.Dictionary(Count: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.99983260233918 > 10.379600335817404.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -27.985454026071398 (T) = (0 -11161.806655564626) / Math.Sqrt((18171.264563411456 / (21)) + (62396.81664550035 / (51))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (51) - 2, .025) and -0.12941891050354545 = (9882.787114471275 - 11161.806655564626) / 9882.787114471275 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.19042
Baseline 9380113a892ec86d147e6e7506a10eda407a82be
Compare 842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IsMatch - Duration of single invocation 108.82 ns 124.07 ns 1.14 0.06 False
IsMatch - Duration of single invocation 69.41 ns 82.95 ns 1.20 0.11 False
IsMatch - Duration of single invocation 118.61 ns 139.03 ns 1.17 0.05 False
IsMatch - Duration of single invocation 114.24 ns 129.23 ns 1.13 0.06 False
IsMatch - Duration of single invocation 124.22 ns 137.47 ns 1.11 0.07 False

graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 10, Options: None)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 124.07029333694665 > 112.18285357223827.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/15/2022 3:27:46 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -17.659561207622726 (T) = (0 -125.78850456044614) / Math.Sqrt((11.069921722261116 / (21)) + (26.65810365859347 / (51))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (51) - 2, .025) and -0.16801605427731614 = (107.69415719912769 - 125.78850456044614) / 107.69415719912769 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 1, Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 82.94520972092253 > 73.08364853189485.
IsChangePoint: Marked as a change because one of 4/22/2022 7:11:21 PM, 5/9/2022 11:56:12 AM, 5/11/2022 9:09:01 PM, 5/18/2022 4:10:31 AM, 5/27/2022 6:33:44 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -5.1530953800046 (T) = (0 -85.95155262244992) / Math.Sqrt((81.60278929021568 / (21)) + (4.037012106153998 / (51))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (51) - 2, .025) and -0.13556524854349644 = (75.69054506792408 - 85.95155262244992) / 75.69054506792408 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 8, Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 139.02711273942995 > 124.77740986557704.
IsChangePoint: Marked as a change because one of 5/11/2022 9:09:01 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -14.528949200403131 (T) = (0 -139.5461346234724) / Math.Sqrt((26.038615258660815 / (21)) + (13.25445868812534 / (51))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (51) - 2, .025) and -0.146142049397908 = (121.75291421929668 - 139.5461346234724) / 121.75291421929668 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 9, Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 129.2276694595754 > 119.90458456311633.
IsChangePoint: Marked as a change because one of 5/11/2022 9:09:01 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -19.881590939517157 (T) = (0 -129.93844950740427) / Math.Sqrt((0.602214722474547 / (21)) + (29.928751821323413 / (51))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (51) - 2, .025) and -0.13641773496881968 = (114.34039218947022 - 129.93844950740427) / 114.34039218947022 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 7, Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 137.46837103762078 > 130.7152144980496.
IsChangePoint: Marked as a change because one of 5/10/2022 1:42:57 AM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -24.0720493565256 (T) = (0 -140.60649842535577) / Math.Sqrt((4.633622733854685 / (21)) + (7.967504829284167 / (51))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (51) - 2, .025) and -0.11744425971045498 = (125.82864621971285 - 140.60649842535577) / 125.82864621971285 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 9380113a892ec86d147e6e7506a10eda407a82be
Compare 842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff Diff

Regressions in System.Tests.Perf_Double

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 115.29 ns 137.06 ns 1.19 0.07 False
Parse - Duration of single invocation 117.15 ns 142.74 ns 1.22 0.07 False
TryParse - Duration of single invocation 111.68 ns 143.84 ns 1.29 0.12 False
TryParse - Duration of single invocation 52.00 ns 66.79 ns 1.28 0.07 False
TryParse - Duration of single invocation 105.78 ns 139.11 ns 1.32 0.12 False
Parse - Duration of single invocation 52.43 ns 59.04 ns 1.13 0.03 False

graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Double*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Double.Parse(value: "1.7976931348623157e+308")


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 137.0568498266847 > 118.42904804825866.
IsChangePoint: Marked as a change because one of 4/11/2022 1:47:52 PM, 4/14/2022 8:50:18 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -10.334340350362416 (T) = (0 -131.80169251558272) / Math.Sqrt((40.321951925641216 / (21)) + (25.28302716541836 / (51))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (51) - 2, .025) and -0.1387831577717734 = (115.73906025575191 - 131.80169251558272) / 115.73906025575191 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.Parse(value: "-1.7976931348623157e+308")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 142.74121511431122 > 120.99850435845686.
IsChangePoint: Marked as a change because one of 3/30/2022 6:37:35 PM, 4/11/2022 1:47:52 PM, 4/14/2022 8:50:18 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -14.275495274950462 (T) = (0 -139.68790711519569) / Math.Sqrt((40.29464348616143 / (21)) + (16.917087763661282 / (51))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (51) - 2, .025) and -0.18107050910384775 = (118.27228437122324 - 139.68790711519569) / 118.27228437122324 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.TryParse(value: "-1.7976931348623157e+308")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 143.83730024294363 > 116.80401521030954.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -13.652462853232594 (T) = (0 -140.33222749282402) / Math.Sqrt((61.162080317859434 / (21)) + (26.22889934670685 / (51))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (51) - 2, .025) and -0.2196501308672507 = (115.05941248334771 - 140.33222749282402) / 115.05941248334771 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.TryParse(value: "12345")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 66.79212207249705 > 54.53303994018309.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -19.83714901356965 (T) = (0 -60.52895437722216) / Math.Sqrt((0.16938005838816028 / (21)) + (9.845877897051679 / (51))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (51) - 2, .025) and -0.1722997763865099 = (51.63265881001553 - 60.52895437722216) / 51.63265881001553 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.TryParse(value: "1.7976931348623157e+308")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 139.10875162319203 > 111.68270757256214.
IsChangePoint: Marked as a change because one of 4/8/2022 7:02:49 PM, 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -22.877109968049588 (T) = (0 -134.3378070688334) / Math.Sqrt((17.450951657834228 / (21)) + (25.889819900769293 / (51))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (51) - 2, .025) and -0.24537840120063906 = (107.86906769807601 - 134.3378070688334) / 107.86906769807601 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.Parse(value: "12345")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 59.04179914094483 > 55.24612904387697.
IsChangePoint: Marked as a change because one of 5/11/2022 9:09:01 PM, 5/20/2022 10:18:01 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -24.738934628181553 (T) = (0 -58.10612080608749) / Math.Sqrt((0.4033465286847956 / (21)) + (1.1662041490321697 / (51))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (51) - 2, .025) and -0.09568655713110737 = (53.031700013030864 - 58.10612080608749) / 53.031700013030864 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 9380113a892ec86d147e6e7506a10eda407a82be
Compare 842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff Diff

Regressions in System.Tests.Perf_Int32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryFormat - Duration of single invocation 8.07 ns 9.38 ns 1.16 0.03 False
TryParse - Duration of single invocation 20.36 ns 26.02 ns 1.28 0.04 False
TryParse - Duration of single invocation 15.75 ns 21.29 ns 1.35 0.03 False
ParseHex - Duration of single invocation 17.38 ns 18.38 ns 1.06 0.05 False
TryParse - Duration of single invocation 19.17 ns 28.61 ns 1.49 0.04 False

graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Int32*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int32.TryFormat(value: 12345)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.37799856842049 > 8.498350861161882.
IsChangePoint: Marked as a change because one of 4/25/2022 11:05:49 AM, 5/5/2022 12:13:02 AM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -17.724318634580737 (T) = (0 -8.919489304094956) / Math.Sqrt((0.0004444891519090147 / (21)) + (0.1115883659989248 / (51))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (51) - 2, .025) and -0.10302164759379544 = (8.086413647050827 - 8.919489304094956) / 8.086413647050827 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.TryParse(value: "2147483647")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 26.017896092048296 > 21.365472259432316.
IsChangePoint: Marked as a change because one of 4/8/2022 8:59:46 PM, 4/9/2022 9:54:15 AM, 6/6/2022 10:51:35 AM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -40.85155008877778 (T) = (0 -25.79118472708775) / Math.Sqrt((0.13984427597728366 / (67)) + (0.08814828878702677 / (5))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (67) + (5) - 2, .025) and -0.28602654588064363 = (20.05493961979338 - 25.79118472708775) / 20.05493961979338 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.TryParse(value: "12345")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.290688502785713 > 16.553673520973263.
IsChangePoint: Marked as a change because one of 3/30/2022 6:37:35 PM, 4/28/2022 4:25:37 PM, 6/6/2022 10:51:35 AM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -29.32840935389612 (T) = (0 -20.88541396048852) / Math.Sqrt((0.24087354605105246 / (67)) + (0.17797818274137664 / (5))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (67) + (5) - 2, .025) and -0.38503233096067585 = (15.079369263533462 - 20.88541396048852) / 15.079369263533462 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.ParseHex(value: "7FFFFFFF")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.38267020153511 > 18.260850227222154.
IsChangePoint: Marked as a change because one of 4/25/2022 5:27:54 PM, 4/28/2022 4:25:37 PM, 6/6/2022 10:51:35 AM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -10.065469884070335 (T) = (0 -18.748892800050733) / Math.Sqrt((0.13664069342588522 / (67)) + (0.08526251015378862 / (5))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (67) + (5) - 2, .025) and -0.08012283860872224 = (17.35811162386001 - 18.748892800050733) / 17.35811162386001 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.TryParse(value: "-2147483648")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 28.606607283299482 > 20.818654764652777.
IsChangePoint: Marked as a change because one of 3/30/2022 12:01:24 PM, 5/3/2022 11:42:39 AM, 6/6/2022 10:51:35 AM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -17.791275883165387 (T) = (0 -28.143571248773707) / Math.Sqrt((0.35942667907018855 / (67)) + (0.9290557314094756 / (5))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (67) + (5) - 2, .025) and -0.3819865555001683 = (20.364576729610796 - 28.143571248773707) / 20.364576729610796 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 9380113a892ec86d147e6e7506a10eda407a82be
Compare 842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff Diff

Regressions in System.Tests.Perf_Decimal

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParse - Duration of single invocation 75.22 ns 86.88 ns 1.16 0.06 False
Parse - Duration of single invocation 69.16 ns 84.46 ns 1.22 0.07 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Decimal*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Decimal.TryParse(value: "123456.789")


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 86.88126959512908 > 77.02922650063351.
IsChangePoint: Marked as a change because one of 4/25/2022 11:05:49 AM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -13.264538653424081 (T) = (0 -86.42769899558687) / Math.Sqrt((13.735816049828893 / (21)) + (16.80500262717185 / (51))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (51) - 2, .025) and -0.17953956201292576 = (73.27240372344524 - 86.42769899558687) / 73.27240372344524 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Decimal.Parse(value: "123456.789")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 84.45614574277016 > 71.40289225034365.
IsChangePoint: Marked as a change because one of 4/8/2022 7:02:49 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -25.353179878127523 (T) = (0 -81.27025313299572) / Math.Sqrt((1.8328652754728065 / (21)) + (7.075182700492638 / (51))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (51) - 2, .025) and -0.17413267063858984 = (69.21726578717401 - 81.27025313299572) / 69.21726578717401 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 9380113a892ec86d147e6e7506a10eda407a82be
Compare 842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff Diff

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Distinct - Duration of single invocation 2.12 μs 2.39 μs 1.12 0.03 True
Count - Duration of single invocation 256.41 ns 285.33 ns 1.11 0.19 False
ToDictionary - Duration of single invocation 1.88 μs 2.28 μs 1.21 0.18 False
Except - Duration of single invocation 2.73 μs 3.05 μs 1.12 0.13 False
Intersect - Duration of single invocation 3.31 μs 3.62 μs 1.09 0.09 False

graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Payloads

Baseline
Compare

Histogram

System.Linq.Tests.Perf_Enumerable.Distinct(input: IEnumerable)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.3851745810055864 > 2.240086085214206.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -34.51877392427077 (T) = (0 -2426.1893046715304) / Math.Sqrt((268.2467991658867 / (21)) + (3033.478339031773 / (51))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (51) - 2, .025) and -0.13757536552800953 = (2132.7723667305386 - 2426.1893046715304) / 2132.7723667305386 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Count(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 285.3330527840488 > 269.24427716325647.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -16.601026435680915 (T) = (0 -296.96140209897493) / Math.Sqrt((0.10455203912235275 / (21)) + (301.69007842752325 / (51))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (51) - 2, .025) and -0.1574385101341319 = (256.5677567308185 - 296.96140209897493) / 256.5677567308185 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.ToDictionary(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.2768355278276857 > 1.9744065370017274.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -68.68267413778204 (T) = (0 -2252.493686408211) / Math.Sqrt((217.01272602105442 / (21)) + (1001.2959460703461 / (51))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (51) - 2, .025) and -0.20036395357270662 = (1876.5089368970096 - 2252.493686408211) / 1876.5089368970096 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Except(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.0540807682563815 > 2.868999124943706.
IsChangePoint: Marked as a change because one of 5/31/2022 6:06:22 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -15.6746404826657 (T) = (0 -3044.42224242225) / Math.Sqrt((5971.528708813805 / (29)) + (3857.198357257714 / (43))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (43) - 2, .025) and -0.0971207152152939 = (2774.920024935293 - 3044.42224242225) / 2774.920024935293 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Intersect(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.6226016893554216 > 3.477522068783281.
IsChangePoint: Marked as a change because one of 6/1/2022 7:21:14 AM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -11.158577653665644 (T) = (0 -3585.9630530058885) / Math.Sqrt((9699.321246184028 / (33)) + (3477.4937798221026 / (39))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (39) - 2, .025) and -0.06485465055320434 = (3367.5610574109237 - 3585.9630530058885) / 3367.5610574109237 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions