Skip to content

[Perf] Windows/arm64: 7 Regressions on 5/19/2023 9:46:56 PM #18097

@performanceautofiler

Description

@performanceautofiler

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 09399050c82b0ff2e9b082c36524f39a1e3ad6c1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Encodings.Web.Tests.Perf_Encoders

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EncodeUtf8 - Duration of single invocation 119.52 ns 129.46 ns 1.08 0.02 False
EncodeUtf8 - Duration of single invocation 63.78 ns 84.10 ns 1.32 0.03 True
EncodeUtf16 - Duration of single invocation 78.62 ns 84.24 ns 1.07 0.00 True

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Encodings.Web.Tests.Perf_Encoders*'

Payloads

Baseline
Compare

Histogram

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: Url,&lorem ipsum=dolor sit amet,512)


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 129.4565470391469 > 125.51529355310763.
IsChangePoint: Marked as a change because one of 2/14/2023 11:21:41 AM, 3/10/2023 4:14:40 PM, 4/27/2023 10:53:57 PM, 5/3/2023 7:05:50 AM, 5/19/2023 1:23:34 PM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -7.996212204546845 (T) = (0 -130.21842741095693) / Math.Sqrt((3.105750204626415 / (16)) + (17.729650741744347 / (12))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (12) - 2, .025) and -0.08623838832397639 = (119.88015596822987 - 130.21842741095693) / 119.88015596822987 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: Url,&lorem ipsum=dolor sit amet,16)


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 84.10347865610282 > 67.02422781170542.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -19.543606332067146 (T) = (0 -80.60045150271827) / Math.Sqrt((0.277060122757908 / (15)) + (9.64557109243251 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.2681436349060964 = (63.557825221183705 - 80.60045150271827) / 63.557825221183705 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: Url,&lorem ipsum=dolor sit amet,16)


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 84.23813569002982 > 82.55047654377887.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -37.07668863034435 (T) = (0 -84.15841862230614) / Math.Sqrt((0.002364276164011589 / (15)) + (0.2870552877439589 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.07031892066367323 = (78.62929169758299 - 84.15841862230614) / 78.62929169758299 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 09399050c82b0ff2e9b082c36524f39a1e3ad6c1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Memory.Span<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LastIndexOfValue - Duration of single invocation 13.74 ns 15.12 ns 1.10 0.05 False
SequenceCompareToDifferent - Duration of single invocation 2.64 ns 3.68 ns 1.39 0.49 False
IndexOfValue - Duration of single invocation 13.49 ns 14.85 ns 1.10 0.05 False

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Byte>.LastIndexOfValue(Size: 512)


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.11903849446876 > 14.751069542230661.
IsChangePoint: Marked as a change because one of 3/10/2023 4:14:40 PM, 4/6/2023 11:23:56 PM, 4/12/2023 8:39:35 PM, 4/27/2023 10:53:57 PM, 5/19/2023 1:23:34 PM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -6.206343144304712 (T) = (0 -15.20907025099283) / Math.Sqrt((0.964409563349339 / (16)) + (0.03859755570440446 / (12))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (12) - 2, .025) and -0.11460801505842126 = (13.645218808332057 - 15.20907025099283) / 13.645218808332057 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.SequenceCompareToDifferent(Size: 512)


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.6783630148283684 > 2.702172668779725.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -10.019576376673744 (T) = (0 -3.182781883323826) / Math.Sqrt((0.04151202725700269 / (15)) + (0.05929402434081391 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.36891685847099465 = (2.325036662108771 - 3.182781883323826) / 2.325036662108771 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.IndexOfValue(Size: 512)


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.846209896167936 > 14.049783107532253.
IsChangePoint: Marked as a change because one of 3/16/2023 2:30:29 AM, 4/3/2023 6:15:03 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -36.28272688819566 (T) = (0 -14.752836388418956) / Math.Sqrt((0.00945310583899296 / (15)) + (0.009503136183282334 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.09979275565177313 = (13.414196731706914 - 14.752836388418956) / 13.414196731706914 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 09399050c82b0ff2e9b082c36524f39a1e3ad6c1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Guid

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GuidToString - Duration of single invocation 22.81 ns 33.52 ns 1.47 0.23 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Guid*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Guid.GuidToString


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 33.51524056459398 > 23.373238819194494.
IsChangePoint: Marked as a change because one of 2/5/2023 4:54:05 PM, 5/19/2023 1:23:34 PM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -61.48679543066845 (T) = (0 -33.23479392397222) / Math.Sqrt((0.23443180408278638 / (15)) + (0.17472978387708846 / (12))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (12) - 2, .025) and -0.47373739577573476 = (22.551367712616358 - 33.23479392397222) / 22.551367712616358 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions