Skip to content

[Perf] Windows/arm64: 5 Improvements on 7/13/2023 7:10:54 AM #19970

@performanceautofiler

Description

@performanceautofiler

Run Information

Name Value
Architecture arm64
OS Windows 10.0.25094
Queue AmpereWindows
Baseline 4e48d2d85ed15140b95323e615a5433726b583b2
Compare c88dbd3f943b8206b4821aaf10ddf3c216100942
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsTrue<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
ImmutableList - Duration of single invocation
📝 - Benchmark Source
772.67 μs 534.66 μs 0.69 0.70 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue&lt;Int32&gt;*'

Payloads

Baseline
Compare

System.Collections.ContainsTrue<Int32>.ImmutableList(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 534.6593888888889 < 722.4387258548461.
IsChangePoint: Marked as a change because one of 6/10/2023 12:43:48 AM, 7/11/2023 7:34:35 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsImprovementStdDev: Marked as improvement because 17.81909047414676 (T) = (0 -575428.1071988196) / Math.Sqrt((12340773212.838934 / (299)) + (2185463509.533611 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.3840809945364179 = (934259.3784156956 - 575428.1071988196) / 934259.3784156956 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture arm64
OS Windows 10.0.25094
Queue AmpereWindows
Baseline 14e8b829e56398e3a12a42ed59c0f82c785ea71d
Compare c88dbd3f943b8206b4821aaf10ddf3c216100942
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_TimeSpan

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
FromSeconds - Duration of single invocation
📝 - Benchmark Source
1.56 ns 0.00 ns 0.00 0.91 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_TimeSpan*'

Payloads

Baseline
Compare

System.Tests.Perf_TimeSpan.FromSeconds

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 1.64452159161827.
IsChangePoint: Marked as a change because one of 6/22/2023 12:06:43 AM, 7/12/2023 6:09:40 AM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsImprovementStdDev: Marked as improvement because 2.8993898107640983 (T) = (0 -0.09936248612883798) / Math.Sqrt((0.37564829491561036 / (299)) + (0.026379836794407235 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.6869063152642727 = (0.317357043508261 - 0.09936248612883798) / 0.317357043508261 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture arm64
OS Windows 10.0.25094
Queue AmpereWindows
Baseline 4e48d2d85ed15140b95323e615a5433726b583b2
Compare c88dbd3f943b8206b4821aaf10ddf3c216100942
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsFalse<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
ImmutableList - Duration of single invocation
📝 - Benchmark Source
1.67 ms 1.26 ms 0.75 0.75 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse&lt;Int32&gt;*'

Payloads

Baseline
Compare

System.Collections.ContainsFalse<Int32>.ImmutableList(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.256945330357143 < 1.563353386067708.
IsChangePoint: Marked as a change because one of 5/27/2023 6:38:16 PM, 7/11/2023 7:34:35 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsImprovementStdDev: Marked as improvement because 13.949075992729865 (T) = (0 -1208011.98893722) / Math.Sqrt((51953265518.154526 / (299)) + (13090017586.962978 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.35914442330044044 = (1884998.7935792746 - 1208011.98893722) / 1884998.7935792746 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture arm64
OS Windows 10.0.25094
Queue AmpereWindows
Baseline 14e8b829e56398e3a12a42ed59c0f82c785ea71d
Compare c88dbd3f943b8206b4821aaf10ddf3c216100942
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Benchstone.BenchI.Fib

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
Test - Duration of single invocation
📝 - Benchmark Source
186.60 μs 144.41 μs 0.77 0.65 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.Fib*'

Payloads

Baseline
Compare

Benchstone.BenchI.Fib.Test

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 144.4115408878505 < 169.31678513907963.
IsChangePoint: Marked as a change because one of 6/11/2023 3:10:18 PM, 7/12/2023 6:09:40 AM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsImprovementStdDev: Marked as improvement because 8.944738276235501 (T) = (0 -132635.4144371888) / Math.Sqrt((392439247.36109716 / (299)) + (296983093.57390106 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.3246474254653368 = (196394.3271091819 - 132635.4144371888) / 196394.3271091819 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture arm64
OS Windows 10.0.25094
Queue AmpereWindows
Baseline 4e48d2d85ed15140b95323e615a5433726b583b2
Compare c88dbd3f943b8206b4821aaf10ddf3c216100942
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Diagnostics.Perf_Process

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
Start - Duration of single invocation
📝 - Benchmark Source
7.10 ms 3.49 ms 0.49 0.31 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Diagnostics.Perf_Process*'

Payloads

Baseline
Compare

System.Diagnostics.Perf_Process.Start

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.492688 < 6.74466275.
IsChangePoint: Marked as a change because one of 4/4/2023 10:11:10 PM, 5/8/2023 9:53:14 PM, 7/11/2023 7:34:35 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsImprovementStdDev: Marked as improvement because 22.578848559782106 (T) = (0 -3252015.4814814813) / Math.Sqrt((669958901835.1849 / (299)) + (82218223971.5687 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.4671456410857535 = (6103010.000908777 - 3252015.4814814813) / 6103010.000908777 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions