Skip to content

[Perf] Linux/arm64: 5 Improvements on 7/13/2023 7:10:54 AM #19961

@performanceautofiler

Description

@performanceautofiler

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d3d910ea88b00c7af2f69df07cbfbbe86da56596
Compare c88dbd3f943b8206b4821aaf10ddf3c216100942
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsFalse<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
ImmutableList - Duration of single invocation
📝 - Benchmark Source
1.36 ms 1.06 ms 0.78 0.23 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse&lt;Int32&gt;*'

Payloads

Baseline
Compare

System.Collections.ContainsFalse<Int32>.ImmutableList(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0642278241666665 < 1.2755322944357639.
IsChangePoint: Marked as a change because one of 4/11/2023 5:52:15 AM, 5/19/2023 4:22:12 AM, 7/12/2023 7:14:29 PM, 7/18/2023 4:12:30 PM falls between 7/10/2023 3:14:43 AM and 7/18/2023 4:12:30 PM.
IsImprovementStdDev: Marked as improvement because 69.26217880729591 (T) = (0 -1064004.881440476) / Math.Sqrt((8545362964.125846 / (299)) + (210502.49598972528 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.25830251691175027 = (1434553.7172516966 - 1064004.881440476) / 1434553.7172516966 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d3d910ea88b00c7af2f69df07cbfbbe86da56596
Compare c88dbd3f943b8206b4821aaf10ddf3c216100942
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsTrue<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
ImmutableList - Duration of single invocation
📝 - Benchmark Source
682.49 μs 524.53 μs 0.77 0.27 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue&lt;Int32&gt;*'

Payloads

Baseline
Compare

System.Collections.ContainsTrue<Int32>.ImmutableList(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 524.5341230555555 < 638.2135721842918.
IsChangePoint: Marked as a change because one of 4/11/2023 5:52:15 AM, 5/19/2023 4:22:12 AM, 7/12/2023 7:14:29 PM, 7/18/2023 4:12:30 PM falls between 7/10/2023 3:14:43 AM and 7/18/2023 4:12:30 PM.
IsImprovementStdDev: Marked as improvement because 52.46308943564502 (T) = (0 -525164.9617413003) / Math.Sqrt((4315839688.018377 / (299)) + (1103046.285013192 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.2766346577873876 = (726002.3823299834 - 525164.9617413003) / 726002.3823299834 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d3d910ea88b00c7af2f69df07cbfbbe86da56596
Compare c88dbd3f943b8206b4821aaf10ddf3c216100942
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Net.Primitives.Tests.CredentialCacheTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
GetCredential_HostPort - Duration of single invocation
📝 - Benchmark Source
28.15 ns 19.44 ns 0.69 0.57 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Net.Primitives.Tests.CredentialCacheTests*'

Payloads

Baseline
Compare

System.Net.Primitives.Tests.CredentialCacheTests.GetCredential_HostPort(host: "notfound", hostPortCount: 0)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 19.444516008780745 < 21.791835942566852.
IsChangePoint: Marked as a change because one of 5/9/2023 11:02:55 PM, 7/12/2023 7:14:29 PM, 7/18/2023 4:12:30 PM falls between 7/10/2023 3:14:43 AM and 7/18/2023 4:12:30 PM.
IsImprovementStdDev: Marked as improvement because 7.198673239606639 (T) = (0 -17.861274013867522) / Math.Sqrt((4.307016913460749 / (299)) + (2.2338693921213486 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.21488569265426632 = (22.74990258966471 - 17.861274013867522) / 22.74990258966471 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d3d910ea88b00c7af2f69df07cbfbbe86da56596
Compare c88dbd3f943b8206b4821aaf10ddf3c216100942
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_TimeSpan

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
FromSeconds - Duration of single invocation
📝 - Benchmark Source
1.47 ns 0.01 ns 0.01 0.42 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_TimeSpan*'

Payloads

Baseline
Compare

System.Tests.Perf_TimeSpan.FromSeconds

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.01221329277590462 < 1.426116986596517.
IsChangePoint: Marked as a change because one of 6/22/2023 12:06:43 AM, 7/12/2023 7:14:29 PM, 7/18/2023 4:12:30 PM falls between 7/10/2023 3:14:43 AM and 7/18/2023 4:12:30 PM.
IsImprovementStdDev: Marked as improvement because 6.09708372630424 (T) = (0 -0.02446903194751749) / Math.Sqrt((0.2756808968659237 / (299)) + (0.0008366898945599469 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.8915883931831897 = (0.2257049098890702 - 0.02446903194751749) / 0.2257049098890702 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d3d910ea88b00c7af2f69df07cbfbbe86da56596
Compare c88dbd3f943b8206b4821aaf10ddf3c216100942
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Benchstone.BenchI.EightQueens

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
Test - Duration of single invocation
📝 - Benchmark Source
2.46 μs 2.31 μs 0.94 0.01 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.EightQueens*'

Payloads

Baseline
Compare

Benchstone.BenchI.EightQueens.Test

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.308366332431887 < 2.3382929331715054.
IsChangePoint: Marked as a change because one of 4/4/2023 10:11:10 PM, 5/19/2023 4:22:12 AM, 7/12/2023 7:14:29 PM, 7/18/2023 4:12:30 PM falls between 7/10/2023 3:14:43 AM and 7/18/2023 4:12:30 PM.
IsImprovementStdDev: Marked as improvement because 44.386905673190284 (T) = (0 -2308.007175342055) / Math.Sqrt((2856.7006294682847 / (299)) + (57.069991013900456 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.08093636159067812 = (2511.259371915377 - 2308.007175342055) / 2511.259371915377 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions