-
Notifications
You must be signed in to change notification settings - Fork 4
Description
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | ubuntu 20.04 |
| Queue | AmpereUbuntu |
| Baseline | d3d910ea88b00c7af2f69df07cbfbbe86da56596 |
| Compare | c88dbd3f943b8206b4821aaf10ddf3c216100942 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Collections.ContainsFalse<Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| ImmutableList - Duration of single invocation 📝 - Benchmark Source |
1.36 ms | 1.06 ms | 0.78 | 0.23 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse<Int32>*'Payloads
System.Collections.ContainsFalse<Int32>.ImmutableList(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0642278241666665 < 1.2755322944357639.
IsChangePoint: Marked as a change because one of 4/11/2023 5:52:15 AM, 5/19/2023 4:22:12 AM, 7/12/2023 7:14:29 PM, 7/18/2023 4:12:30 PM falls between 7/10/2023 3:14:43 AM and 7/18/2023 4:12:30 PM.
IsImprovementStdDev: Marked as improvement because 69.26217880729591 (T) = (0 -1064004.881440476) / Math.Sqrt((8545362964.125846 / (299)) + (210502.49598972528 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.25830251691175027 = (1434553.7172516966 - 1064004.881440476) / 1434553.7172516966 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | ubuntu 20.04 |
| Queue | AmpereUbuntu |
| Baseline | d3d910ea88b00c7af2f69df07cbfbbe86da56596 |
| Compare | c88dbd3f943b8206b4821aaf10ddf3c216100942 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Collections.ContainsTrue<Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| ImmutableList - Duration of single invocation 📝 - Benchmark Source |
682.49 μs | 524.53 μs | 0.77 | 0.27 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue<Int32>*'Payloads
System.Collections.ContainsTrue<Int32>.ImmutableList(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 524.5341230555555 < 638.2135721842918.
IsChangePoint: Marked as a change because one of 4/11/2023 5:52:15 AM, 5/19/2023 4:22:12 AM, 7/12/2023 7:14:29 PM, 7/18/2023 4:12:30 PM falls between 7/10/2023 3:14:43 AM and 7/18/2023 4:12:30 PM.
IsImprovementStdDev: Marked as improvement because 52.46308943564502 (T) = (0 -525164.9617413003) / Math.Sqrt((4315839688.018377 / (299)) + (1103046.285013192 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.2766346577873876 = (726002.3823299834 - 525164.9617413003) / 726002.3823299834 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | ubuntu 20.04 |
| Queue | AmpereUbuntu |
| Baseline | d3d910ea88b00c7af2f69df07cbfbbe86da56596 |
| Compare | c88dbd3f943b8206b4821aaf10ddf3c216100942 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Net.Primitives.Tests.CredentialCacheTests
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| GetCredential_HostPort - Duration of single invocation 📝 - Benchmark Source |
28.15 ns | 19.44 ns | 0.69 | 0.57 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Net.Primitives.Tests.CredentialCacheTests*'Payloads
System.Net.Primitives.Tests.CredentialCacheTests.GetCredential_HostPort(host: "notfound", hostPortCount: 0)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 19.444516008780745 < 21.791835942566852.
IsChangePoint: Marked as a change because one of 5/9/2023 11:02:55 PM, 7/12/2023 7:14:29 PM, 7/18/2023 4:12:30 PM falls between 7/10/2023 3:14:43 AM and 7/18/2023 4:12:30 PM.
IsImprovementStdDev: Marked as improvement because 7.198673239606639 (T) = (0 -17.861274013867522) / Math.Sqrt((4.307016913460749 / (299)) + (2.2338693921213486 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.21488569265426632 = (22.74990258966471 - 17.861274013867522) / 22.74990258966471 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | ubuntu 20.04 |
| Queue | AmpereUbuntu |
| Baseline | d3d910ea88b00c7af2f69df07cbfbbe86da56596 |
| Compare | c88dbd3f943b8206b4821aaf10ddf3c216100942 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Tests.Perf_TimeSpan
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| FromSeconds - Duration of single invocation 📝 - Benchmark Source |
1.47 ns | 0.01 ns | 0.01 | 0.42 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_TimeSpan*'Payloads
System.Tests.Perf_TimeSpan.FromSeconds
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.01221329277590462 < 1.426116986596517.
IsChangePoint: Marked as a change because one of 6/22/2023 12:06:43 AM, 7/12/2023 7:14:29 PM, 7/18/2023 4:12:30 PM falls between 7/10/2023 3:14:43 AM and 7/18/2023 4:12:30 PM.
IsImprovementStdDev: Marked as improvement because 6.09708372630424 (T) = (0 -0.02446903194751749) / Math.Sqrt((0.2756808968659237 / (299)) + (0.0008366898945599469 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.8915883931831897 = (0.2257049098890702 - 0.02446903194751749) / 0.2257049098890702 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | ubuntu 20.04 |
| Queue | AmpereUbuntu |
| Baseline | d3d910ea88b00c7af2f69df07cbfbbe86da56596 |
| Compare | c88dbd3f943b8206b4821aaf10ddf3c216100942 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in Benchstone.BenchI.EightQueens
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| Test - Duration of single invocation 📝 - Benchmark Source |
2.46 μs | 2.31 μs | 0.94 | 0.01 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.EightQueens*'Payloads
Benchstone.BenchI.EightQueens.Test
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.308366332431887 < 2.3382929331715054.
IsChangePoint: Marked as a change because one of 4/4/2023 10:11:10 PM, 5/19/2023 4:22:12 AM, 7/12/2023 7:14:29 PM, 7/18/2023 4:12:30 PM falls between 7/10/2023 3:14:43 AM and 7/18/2023 4:12:30 PM.
IsImprovementStdDev: Marked as improvement because 44.386905673190284 (T) = (0 -2308.007175342055) / Math.Sqrt((2856.7006294682847 / (299)) + (57.069991013900456 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.08093636159067812 = (2511.259371915377 - 2308.007175342055) / 2511.259371915377 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository




