Skip to content

[Perf] Linux/arm64: 29 Improvements on 8/17/2022 3:53:31 PM #8019

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 96d1c8e3ba4bc94afdcf15ee2896a3bc984820e7
Compare 6521c2b7c01660873fa978c9e73038a1d75e7962
Diff Diff

Improvements in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Replace_Char - Duration of single invocation 12.42 ns 6.52 ns 0.53 0.24 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.Replace_Char(text: "This is a very nice sentence", oldChar: 'z', newChar: 'y')


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.5231315475069795 < 11.79517534599318.
IsChangePoint: Marked as a change because one of 8/17/2022 10:14:53 AM, 8/25/2022 6:55:57 AM falls between 8/16/2022 2:19:13 PM and 8/25/2022 6:55:57 AM.
IsImprovementStdDev: Marked as improvement because 62.73948931126887 (T) = (0 -6.75886029661809) / Math.Sqrt((0.4465356628006159 / (299)) + (0.29335227870669617 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.48608832435167293 = (13.151793619188407 - 6.75886029661809) / 13.151793619188407 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS ubuntu 20.04
Baseline 96d1c8e3ba4bc94afdcf15ee2896a3bc984820e7
Compare 6521c2b7c01660873fa978c9e73038a1d75e7962
Diff Diff

Improvements in System.Collections.ContainsTrue<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ICollection - Duration of single invocation 44.63 μs 34.73 μs 0.78 0.01 False
Span - Duration of single invocation 40.86 μs 30.99 μs 0.76 0.01 False
ImmutableArray - Duration of single invocation 46.34 μs 35.19 μs 0.76 0.01 False
Array - Duration of single invocation 45.34 μs 34.82 μs 0.77 0.01 False
Queue - Duration of single invocation 45.63 μs 34.28 μs 0.75 0.01 False
List - Duration of single invocation 43.21 μs 33.93 μs 0.79 0.01 False
Stack - Duration of single invocation 53.14 μs 34.46 μs 0.65 0.00 False

graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrue&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsTrue<Int32>.ICollection(Size: 512)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 34.72802012806236 < 42.41065817891695.
IsChangePoint: Marked as a change because one of 7/9/2022 5:50:00 AM, 7/27/2022 9:27:06 AM, 8/17/2022 10:14:53 AM, 8/25/2022 6:55:57 AM falls between 8/16/2022 2:19:13 PM and 8/25/2022 6:55:57 AM.
IsImprovementStdDev: Marked as improvement because 48.23028228355623 (T) = (0 -34653.8017536975) / Math.Sqrt((8439848.663259981 / (299)) + (89142.50295225113 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.1966344072557995 = (43135.78035539744 - 34653.8017536975) / 43135.78035539744 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.Span(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 30.987820256916994 < 38.81887260467175.
IsChangePoint: Marked as a change because one of 7/9/2022 5:50:00 AM, 7/27/2022 9:27:06 AM, 8/17/2022 10:14:53 AM, 8/25/2022 6:55:57 AM falls between 8/16/2022 2:19:13 PM and 8/25/2022 6:55:57 AM.
IsImprovementStdDev: Marked as improvement because 50.067394528286634 (T) = (0 -30744.00857484454) / Math.Sqrt((7109320.796897707 / (299)) + (230899.03150174508 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.22220393377347555 = (39527.081596078286 - 30744.00857484454) / 39527.081596078286 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.ImmutableArray(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 35.19234878277153 < 44.04607807856368.
IsChangePoint: Marked as a change because one of 7/9/2022 5:50:00 AM, 7/27/2022 9:27:06 AM, 8/17/2022 10:14:53 AM, 8/25/2022 6:55:57 AM falls between 8/16/2022 2:19:13 PM and 8/25/2022 6:55:57 AM.
IsImprovementStdDev: Marked as improvement because 55.9281162679861 (T) = (0 -35439.151686886165) / Math.Sqrt((6830494.382950892 / (299)) + (93158.27999421963 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.20181065306128174 = (44399.42956242817 - 35439.151686886165) / 44399.42956242817 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.Array(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 34.82390755382332 < 43.18217401846453.
IsChangePoint: Marked as a change because one of 7/9/2022 5:50:00 AM, 7/27/2022 9:27:06 AM, 8/17/2022 10:14:53 AM, 8/25/2022 6:55:57 AM falls between 8/16/2022 2:19:13 PM and 8/25/2022 6:55:57 AM.
IsImprovementStdDev: Marked as improvement because 54.10975524046876 (T) = (0 -34804.546152283314) / Math.Sqrt((8303620.337159046 / (299)) + (33327.18722123325 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.20870491015066514 = (43984.28171582641 - 34804.546152283314) / 43984.28171582641 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.Queue(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 34.28146618131868 < 43.31119880776569.
IsChangePoint: Marked as a change because one of 7/9/2022 5:50:00 AM, 7/27/2022 9:27:06 AM, 8/17/2022 10:14:53 AM, 8/25/2022 6:55:57 AM falls between 8/16/2022 2:19:13 PM and 8/25/2022 6:55:57 AM.
IsImprovementStdDev: Marked as improvement because 59.51203033578766 (T) = (0 -34371.81313649882) / Math.Sqrt((7350849.133431905 / (299)) + (46918.048478518285 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.218272488202218 = (43969.046269654835 - 34371.81313649882) / 43969.046269654835 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.List(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 33.92801268155411 < 41.0785154471685.
IsChangePoint: Marked as a change because one of 7/9/2022 5:50:00 AM, 7/27/2022 9:27:06 AM, 8/17/2022 10:14:53 AM, 8/25/2022 6:55:57 AM falls between 8/16/2022 2:19:13 PM and 8/25/2022 6:55:57 AM.
IsImprovementStdDev: Marked as improvement because 48.14745919069968 (T) = (0 -33873.690572516105) / Math.Sqrt((8023880.333197511 / (299)) + (35966.383448329914 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.1919365413032257 = (41919.592091377075 - 33873.690572516105) / 41919.592091377075 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.Stack(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 34.46220171622614 < 50.460943178354725.
IsChangePoint: Marked as a change because one of 8/17/2022 10:14:53 AM, 8/25/2022 6:55:57 AM falls between 8/16/2022 2:19:13 PM and 8/25/2022 6:55:57 AM.
IsImprovementStdDev: Marked as improvement because 190.64762470518656 (T) = (0 -33889.58188633876) / Math.Sqrt((6118.830866395206 / (299)) + (336553.8288407974 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.36252285511034754 = (53162.034369412664 - 33889.58188633876) / 53162.034369412664 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 96d1c8e3ba4bc94afdcf15ee2896a3bc984820e7
Compare 6521c2b7c01660873fa978c9e73038a1d75e7962
Diff Diff

Improvements in System.Tests.Perf_Array

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfShort - Duration of single invocation 16.42 ns 10.65 ns 0.65 0.20 False
IndexOfChar - Duration of single invocation 14.17 ns 11.22 ns 0.79 0.18 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Array*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Array.IndexOfShort


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.649194573085937 < 15.433609587801284.
IsChangePoint: Marked as a change because one of 5/31/2022 5:02:26 PM, 6/21/2022 3:41:14 AM, 8/17/2022 10:14:53 AM, 8/25/2022 6:55:57 AM falls between 8/16/2022 2:19:13 PM and 8/25/2022 6:55:57 AM.
IsImprovementStdDev: Marked as improvement because 53.11044476233621 (T) = (0 -11.826736650997166) / Math.Sqrt((1.0137946413454975 / (299)) + (0.19952261332451512 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.3037379852877427 = (16.986043186464475 - 11.826736650997166) / 16.986043186464475 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Array.IndexOfChar

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.215209773889082 < 13.027853196637492.
IsChangePoint: Marked as a change because one of 5/31/2022 5:02:26 PM, 6/6/2022 8:15:49 PM, 6/19/2022 8:44:36 PM, 7/28/2022 12:19:57 AM, 8/17/2022 10:14:53 AM, 8/25/2022 6:55:57 AM falls between 8/16/2022 2:19:13 PM and 8/25/2022 6:55:57 AM.
IsImprovementStdDev: Marked as improvement because 52.62154063247325 (T) = (0 -11.126090772955093) / Math.Sqrt((1.2654810938407839 / (299)) + (0.00923389875040985 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.24127807093317116 = (14.664253591087517 - 11.126090772955093) / 14.664253591087517 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 96d1c8e3ba4bc94afdcf15ee2896a3bc984820e7
Compare 6521c2b7c01660873fa978c9e73038a1d75e7962
Diff Diff

Improvements in System.Memory.ReadOnlySpan

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfString - Duration of single invocation 71.01 ns 61.49 ns 0.87 0.01 False
IndexOfString - Duration of single invocation 18.71 ns 14.50 ns 0.78 0.02 False
IndexOfString - Duration of single invocation 18.08 ns 12.15 ns 0.67 0.01 False
IndexOfString - Duration of single invocation 14.52 ns 12.60 ns 0.87 0.02 False
IndexOfString - Duration of single invocation 70.96 ns 61.47 ns 0.87 0.01 False
IndexOfString - Duration of single invocation 32.91 ns 27.50 ns 0.84 0.03 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.ReadOnlySpan*'

Payloads

Baseline
Compare

Histogram

System.Memory.ReadOnlySpan.IndexOfString(input: "だだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだxだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだ", value: "x", comparisonType: Ordinal)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 61.48677642487121 < 67.46144209617378.
IsChangePoint: Marked as a change because one of 5/31/2022 5:02:26 PM, 6/6/2022 8:15:49 PM, 8/17/2022 10:14:53 AM, 8/25/2022 6:55:57 AM falls between 8/16/2022 2:19:13 PM and 8/25/2022 6:55:57 AM.
IsImprovementStdDev: Marked as improvement because 120.25621663306836 (T) = (0 -61.46699594474983) / Math.Sqrt((1.2649407581760685 / (299)) + (0.002233120964248395 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.11368404507941063 = (69.35111074499054 - 61.46699594474983) / 69.35111074499054 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "だだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだすだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだ", value: "す", comparisonType: Ordinal)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.500239911574536 < 17.76303295353707.
IsChangePoint: Marked as a change because one of 5/31/2022 5:02:26 PM, 6/6/2022 8:15:49 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 8/5/2022 6:57:51 AM, 8/17/2022 10:14:53 AM, 8/25/2022 6:55:57 AM falls between 8/16/2022 2:19:13 PM and 8/25/2022 6:55:57 AM.
IsImprovementStdDev: Marked as improvement because 67.16022623426625 (T) = (0 -14.528585053881052) / Math.Sqrt((0.6493278346720135 / (299)) + (0.002209381763136355 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.179463635434491 = (17.70620496700867 - 14.528585053881052) / 17.70620496700867 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "Hello Worldbbbbbbbbbbbbbbcbbbbbbbbbbbbbbbbbbba!", value: "y", comparisonType: Ordinal)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.152080452773252 < 17.199338289356664.
IsChangePoint: Marked as a change because one of 8/17/2022 10:14:53 AM, 8/25/2022 6:55:57 AM falls between 8/16/2022 2:19:13 PM and 8/25/2022 6:55:57 AM.
IsImprovementStdDev: Marked as improvement because 177.05049479250712 (T) = (0 -12.150656921670395) / Math.Sqrt((0.33761072694985805 / (299)) + (0.001815890505464511 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.33396430840309466 = (18.24325193825221 - 12.150656921670395) / 18.24325193825221 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "Hello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello Worldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylong!xyz", value: "w", comparisonType: OrdinalIgnoreCase)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.598568813611779 < 13.770746576915569.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 8/17/2022 10:14:53 AM, 8/25/2022 6:55:57 AM falls between 8/16/2022 2:19:13 PM and 8/25/2022 6:55:57 AM.
IsImprovementStdDev: Marked as improvement because 17.342987889253486 (T) = (0 -12.620208239465788) / Math.Sqrt((3.2560039275464274 / (299)) + (0.001055900393090534 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.12558024684211588 = (14.432665998096567 - 12.620208239465788) / 14.432665998096567 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", value: "X", comparisonType: Ordinal)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 61.47040166037251 < 67.46701166580884.
IsChangePoint: Marked as a change because one of 5/31/2022 5:02:26 PM, 6/6/2022 8:15:49 PM, 8/17/2022 10:14:53 AM, 8/25/2022 6:55:57 AM falls between 8/16/2022 2:19:13 PM and 8/25/2022 6:55:57 AM.
IsImprovementStdDev: Marked as improvement because 124.9083369074399 (T) = (0 -61.452813092344904) / Math.Sqrt((1.1725789972381613 / (299)) + (0.002273056568323024 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.11378966374206156 = (69.3433720845912 - 61.452813092344904) / 69.3433720845912 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "Hello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello Worldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylong!xyz", value: "~", comparisonType: Ordinal)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.503908734411084 < 31.29310345494526.
IsChangePoint: Marked as a change because one of 5/31/2022 5:02:26 PM, 6/6/2022 8:15:49 PM, 8/17/2022 10:14:53 AM, 8/25/2022 6:55:57 AM falls between 8/16/2022 2:19:13 PM and 8/25/2022 6:55:57 AM.
IsImprovementStdDev: Marked as improvement because 89.64002776127477 (T) = (0 -27.615359671909303) / Math.Sqrt((1.2083728582286504 / (299)) + (0.011603186494285813 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.17705302892174452 = (33.556669679124816 - 27.615359671909303) / 33.556669679124816 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 96d1c8e3ba4bc94afdcf15ee2896a3bc984820e7
Compare 6521c2b7c01660873fa978c9e73038a1d75e7962
Diff Diff

Improvements in System.Collections.ContainsFalse<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
List - Duration of single invocation 68.99 μs 57.39 μs 0.83 0.01 False
Array - Duration of single invocation 70.37 μs 58.24 μs 0.83 0.01 False
Stack - Duration of single invocation 102.49 μs 56.51 μs 0.55 0.00 False
ImmutableArray - Duration of single invocation 70.36 μs 60.38 μs 0.86 0.01 False
ICollection - Duration of single invocation 69.59 μs 58.03 μs 0.83 0.01 False
Queue - Duration of single invocation 72.43 μs 61.61 μs 0.85 0.01 False
Span - Duration of single invocation 67.68 μs 53.24 μs 0.79 0.02 False

graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsFalse&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsFalse<Int32>.List(Size: 512)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 57.39248531527996 < 65.59012537340047.
IsChangePoint: Marked as a change because one of 7/9/2022 5:50:00 AM, 7/27/2022 9:27:06 AM, 8/17/2022 10:14:53 AM, 8/25/2022 6:55:57 AM falls between 8/16/2022 2:19:13 PM and 8/25/2022 6:55:57 AM.
IsImprovementStdDev: Marked as improvement because 76.69986882427078 (T) = (0 -57602.991101675936) / Math.Sqrt((4025806.8765011714 / (299)) + (137796.4679981946 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.15027181043776552 = (67789.90247616949 - 57602.991101675936) / 67789.90247616949 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.Array(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 58.235733771804384 < 66.51402150521255.
IsChangePoint: Marked as a change because one of 7/9/2022 5:50:00 AM, 7/27/2022 9:27:06 AM, 8/17/2022 10:14:53 AM, 8/25/2022 6:55:57 AM falls between 8/16/2022 2:19:13 PM and 8/25/2022 6:55:57 AM.
IsImprovementStdDev: Marked as improvement because 75.54724997044683 (T) = (0 -58310.01247635288) / Math.Sqrt((5151847.150959256 / (299)) + (75762.24781371198 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.15329118467356376 = (68866.66516383475 - 58310.01247635288) / 68866.66516383475 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.Stack(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 56.51486356220495 < 97.36379672792994.
IsChangePoint: Marked as a change because one of 8/17/2022 10:14:53 AM, 8/25/2022 6:55:57 AM falls between 8/16/2022 2:19:13 PM and 8/25/2022 6:55:57 AM.
IsImprovementStdDev: Marked as improvement because 245.32011854271116 (T) = (0 -56796.51071135819) / Math.Sqrt((1759.5656243410515 / (299)) + (1144899.7984588745 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.4458577802180442 = (102494.46565126641 - 56796.51071135819) / 102494.46565126641 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.ImmutableArray(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 60.3755062097812 < 66.84699489810217.
IsChangePoint: Marked as a change because one of 7/9/2022 5:50:00 AM, 7/27/2022 9:27:06 AM, 8/17/2022 10:14:53 AM, 8/25/2022 6:55:57 AM falls between 8/16/2022 2:19:13 PM and 8/25/2022 6:55:57 AM.
IsImprovementStdDev: Marked as improvement because 61.41840531870277 (T) = (0 -60047.38219103274) / Math.Sqrt((5241924.327868077 / (299)) + (199450.8164358406 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.13573225593982288 = (69477.7545543245 - 60047.38219103274) / 69477.7545543245 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.ICollection(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 58.025180003561246 < 66.6751223024103.
IsChangePoint: Marked as a change because one of 7/9/2022 5:50:00 AM, 7/27/2022 9:27:06 AM, 8/17/2022 10:14:53 AM, 8/25/2022 6:55:57 AM falls between 8/16/2022 2:19:13 PM and 8/25/2022 6:55:57 AM.
IsImprovementStdDev: Marked as improvement because 74.51418040720695 (T) = (0 -58199.11808562548) / Math.Sqrt((4626959.855632644 / (299)) + (135989.49894464496 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.15198661952989345 = (68629.95257617526 - 58199.11808562548) / 68629.95257617526 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.Queue(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 61.606944701443574 < 68.78396447311995.
IsChangePoint: Marked as a change because one of 7/9/2022 5:50:00 AM, 7/27/2022 9:27:06 AM, 8/17/2022 10:14:53 AM, 8/25/2022 6:55:57 AM falls between 8/16/2022 2:19:13 PM and 8/25/2022 6:55:57 AM.
IsImprovementStdDev: Marked as improvement because 62.001135712958245 (T) = (0 -61446.00774220529) / Math.Sqrt((5656578.24372516 / (299)) + (169226.95927961366 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.1352996697011809 = (71060.46521454556 - 61446.00774220529) / 71060.46521454556 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.Span(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 53.2442087585034 < 64.2150628165064.
IsChangePoint: Marked as a change because one of 7/9/2022 5:50:00 AM, 7/27/2022 9:27:06 AM, 8/17/2022 10:14:53 AM, 8/25/2022 6:55:57 AM falls between 8/16/2022 2:19:13 PM and 8/25/2022 6:55:57 AM.
IsImprovementStdDev: Marked as improvement because 59.27652881784904 (T) = (0 -53743.542896733736) / Math.Sqrt((4281492.135780292 / (299)) + (993210.5272234525 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.18860801180496362 = (66236.22574372187 - 53743.542896733736) / 66236.22574372187 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions