[Perf] Changes at 5/30/2022 8:02:30 PM #5771
Description
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.25094 |
| Baseline | 486b4d1a36aef5bbe6a77bb2c3412772e712961e |
| Compare | afacf6a99e1b98331af958caec638bd93dc46ffe |
| Diff | Diff |
Improvements in System.Collections.TryGetValueTrue<Int32, Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Dictionary - Duration of single invocation | 8.88 μs | 6.93 μs | 0.78 | 0.63 | False |
_1.png)
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryGetValueTrue<Int32, Int32>*'Payloads
Histogram
System.Collections.TryGetValueTrue<Int32, Int32>.Dictionary(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.934850344036697 < 8.063231472446585.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 5/30/2022 4:33:23 PM, 6/2/2022 10:01:42 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 10:01:42 AM.
IsImprovementStdDev: Marked as improvement because 15.872444951702478 (T) = (0 -6564.509887384387) / Math.Sqrt((969779.1685765414 / (85)) + (215055.1715459462 / (10))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (85) + (10) - 2, .975) and 0.30491280082209077 = (9444.153043169747 - 6564.509887384387) / 9444.153043169747 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.25094 |
| Baseline | 486b4d1a36aef5bbe6a77bb2c3412772e712961e |
| Compare | afacf6a99e1b98331af958caec638bd93dc46ffe |
| Diff | Diff |
Improvements in System.Buffers.Tests.ReadOnlySequenceTests<Char>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| IterateTryGetTenSegments - Duration of single invocation | 84.48 ns | 70.45 ns | 0.83 | 0.61 | False | |||||
| SliceMemory - Duration of single invocation | 17.02 ns | 14.51 ns | 0.85 | 0.58 | False | |||||
| SliceSingleSegment - Duration of single invocation | 13.88 ns | 12.12 ns | 0.87 | 0.61 | False |
_1.png)
_2.png)
_3.png)
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests<Char>*'Payloads
Histogram
System.Buffers.Tests.ReadOnlySequenceTests<Char>.IterateTryGetTenSegments
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 70.44792869704447 < 78.27017354420927.
IsChangePoint: Marked as a change because one of 5/30/2022 4:33:23 PM, 6/2/2022 10:01:42 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 10:01:42 AM.
IsImprovementStdDev: Marked as improvement because 7.775410793799989 (T) = (0 -64.92936446083436) / Math.Sqrt((36.081184065764766 / (85)) + (16.67676919620312 / (10))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (85) + (10) - 2, .975) and 0.14763965573892568 = (76.17595644612342 - 64.92936446083436) / 76.17595644612342 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Tests.ReadOnlySequenceTests<Char>.SliceMemory
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.51275750437141 < 15.207959585897788.
IsChangePoint: Marked as a change because one of 5/24/2022 3:42:52 PM, 6/2/2022 10:01:42 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 10:01:42 AM.
IsImprovementStdDev: Marked as improvement because 10.893789376145843 (T) = (0 -13.448005721108872) / Math.Sqrt((1.6362140792863586 / (61)) + (0.9686854954424382 / (34))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (61) + (34) - 2, .975) and 0.16003022372253123 = (16.010106673965097 - 13.448005721108872) / 16.010106673965097 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Tests.ReadOnlySequenceTests<Char>.SliceSingleSegment
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.122549894391998 < 12.640867111907493.
IsChangePoint: Marked as a change because one of 5/24/2022 3:42:52 PM, 6/2/2022 10:01:42 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 10:01:42 AM.
IsImprovementStdDev: Marked as improvement because 11.181124605679171 (T) = (0 -11.597411149371338) / Math.Sqrt((0.6557225323556414 / (61)) + (0.49876228273282386 / (34))) is greater than 1.9858018143453997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (61) + (34) - 2, .975) and 0.13323152947124983 = (13.380056547622955 - 11.597411149371338) / 13.380056547622955 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository