-
Notifications
You must be signed in to change notification settings - Fork 5.2k
Closed
Labels
arch-arm64area-System.IO.Compressiontenet-performancePerformance related issuePerformance related issuetenet-performance-benchmarksIssue from performance benchmarkIssue from performance benchmark
Milestone
Description
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 54c717a4ed822f46a23893479b8d4398596c041d |
| Compare | 1926f0939a855d6ef41593377c75ab4c31e9fcbb |
| Diff | Diff |
Regressions in System.IO.Compression.Gzip
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Compress - Duration of single invocation | 2.85 ms | 3.00 ms | 1.05 | 0.00 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Compression.Gzip*'Payloads
Histogram
System.IO.Compression.Gzip.Compress(level: Fastest, file: "TestDocument.pdf")
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.9996544642857135 > 2.9892825729166663.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 8/2/2022 9:32:02 PM, 8/4/2022 5:17:15 AM falls between 7/26/2022 2:47:26 PM and 8/4/2022 5:17:15 AM.
IsRegressionStdDev: Marked as regression because -68.96288776311692 (T) = (0 -2997694.554906898) / Math.Sqrt((117543328.83053406 / (41)) + (7034096.18717764 / (4))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (4) - 2, .025) and -0.052052518626114985 = (2849377.2904242603 - 2997694.554906898) / 2849377.2904242603 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 54c717a4ed822f46a23893479b8d4398596c041d |
| Compare | 1926f0939a855d6ef41593377c75ab4c31e9fcbb |
| Diff | Diff |
Regressions in System.Buffers.Tests.NonStandardArrayPoolTests<Object>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| RentNoReturn - Duration of single invocation | 51.08 ns | 54.81 ns | 1.07 | 0.01 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.NonStandardArrayPoolTests<Object>*'Payloads
Histogram
System.Buffers.Tests.NonStandardArrayPoolTests<Object>.RentNoReturn(RentalSize: 64, UseSharedPool: True)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 54.809619047619044 > 53.66642539062501.
IsChangePoint: Marked as a change because one of 5/7/2022 9:43:21 AM, 8/2/2022 9:32:02 PM, 8/4/2022 5:17:15 AM falls between 7/26/2022 2:47:26 PM and 8/4/2022 5:17:15 AM.
IsRegressionStdDev: Marked as regression because -18.036614790113084 (T) = (0 -54.494937921626985) / Math.Sqrt((0.35170233208069396 / (41)) + (0.10175683495165423 / (4))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (4) - 2, .025) and -0.06501357108349988 = (51.168303767421605 - 54.494937921626985) / 51.168303767421605 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 54c717a4ed822f46a23893479b8d4398596c041d |
| Compare | 1926f0939a855d6ef41593377c75ab4c31e9fcbb |
| Diff | Diff |
Regressions in System.IO.Compression.ZLib
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Compress - Duration of single invocation | 2.80 ms | 2.99 ms | 1.07 | 0.00 | False | |||||
| Compress - Duration of single invocation | 440.94 μs | 469.89 μs | 1.07 | 0.00 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Compression.ZLib*'Payloads
Histogram
System.IO.Compression.ZLib.Compress(level: Fastest, file: "TestDocument.pdf")
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.989092638888889 > 2.9355226614583327.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 8/2/2022 9:32:02 PM, 8/4/2022 5:17:15 AM falls between 7/26/2022 2:47:26 PM and 8/4/2022 5:17:15 AM.
IsRegressionStdDev: Marked as regression because -77.61854139659978 (T) = (0 -2990808.179372711) / Math.Sqrt((47550707.749999985 / (41)) + (19984158.008067783 / (4))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (4) - 2, .025) and -0.06882161333651177 = (2798229.510008116 - 2990808.179372711) / 2798229.510008116 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.IO.Compression.ZLib.Compress(level: Fastest, file: "sum")
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 469.8932116596639 > 461.38697743055553.
IsChangePoint: Marked as a change because one of 6/15/2022 7:24:26 PM, 7/27/2022 2:14:42 PM, 8/2/2022 9:32:02 PM, 8/4/2022 5:17:15 AM falls between 7/26/2022 2:47:26 PM and 8/4/2022 5:17:15 AM.
IsRegressionStdDev: Marked as regression because -15.066854202865196 (T) = (0 -469237.5212257106) / Math.Sqrt((34829965.80911427 / (41)) + (5512974.956863488 / (4))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (4) - 2, .025) and -0.050337598102365544 = (446749.2376484259 - 469237.5212257106) / 446749.2376484259 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 54c717a4ed822f46a23893479b8d4398596c041d |
| Compare | 1926f0939a855d6ef41593377c75ab4c31e9fcbb |
| Diff | Diff |
Regressions in System.IO.Compression.Deflate
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Compress - Duration of single invocation | 2.75 ms | 2.94 ms | 1.07 | 0.00 | False | |||||
| Compress - Duration of single invocation | 424.17 μs | 458.61 μs | 1.08 | 0.00 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Compression.Deflate*'Payloads
Histogram
System.IO.Compression.Deflate.Compress(level: Fastest, file: "TestDocument.pdf")
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.944779722222223 > 2.8891870000000006.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 8/2/2022 9:32:02 PM, 8/4/2022 5:17:15 AM falls between 7/26/2022 2:47:26 PM and 8/4/2022 5:17:15 AM.
IsRegressionStdDev: Marked as regression because -91.72982329289043 (T) = (0 -2947826.1991567463) / Math.Sqrt((71882321.59658116 / (41)) + (11089664.504111752 / (4))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (4) - 2, .025) and -0.07089152752181736 = (2752684.210676688 - 2947826.1991567463) / 2752684.210676688 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.IO.Compression.Deflate.Compress(level: Fastest, file: "sum")
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 458.6060171568628 > 445.7794805743244.
IsChangePoint: Marked as a change because one of 6/15/2022 7:24:26 PM, 7/27/2022 2:14:42 PM, 8/2/2022 9:32:02 PM, 8/4/2022 5:17:15 AM falls between 7/26/2022 2:47:26 PM and 8/4/2022 5:17:15 AM.
IsRegressionStdDev: Marked as regression because -14.913874638375276 (T) = (0 -458959.1882002802) / Math.Sqrt((28626618.235419188 / (40)) + (10684463.222928915 / (4))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (4) - 2, .025) and -0.06360486089072237 = (431512.8720039151 - 458959.1882002802) / 431512.8720039151 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 54c717a4ed822f46a23893479b8d4398596c041d |
| Compare | 1926f0939a855d6ef41593377c75ab4c31e9fcbb |
| Diff | Diff |
Regressions in System.Collections.CtorFromCollection<String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ImmutableArray - Duration of single invocation | 468.98 ns | 495.24 ns | 1.06 | 0.00 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CtorFromCollection<String>*'Payloads
Histogram
System.Collections.CtorFromCollection<String>.ImmutableArray(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 495.243094848904 > 487.6974805387518.
IsChangePoint: Marked as a change because one of 8/2/2022 9:32:02 PM, 8/4/2022 5:17:15 AM falls between 7/26/2022 2:47:26 PM and 8/4/2022 5:17:15 AM.
IsRegressionStdDev: Marked as regression because -18.059649380719744 (T) = (0 -498.1264953871379) / Math.Sqrt((27.368457010300922 / (41)) + (7.571560273277371 / (4))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (4) - 2, .025) and -0.06158568436984263 = (469.22872333458963 - 498.1264953871379) / 469.22872333458963 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Metadata
Metadata
Assignees
Labels
arch-arm64area-System.IO.Compressiontenet-performancePerformance related issuePerformance related issuetenet-performance-benchmarksIssue from performance benchmarkIssue from performance benchmark

_1.png)




_1.png)