Skip to content

Regressions in System.IO.Compression.Gzip #73389

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 54c717a4ed822f46a23893479b8d4398596c041d
Compare 1926f0939a855d6ef41593377c75ab4c31e9fcbb
Diff Diff

Regressions in System.IO.Compression.Gzip

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Compress - Duration of single invocation 2.85 ms 3.00 ms 1.05 0.00 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Compression.Gzip*'

Payloads

Baseline
Compare

Histogram

System.IO.Compression.Gzip.Compress(level: Fastest, file: "TestDocument.pdf")


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.9996544642857135 > 2.9892825729166663.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 8/2/2022 9:32:02 PM, 8/4/2022 5:17:15 AM falls between 7/26/2022 2:47:26 PM and 8/4/2022 5:17:15 AM.
IsRegressionStdDev: Marked as regression because -68.96288776311692 (T) = (0 -2997694.554906898) / Math.Sqrt((117543328.83053406 / (41)) + (7034096.18717764 / (4))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (4) - 2, .025) and -0.052052518626114985 = (2849377.2904242603 - 2997694.554906898) / 2849377.2904242603 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS Windows 10.0.19041
Baseline 54c717a4ed822f46a23893479b8d4398596c041d
Compare 1926f0939a855d6ef41593377c75ab4c31e9fcbb
Diff Diff

Regressions in System.Buffers.Tests.NonStandardArrayPoolTests<Object>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
RentNoReturn - Duration of single invocation 51.08 ns 54.81 ns 1.07 0.01 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.NonStandardArrayPoolTests&lt;Object&gt;*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.NonStandardArrayPoolTests<Object>.RentNoReturn(RentalSize: 64, UseSharedPool: True)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 54.809619047619044 > 53.66642539062501.
IsChangePoint: Marked as a change because one of 5/7/2022 9:43:21 AM, 8/2/2022 9:32:02 PM, 8/4/2022 5:17:15 AM falls between 7/26/2022 2:47:26 PM and 8/4/2022 5:17:15 AM.
IsRegressionStdDev: Marked as regression because -18.036614790113084 (T) = (0 -54.494937921626985) / Math.Sqrt((0.35170233208069396 / (41)) + (0.10175683495165423 / (4))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (4) - 2, .025) and -0.06501357108349988 = (51.168303767421605 - 54.494937921626985) / 51.168303767421605 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 54c717a4ed822f46a23893479b8d4398596c041d
Compare 1926f0939a855d6ef41593377c75ab4c31e9fcbb
Diff Diff

Regressions in System.IO.Compression.ZLib

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Compress - Duration of single invocation 2.80 ms 2.99 ms 1.07 0.00 False
Compress - Duration of single invocation 440.94 μs 469.89 μs 1.07 0.00 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Compression.ZLib*'

Payloads

Baseline
Compare

Histogram

System.IO.Compression.ZLib.Compress(level: Fastest, file: "TestDocument.pdf")


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.989092638888889 > 2.9355226614583327.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 8/2/2022 9:32:02 PM, 8/4/2022 5:17:15 AM falls between 7/26/2022 2:47:26 PM and 8/4/2022 5:17:15 AM.
IsRegressionStdDev: Marked as regression because -77.61854139659978 (T) = (0 -2990808.179372711) / Math.Sqrt((47550707.749999985 / (41)) + (19984158.008067783 / (4))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (4) - 2, .025) and -0.06882161333651177 = (2798229.510008116 - 2990808.179372711) / 2798229.510008116 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Compression.ZLib.Compress(level: Fastest, file: "sum")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 469.8932116596639 > 461.38697743055553.
IsChangePoint: Marked as a change because one of 6/15/2022 7:24:26 PM, 7/27/2022 2:14:42 PM, 8/2/2022 9:32:02 PM, 8/4/2022 5:17:15 AM falls between 7/26/2022 2:47:26 PM and 8/4/2022 5:17:15 AM.
IsRegressionStdDev: Marked as regression because -15.066854202865196 (T) = (0 -469237.5212257106) / Math.Sqrt((34829965.80911427 / (41)) + (5512974.956863488 / (4))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (4) - 2, .025) and -0.050337598102365544 = (446749.2376484259 - 469237.5212257106) / 446749.2376484259 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 54c717a4ed822f46a23893479b8d4398596c041d
Compare 1926f0939a855d6ef41593377c75ab4c31e9fcbb
Diff Diff

Regressions in System.IO.Compression.Deflate

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Compress - Duration of single invocation 2.75 ms 2.94 ms 1.07 0.00 False
Compress - Duration of single invocation 424.17 μs 458.61 μs 1.08 0.00 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Compression.Deflate*'

Payloads

Baseline
Compare

Histogram

System.IO.Compression.Deflate.Compress(level: Fastest, file: "TestDocument.pdf")


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.944779722222223 > 2.8891870000000006.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 8/2/2022 9:32:02 PM, 8/4/2022 5:17:15 AM falls between 7/26/2022 2:47:26 PM and 8/4/2022 5:17:15 AM.
IsRegressionStdDev: Marked as regression because -91.72982329289043 (T) = (0 -2947826.1991567463) / Math.Sqrt((71882321.59658116 / (41)) + (11089664.504111752 / (4))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (4) - 2, .025) and -0.07089152752181736 = (2752684.210676688 - 2947826.1991567463) / 2752684.210676688 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Compression.Deflate.Compress(level: Fastest, file: "sum")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 458.6060171568628 > 445.7794805743244.
IsChangePoint: Marked as a change because one of 6/15/2022 7:24:26 PM, 7/27/2022 2:14:42 PM, 8/2/2022 9:32:02 PM, 8/4/2022 5:17:15 AM falls between 7/26/2022 2:47:26 PM and 8/4/2022 5:17:15 AM.
IsRegressionStdDev: Marked as regression because -14.913874638375276 (T) = (0 -458959.1882002802) / Math.Sqrt((28626618.235419188 / (40)) + (10684463.222928915 / (4))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (4) - 2, .025) and -0.06360486089072237 = (431512.8720039151 - 458959.1882002802) / 431512.8720039151 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 54c717a4ed822f46a23893479b8d4398596c041d
Compare 1926f0939a855d6ef41593377c75ab4c31e9fcbb
Diff Diff

Regressions in System.Collections.CtorFromCollection<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableArray - Duration of single invocation 468.98 ns 495.24 ns 1.06 0.00 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CtorFromCollection&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorFromCollection<String>.ImmutableArray(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 495.243094848904 > 487.6974805387518.
IsChangePoint: Marked as a change because one of 8/2/2022 9:32:02 PM, 8/4/2022 5:17:15 AM falls between 7/26/2022 2:47:26 PM and 8/4/2022 5:17:15 AM.
IsRegressionStdDev: Marked as regression because -18.059649380719744 (T) = (0 -498.1264953871379) / Math.Sqrt((27.368457010300922 / (41)) + (7.571560273277371 / (4))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (4) - 2, .025) and -0.06158568436984263 = (469.22872333458963 - 498.1264953871379) / 469.22872333458963 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions