-
Notifications
You must be signed in to change notification settings - Fork 5.2k
Closed
Labels
Milestone
Description
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | fe8422cdff450941bc3ba4b120576e8822459df4 |
| Compare | ef7ff071099d185a5f66b55966866fcc964e287c |
| Diff | Diff |
Regressions in BenchmarksGame.KNucleotide_9
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| RunBench - Duration of single invocation | 56.91 ms | 62.13 ms | 1.09 | 0.08 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'BenchmarksGame.KNucleotide_9*'Payloads
Histogram
BenchmarksGame.KNucleotide_9.RunBench
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 62.132452575 > 59.921577299062506.
IsChangePoint: Marked as a change because one of 12/7/2021 9:49:54 AM, 1/11/2022 2:18:34 PM, 1/18/2022 5:19:20 AM falls between 1/8/2022 8:22:56 PM and 1/18/2022 5:19:20 AM.
IsRegressionStdDev: Marked as regression because -22.718367898215927 (T) = (0 -62105451.3935966) / Math.Sqrt((395282169985.69183 / (32)) + (919908986491.3501 / (33))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (33) - 2, .025) and -0.07917867059962681 = (57548812.89405839 - 62105451.3935966) / 57548812.89405839 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | fe8422cdff450941bc3ba4b120576e8822459df4 |
| Compare | ef7ff071099d185a5f66b55966866fcc964e287c |
| Diff | Diff |
Regressions in System.Collections.CtorFromCollectionNonGeneric<Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Queue - Duration of single invocation | 23.05 μs | 24.77 μs | 1.07 | 0.00 | False | |||||
| Stack - Duration of single invocation | 21.92 μs | 23.80 μs | 1.09 | 0.03 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CtorFromCollectionNonGeneric<Int32>*'Payloads
Histogram
System.Collections.CtorFromCollectionNonGeneric<Int32>.Queue(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.767194148235916 > 24.231584690585265.
IsChangePoint: Marked as a change because one of 1/11/2022 2:18:34 PM, 1/18/2022 5:19:20 AM falls between 1/8/2022 8:22:56 PM and 1/18/2022 5:19:20 AM.
IsRegressionStdDev: Marked as regression because -32.38804485194529 (T) = (0 -24850.402096972426) / Math.Sqrt((15720.062709109392 / (32)) + (83195.71292542249 / (33))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (33) - 2, .025) and -0.07704352403223026 = (23072.792828220732 - 24850.402096972426) / 23072.792828220732 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.CtorFromCollectionNonGeneric<Int32>.Stack(Size: 512)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 23.796464449949188 > 22.789753051162776.
IsChangePoint: Marked as a change because one of 1/11/2022 2:18:34 PM, 1/18/2022 5:19:20 AM falls between 1/8/2022 8:22:56 PM and 1/18/2022 5:19:20 AM.
IsRegressionStdDev: Marked as regression because -19.54017533902972 (T) = (0 -23993.40668573774) / Math.Sqrt((29843.46554165946 / (32)) + (399530.12372097117 / (33))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (33) - 2, .025) and -0.10253203806089282 = (21762.094757751234 - 23993.40668573774) / 21762.094757751234 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | fe8422cdff450941bc3ba4b120576e8822459df4 |
| Compare | ef7ff071099d185a5f66b55966866fcc964e287c |
| Diff | Diff |
Regressions in System.Tests.Perf_GC<Byte>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| NewOperator_Array - Duration of single invocation | 33.53 ns | 35.75 ns | 1.07 | 0.06 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_GC<Byte>*'Payloads
Histogram
System.Tests.Perf_GC<Byte>.NewOperator_Array(length: 1000)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.75326767840259 > 35.15473559389209.
IsChangePoint: Marked as a change because one of 1/11/2022 2:18:34 PM, 1/18/2022 5:19:20 AM falls between 1/8/2022 8:22:56 PM and 1/18/2022 5:19:20 AM.
IsRegressionStdDev: Marked as regression because -17.408231406640766 (T) = (0 -35.88087983769851) / Math.Sqrt((0.4308079323728019 / (32)) + (0.2601888585838306 / (33))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (33) - 2, .025) and -0.07629459721512617 = (33.3374151747477 - 35.88087983769851) / 33.3374151747477 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | fe8422cdff450941bc3ba4b120576e8822459df4 |
| Compare | ef7ff071099d185a5f66b55966866fcc964e287c |
| Diff | Diff |
Regressions in System.Diagnostics.Perf_Activity
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ActivityAllocations - Duration of single invocation | 157.30 ns | 175.74 ns | 1.12 | 0.03 | True | |||||
| EnumerateActivityLinkTagsLarge - Duration of single invocation | 9.47 μs | 10.39 μs | 1.10 | 0.04 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Diagnostics.Perf_Activity*'Payloads
Histogram
System.Diagnostics.Perf_Activity.ActivityAllocations(idFormat: Hierarchical)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 175.74116709608057 > 164.34910749867822.
IsChangePoint: Marked as a change because one of 1/11/2022 2:18:34 PM, 1/18/2022 5:19:20 AM falls between 1/8/2022 8:22:56 PM and 1/18/2022 5:19:20 AM.
IsRegressionStdDev: Marked as regression because -31.019760794326274 (T) = (0 -179.71304496587595) / Math.Sqrt((7.147932236036577 / (32)) + (9.979676620655603 / (33))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (33) - 2, .025) and -0.14306553699152283 = (157.22024604019597 - 179.71304496587595) / 157.22024604019597 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
```#### System.Diagnostics.Perf_Activity.EnumerateActivityLinkTagsLarge
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.392829875135723 > 9.942452779283839.
IsChangePoint: Marked as a change because one of 12/16/2021 8:45:41 PM, 1/12/2022 5:25:46 AM, 1/18/2022 5:19:20 AM falls between 1/8/2022 8:22:56 PM and 1/18/2022 5:19:20 AM.
IsRegressionStdDev: Marked as regression because -9.311624945530044 (T) = (0 -9863.282544360974) / Math.Sqrt((17788.074286222753 / (34)) + (92154.89737234468 / (31))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (31) - 2, .025) and -0.059119325068669744 = (9312.720777445424 - 9863.282544360974) / 9312.720777445424 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

_1.png)
_2.png)
_1.png)

