-
Notifications
You must be signed in to change notification settings - Fork 4
Description
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | ubuntu 18.04 |
| Queue | TigerUbuntu |
| Baseline | c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8 |
| Compare | 9aefa9daa141bb7d9ba3f2b373d4b050c9b243fe |
| Diff | Diff |
| Configs | AOT:true, CompilationMode:wasm, RunKind:micro |
Regressions in Benchstone.BenchI.BubbleSort
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Test - Duration of single invocation | 13.98 μs | 16.66 μs | 1.19 | 0.00 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.BubbleSort*'Payloads
Histogram
Benchstone.BenchI.BubbleSort.Test
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.658781892561645 > 14.701866402841699.
IsChangePoint: Marked as a change because one of 4/17/2023 1:54:35 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM.
IsRegressionStdDev: Marked as regression because -153.16241279580066 (T) = (0 -16689.517611894484) / Math.Sqrt((618.071970332376 / (34)) + (863.4151245252449 / (3))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (3) - 2, .025) and -0.19122848291582448 = (14010.341299968573 - 16689.517611894484) / 14010.341299968573 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | ubuntu 18.04 |
| Queue | TigerUbuntu |
| Baseline | c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8 |
| Compare | 9aefa9daa141bb7d9ba3f2b373d4b050c9b243fe |
| Diff | Diff |
| Configs | AOT:true, CompilationMode:wasm, RunKind:micro |
Regressions in System.Tests.Perf_Int16
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| TryParse - Duration of single invocation | 32.74 ns | 41.61 ns | 1.27 | 0.63 | False | |||||
| Parse - Duration of single invocation | 34.85 ns | 55.31 ns | 1.59 | 0.64 | False | |||||
| Parse - Duration of single invocation | 29.62 ns | 45.14 ns | 1.52 | 0.64 | False | |||||
| Parse - Duration of single invocation | 32.51 ns | 62.35 ns | 1.92 | 0.60 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int16*'Payloads
Histogram
System.Tests.Perf_Int16.TryParse(value: "32767")
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 41.60801255486559 > 35.383080307333366.
IsChangePoint: Marked as a change because one of 4/17/2023 1:54:35 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM.
IsRegressionStdDev: Marked as regression because -4.867418475011111 (T) = (0 -42.003359650420606) / Math.Sqrt((4.703727716160876 / (33)) + (6.219593841269267 / (3))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (3) - 2, .025) and -0.20845021318753362 = (34.75803900901154 - 42.003359650420606) / 34.75803900901154 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
System.Tests.Perf_Int16.Parse(value: "-32768")
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 55.310154545609116 > 36.84920275796996.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM.
IsRegressionStdDev: Marked as regression because -4.94434288066354 (T) = (0 -60.040635983106384) / Math.Sqrt((9.224378537163398 / (35)) + (44.7549092610125 / (2))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (2) - 2, .025) and -0.6443100118549693 = (36.514182575202895 - 60.040635983106384) / 36.514182575202895 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
System.Tests.Perf_Int16.Parse(value: "0")
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 45.137692037892045 > 31.175075851605392.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM.
IsRegressionStdDev: Marked as regression because -16.136560146275635 (T) = (0 -45.962324640048685) / Math.Sqrt((5.087574564505079 / (34)) + (1.3600378570792637 / (2))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (2) - 2, .025) and -0.47012223074977505 = (31.26428787938776 - 45.962324640048685) / 31.26428787938776 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
System.Tests.Perf_Int16.Parse(value: "32767")
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 62.349159704857215 > 35.05085057111022.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM.
IsRegressionStdDev: Marked as regression because -14.244584051658638 (T) = (0 -60.5671271903529) / Math.Sqrt((3.874091858800674 / (34)) + (6.351279765501148 / (2))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (2) - 2, .025) and -0.7438709986317713 = (34.73142637148816 - 60.5671271903529) / 34.73142637148816 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | ubuntu 18.04 |
| Queue | TigerUbuntu |
| Baseline | c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8 |
| Compare | 9aefa9daa141bb7d9ba3f2b373d4b050c9b243fe |
| Diff | Diff |
| Configs | AOT:true, CompilationMode:wasm, RunKind:micro |
Regressions in System.MathBenchmarks.Double
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Max - Duration of single invocation | 4.37 μs | 4.72 μs | 1.08 | 0.00 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.MathBenchmarks.Double*'Payloads
Histogram
System.MathBenchmarks.Double.Max
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.7246419905144075 > 4.590642361590055.
IsChangePoint: Marked as a change because one of 4/17/2023 1:54:35 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM.
IsRegressionStdDev: Marked as regression because -60.99946862483759 (T) = (0 -4727.158473363669) / Math.Sqrt((843.8994465338607 / (34)) + (18.170442292055185 / (3))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (3) - 2, .025) and -0.07724313445729859 = (4388.200139929554 - 4727.158473363669) / 4388.200139929554 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | ubuntu 18.04 |
| Queue | TigerUbuntu |
| Baseline | c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8 |
| Compare | 9aefa9daa141bb7d9ba3f2b373d4b050c9b243fe |
| Diff | Diff |
| Configs | AOT:true, CompilationMode:wasm, RunKind:micro |
Regressions in System.Collections.IndexerSetReverse<Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| IList - Duration of single invocation | 5.81 μs | 6.58 μs | 1.13 | 0.38 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IndexerSetReverse<Int32>*'Payloads
Histogram
System.Collections.IndexerSetReverse<Int32>.IList(Size: 512)
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.575333374423472 > 6.14265784724513.
IsChangePoint: Marked as a change because one of 4/17/2023 1:54:35 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM.
IsRegressionStdDev: Marked as regression because -6.16439574384889 (T) = (0 -6792.449780814076) / Math.Sqrt((99031.42270704478 / (33)) + (36915.49467640371 / (3))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (3) - 2, .025) and -0.12647941604711804 = (6029.803726595536 - 6792.449780814076) / 6029.803726595536 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | ubuntu 18.04 |
| Queue | TigerUbuntu |
| Baseline | c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8 |
| Compare | 9aefa9daa141bb7d9ba3f2b373d4b050c9b243fe |
| Diff | Diff |
| Configs | AOT:true, CompilationMode:wasm, RunKind:micro |
Regressions in System.Tests.Perf_Version
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Parse3 - Duration of single invocation | 161.51 ns | 176.86 ns | 1.10 | 0.27 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Version*'Payloads
Histogram
System.Tests.Perf_Version.Parse3
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 176.86332079447192 > 171.45890788116245.
IsChangePoint: Marked as a change because one of 4/17/2023 1:54:35 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM.
IsRegressionStdDev: Marked as regression because -2.0718539292452953 (T) = (0 -187.8179065847211) / Math.Sqrt((46.00785749593048 / (34)) + (273.5937390357462 / (3))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (3) - 2, .025) and -0.11872318359622708 = (167.88595189469936 - 187.8179065847211) / 167.88595189469936 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | ubuntu 18.04 |
| Queue | TigerUbuntu |
| Baseline | c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8 |
| Compare | 9aefa9daa141bb7d9ba3f2b373d4b050c9b243fe |
| Diff | Diff |
| Configs | AOT:true, CompilationMode:wasm, RunKind:micro |
Regressions in System.Tests.Perf_Byte
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Parse - Duration of single invocation | 31.81 ns | 48.44 ns | 1.52 | 0.69 | False | |||||
| Parse - Duration of single invocation | 38.14 ns | 49.27 ns | 1.29 | 0.67 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Byte*'Payloads
Histogram
System.Tests.Perf_Byte.Parse(value: "0")
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 48.442604495652844 > 32.13360356781855.
IsChangePoint: Marked as a change because one of 2/16/2023 3:28:45 AM, 4/17/2023 1:54:35 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM.
IsRegressionStdDev: Marked as regression because -19.97950445458046 (T) = (0 -49.11717427390863) / Math.Sqrt((9.057586314823975 / (34)) + (1.5246280098980298 / (3))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (3) - 2, .025) and -0.5576515247569722 = (31.532838695466232 - 49.11717427390863) / 31.532838695466232 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
System.Tests.Perf_Byte.Parse(value: "255")
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 49.27370689275745 > 34.63949147109729.
IsChangePoint: Marked as a change because one of 2/14/2023 9:29:25 PM, 4/17/2023 6:40:18 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM.
IsRegressionStdDev: Marked as regression because -21.644043043609884 (T) = (0 -48.72163261037674) / Math.Sqrt((8.11892158472017 / (34)) + (0.6095720265323639 / (2))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (2) - 2, .025) and -0.48704907541745585 = (32.763970884215254 - 48.72163261037674) / 32.763970884215254 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository









