-
Notifications
You must be signed in to change notification settings - Fork 4
Closed
Description
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | Windows 10.0.19041 |
| Queue | SurfaceWindows |
| Baseline | 2f49fcff6df15a200ef01eea16b3ce7930f75c5c |
| Compare | 09399050c82b0ff2e9b082c36524f39a1e3ad6c1 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Text.Encodings.Web.Tests.Perf_Encoders
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| EncodeUtf8 - Duration of single invocation | 119.52 ns | 129.46 ns | 1.08 | 0.02 | False | |||||
| EncodeUtf8 - Duration of single invocation | 63.78 ns | 84.10 ns | 1.32 | 0.03 | True | |||||
| EncodeUtf16 - Duration of single invocation | 78.62 ns | 84.24 ns | 1.07 | 0.00 | True |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Encodings.Web.Tests.Perf_Encoders*'Payloads
Histogram
System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: Url,&lorem ipsum=dolor sit amet,512)
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 129.4565470391469 > 125.51529355310763.
IsChangePoint: Marked as a change because one of 2/14/2023 11:21:41 AM, 3/10/2023 4:14:40 PM, 4/27/2023 10:53:57 PM, 5/3/2023 7:05:50 AM, 5/19/2023 1:23:34 PM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -7.996212204546845 (T) = (0 -130.21842741095693) / Math.Sqrt((3.105750204626415 / (16)) + (17.729650741744347 / (12))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (12) - 2, .025) and -0.08623838832397639 = (119.88015596822987 - 130.21842741095693) / 119.88015596822987 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: Url,&lorem ipsum=dolor sit amet,16)
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 84.10347865610282 > 67.02422781170542.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -19.543606332067146 (T) = (0 -80.60045150271827) / Math.Sqrt((0.277060122757908 / (15)) + (9.64557109243251 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.2681436349060964 = (63.557825221183705 - 80.60045150271827) / 63.557825221183705 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: Url,&lorem ipsum=dolor sit amet,16)
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 84.23813569002982 > 82.55047654377887.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -37.07668863034435 (T) = (0 -84.15841862230614) / Math.Sqrt((0.002364276164011589 / (15)) + (0.2870552877439589 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.07031892066367323 = (78.62929169758299 - 84.15841862230614) / 78.62929169758299 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | Windows 10.0.19041 |
| Queue | SurfaceWindows |
| Baseline | 2f49fcff6df15a200ef01eea16b3ce7930f75c5c |
| Compare | 09399050c82b0ff2e9b082c36524f39a1e3ad6c1 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Memory.Span<Byte>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| LastIndexOfValue - Duration of single invocation | 13.74 ns | 15.12 ns | 1.10 | 0.05 | False | |||||
| SequenceCompareToDifferent - Duration of single invocation | 2.64 ns | 3.68 ns | 1.39 | 0.49 | False | |||||
| IndexOfValue - Duration of single invocation | 13.49 ns | 14.85 ns | 1.10 | 0.05 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span<Byte>*'Payloads
Histogram
System.Memory.Span<Byte>.LastIndexOfValue(Size: 512)
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.11903849446876 > 14.751069542230661.
IsChangePoint: Marked as a change because one of 3/10/2023 4:14:40 PM, 4/6/2023 11:23:56 PM, 4/12/2023 8:39:35 PM, 4/27/2023 10:53:57 PM, 5/19/2023 1:23:34 PM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -6.206343144304712 (T) = (0 -15.20907025099283) / Math.Sqrt((0.964409563349339 / (16)) + (0.03859755570440446 / (12))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (12) - 2, .025) and -0.11460801505842126 = (13.645218808332057 - 15.20907025099283) / 13.645218808332057 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Memory.Span<Byte>.SequenceCompareToDifferent(Size: 512)
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.6783630148283684 > 2.702172668779725.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -10.019576376673744 (T) = (0 -3.182781883323826) / Math.Sqrt((0.04151202725700269 / (15)) + (0.05929402434081391 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.36891685847099465 = (2.325036662108771 - 3.182781883323826) / 2.325036662108771 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Memory.Span<Byte>.IndexOfValue(Size: 512)
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.846209896167936 > 14.049783107532253.
IsChangePoint: Marked as a change because one of 3/16/2023 2:30:29 AM, 4/3/2023 6:15:03 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -36.28272688819566 (T) = (0 -14.752836388418956) / Math.Sqrt((0.00945310583899296 / (15)) + (0.009503136183282334 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.09979275565177313 = (13.414196731706914 - 14.752836388418956) / 13.414196731706914 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | Windows 10.0.19041 |
| Queue | SurfaceWindows |
| Baseline | 2f49fcff6df15a200ef01eea16b3ce7930f75c5c |
| Compare | 09399050c82b0ff2e9b082c36524f39a1e3ad6c1 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Tests.Perf_Guid
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| GuidToString - Duration of single invocation | 22.81 ns | 33.52 ns | 1.47 | 0.23 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Guid*'Payloads
Histogram
System.Tests.Perf_Guid.GuidToString
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 33.51524056459398 > 23.373238819194494.
IsChangePoint: Marked as a change because one of 2/5/2023 4:54:05 PM, 5/19/2023 1:23:34 PM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -61.48679543066845 (T) = (0 -33.23479392397222) / Math.Sqrt((0.23443180408278638 / (15)) + (0.17472978387708846 / (12))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (12) - 2, .025) and -0.47373739577573476 = (22.551367712616358 - 33.23479392397222) / 22.551367712616358 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository






