[Perf] Windows/x64: 67 Improvements on 4/25/2023 9:59:52 PM #17240
Description
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.18362 |
| Queue | TigerWindows |
| Baseline | 2e0033cf25cee5457ee326dc1eaff013b939d00d |
| Compare | 9a7db5556f4ad892c611b2e174594db17fdec3af |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Tests.Perf_Single
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ToStringWithFormat - Duration of single invocation | 171.57 ns | 157.65 ns | 0.92 | 0.04 | False | 1392.205157925239 | 1340.295467031987 | 0.9627140507289806) | Trace | Trace |
| ToStringWithFormat - Duration of single invocation | 229.17 ns | 198.48 ns | 0.87 | 0.01 | False | 1826.8876886196574 | 1721.0768226409766 | 0.9420813514493448) | Trace | Trace |
| ToStringWithFormat - Duration of single invocation | 190.76 ns | 175.73 ns | 0.92 | 0.02 | False | 1782.6302740573574 | 1729.0948901048685 | 0.9699683188759946) | Trace | Trace |
| ToString - Duration of single invocation | 158.26 ns | 146.57 ns | 0.93 | 0.03 | False | Trace | Trace | |||
| ToStringWithCultureInfo - Duration of single invocation | 150.19 ns | 140.36 ns | 0.93 | 0.00 | False | 1151.083880260902 | 1122.0988640995765 | 0.9748193709786328) | Trace | Trace |
| ToStringWithFormat - Duration of single invocation | 201.32 ns | 186.10 ns | 0.92 | 0.01 | False | Trace | Trace |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Single*'Payloads
Histogram
System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "E")
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 157.65063027618126 < 162.5504298889227.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 6.47988110526923 (T) = (0 -159.2942002451505) / Math.Sqrt((7.509362883716798 / (299)) + (131.22947500349196 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.08249515290594363 = (173.61673973676648 - 159.2942002451505) / 173.61673973676648 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "G17")
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 198.4755990829964 < 218.92631865581163.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 8:51:41 AM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 21.62262921775599 (T) = (0 -197.66655000953887) / Math.Sqrt((71.48203218713009 / (299)) + (1.510380959181054 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.05608257778856458 = (209.41085031192696 - 197.66655000953887) / 209.41085031192696 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "G17")
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 175.73385044573823 < 182.28969691686294.
IsChangePoint: Marked as a change because one of 2/21/2023 5:46:41 AM, 3/25/2023 12:18:49 AM, 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 52.80767066881396 (T) = (0 -177.21521535569994) / Math.Sqrt((16.774291480251527 / (299)) + (1.609707887711128 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.09203818127132 = (195.17914927726267 - 177.21521535569994) / 195.17914927726267 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Single.ToString(value: 12345)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 146.5686300820553 < 148.79502451876598.
IsChangePoint: Marked as a change because one of 3/25/2023 12:18:49 AM, 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 34.65211474651841 (T) = (0 -146.7161053817818) / Math.Sqrt((7.865754507138263 / (299)) + (1.0721481434255364 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.057212456586949456 = (155.61947801160446 - 146.7161053817818) / 155.61947801160446 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Single.ToStringWithCultureInfo(value: 12345, culture: zh)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 140.35705990098356 < 143.29583176546072.
IsChangePoint: Marked as a change because one of 4/17/2023 1:30:24 AM, 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 76.70926661633406 (T) = (0 -140.12061689444283) / Math.Sqrt((2.275586289399877 / (299)) + (0.22844456373423883 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.06489835831826742 = (149.84533300833806 - 140.12061689444283) / 149.84533300833806 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "E")
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 186.104632076472 < 192.89049591352736.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 50.892514761368574 (T) = (0 -186.16886421607762) / Math.Sqrt((5.538392038247715 / (299)) + (2.8675885943081756 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.08804520256624876 = (204.14264472313587 - 186.16886421607762) / 204.14264472313587 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository