Skip to content

[Perf] Windows/arm64: 15 Improvements on 5/1/2023 10:26:58 PM #17385

@performanceautofiler

Description

@performanceautofiler

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 0c419b05455970ea4b009c14984785b59ba7b624
Compare 3e8f17a65a068fca3d19fa5cd43a7e1cd414a5ae
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 855.29 μs 237.11 μs 0.28 0.00 True
Count - Duration of single invocation 220.21 μs 165.54 μs 0.75 0.00 True
Count - Duration of single invocation 1.82 ms 1.14 ms 0.62 0.00 True
Count - Duration of single invocation 213.75 μs 161.44 μs 0.76 0.00 True
Count - Duration of single invocation 182.57 μs 153.25 μs 0.84 0.00 True
Count - Duration of single invocation 1.13 ms 711.89 μs 0.63 0.00 True
Count - Duration of single invocation 2.00 ms 1.33 ms 0.67 0.00 True
Count - Duration of single invocation 676.65 μs 200.57 μs 0.30 0.00 True
Count - Duration of single invocation 213.41 μs 163.95 μs 0.77 0.00 True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Holmes", Options: None)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 237.10953914141413 < 811.3558482142857.
IsChangePoint: Marked as a change because one of 5/1/2023 3:45:46 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsImprovementStdDev: Marked as improvement because 1901.3588442961814 (T) = (0 -237299.34127132792) / Math.Sqrt((16222527.83631918 / (299)) + (304693.06151680514 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.7219762583077299 = (853521.8604962957 - 237299.34127132792) / 853521.8604962957 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock", Options: None)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 165.53600202429152 < 208.9413254736754.
IsChangePoint: Marked as a change because one of 5/1/2023 3:45:46 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsImprovementStdDev: Marked as improvement because 407.0915307633796 (T) = (0 -165285.2475138166) / Math.Sqrt((3690029.2480639163 / (299)) + (28890.10533983758 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.24391555323683953 = (218606.86094180608 - 165285.2475138166) / 218606.86094180608 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)the", Options: None)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.135046188186813 < 1.7269367894345233.
IsChangePoint: Marked as a change because one of 5/1/2023 3:45:46 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsImprovementStdDev: Marked as improvement because 781.083294880858 (T) = (0 -1134772.1227705826) / Math.Sqrt((126697335.81550348 / (299)) + (2033794.94498083 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.3754399826713444 = (1816914.4538329346 - 1134772.1227705826) / 1816914.4538329346 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock Holmes", Options: NonBacktracking)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 161.4389103803998 < 204.33944717799938.
IsChangePoint: Marked as a change because one of 5/1/2023 3:45:46 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsImprovementStdDev: Marked as improvement because 402.7052418992018 (T) = (0 -161237.0266045147) / Math.Sqrt((2966034.5061215213 / (299)) + (51532.61051843334 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.2536138861124047 = (216023.61512957723 - 161237.0266045147) / 216023.61512957723 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock", Options: Compiled)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 153.24848334655033 < 173.22947700373754.
IsChangePoint: Marked as a change because one of 5/1/2023 3:45:46 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsImprovementStdDev: Marked as improvement because 188.51979960703378 (T) = (0 -153808.8386927425) / Math.Sqrt((2081517.9483867835 / (299)) + (81058.32412490237 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.1492021188816733 = (180781.87793623708 - 153808.8386927425) / 180781.87793623708 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)the", Options: Compiled)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 711.8908653846154 < 1.0675931265943877.
IsChangePoint: Marked as a change because one of 5/1/2023 3:45:46 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsImprovementStdDev: Marked as improvement because 606.9623599426599 (T) = (0 -712834.8653083028) / Math.Sqrt((47771563.9358887 / (299)) + (1741965.1740397322 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.36356428991734496 = (1120042.2195287026 - 712834.8653083028) / 1120042.2195287026 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)the", Options: NonBacktracking)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.3331426799007444 < 1.9008552187667889.
IsChangePoint: Marked as a change because one of 5/1/2023 3:45:46 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsImprovementStdDev: Marked as improvement because 202.48206636252516 (T) = (0 -1336136.2609038793) / Math.Sqrt((207259179.4852225 / (299)) + (59295953.03529495 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.3301293125105121 = (1994618.1940150156 - 1336136.2609038793) / 1994618.1940150156 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Holmes", Options: Compiled)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 200.56568047337277 < 642.8926179800724.
IsChangePoint: Marked as a change because one of 5/1/2023 3:45:46 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsImprovementStdDev: Marked as improvement because 1027.0428330755656 (T) = (0 -200238.84254701808) / Math.Sqrt((48702060.67243294 / (299)) + (268516.80083599716 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.7003454817761504 = (668232.348819231 - 200238.84254701808) / 668232.348819231 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock", Options: NonBacktracking)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 163.95467247596153 < 203.72168853628426.
IsChangePoint: Marked as a change because one of 5/1/2023 3:45:46 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsImprovementStdDev: Marked as improvement because 404.75172214051355 (T) = (0 -163973.74136337437) / Math.Sqrt((3580375.2815040764 / (299)) + (20814.37945760195 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.23474416015698754 = (214273.1003490448 - 163973.74136337437) / 214273.1003490448 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 0c419b05455970ea4b009c14984785b59ba7b624
Compare 3e8f17a65a068fca3d19fa5cd43a7e1cd414a5ae
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Numerics.Tests.Perf_BigInteger

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Ctor_ByteArray - Duration of single invocation 142.13 ns 128.13 ns 0.90 0.00 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_BigInteger*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_BigInteger.Ctor_ByteArray(numberString: 12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 128.13155995380765 < 134.09067956378848.
IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 4/1/2023 8:54:51 PM, 5/1/2023 3:45:46 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsImprovementStdDev: Marked as improvement because 17.900802708518235 (T) = (0 -129.74281292499492) / Math.Sqrt((2.745636621246792 / (299)) + (1.4169569293810542 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.06396845015693728 = (138.60944425083525 - 129.74281292499492) / 138.60944425083525 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 0c419b05455970ea4b009c14984785b59ba7b624
Compare 3e8f17a65a068fca3d19fa5cd43a7e1cd414a5ae
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 10.03 ms 5.45 ms 0.54 0.01 True
Count - Duration of single invocation 7.96 ms 5.35 ms 0.67 0.01 True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "(?i)Twain", Options: None)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.452941426282052 < 9.53619893491124.
IsChangePoint: Marked as a change because one of 5/1/2023 3:45:46 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsImprovementStdDev: Marked as improvement because 722.4350674178155 (T) = (0 -5459589.00144994) / Math.Sqrt((4509430984.75421 / (299)) + (153568018.68895915 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.4576821123451683 = (10067137.975217953 - 5459589.00144994) / 10067137.975217953 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "(?i)Twain", Options: Compiled)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.353176488095238 < 7.548909578525641.
IsChangePoint: Marked as a change because one of 5/1/2023 3:45:46 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsImprovementStdDev: Marked as improvement because 569.2836538145324 (T) = (0 -5349629.183709809) / Math.Sqrt((5726372949.224233 / (299)) + (9926322.858041655 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.32678116173983857 = (7946345.051090915 - 5349629.183709809) / 7946345.051090915 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 0c419b05455970ea4b009c14984785b59ba7b624
Compare 3e8f17a65a068fca3d19fa5cd43a7e1cd414a5ae
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 2.88 secs 1.66 secs 0.57 0.00 True
Count - Duration of single invocation 3.40 secs 1.86 secs 0.55 0.00 True
Count - Duration of single invocation 3.43 secs 1.85 secs 0.54 0.00 True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice.Count(Options: IgnoreCase, Compiled)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.6552369333333332 < 2.7326848166666666.
IsChangePoint: Marked as a change because one of 5/1/2023 3:45:46 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsImprovementStdDev: Marked as improvement because 1225.5269413975666 (T) = (0 -1655684938.888889) / Math.Sqrt((228414775163379.16 / (299)) + (1164413483740.7212 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.42011563752243 = (2855198460.28428 - 1655684938.888889) / 2855198460.28428 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice.Count(Options: IgnoreCase)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.8568557333333333 < 3.2327221775.
IsChangePoint: Marked as a change because one of 5/1/2023 3:45:46 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsImprovementStdDev: Marked as improvement because 1068.2502462150906 (T) = (0 -1861193517.777778) / Math.Sqrt((234129614432686.47 / (299)) + (8188556698607.609 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.4568641180880758 = (3426754850.4180617 - 1861193517.777778) / 3426754850.4180617 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice.Count(Options: IgnoreCase, NonBacktracking)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.851283275 < 3.2655566158333333.
IsChangePoint: Marked as a change because one of 5/1/2023 3:45:46 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsImprovementStdDev: Marked as improvement because 1483.6130302971526 (T) = (0 -1850253195.8333333) / Math.Sqrt((181555942369789.44 / (299)) + (3411653623993.07 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.46509201100689057 = (3459012080.4069123 - 1850253195.8333333) / 3459012080.4069123 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions