Regressions in System.Text.Perf_Ascii

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	9ea317f17d1ef5988921f611f80834d15a3e8070
Compare	0224f86a886e0aaf72acb6a4f5573236ba929292
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Perf_Ascii

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
Equals_Chars - Duration of single invocation	5.00 ns	6.95 ns	1.39	0.09	False
EqualsIgnoreCase_DifferentCase_Chars - Duration of single invocation	7.45 ns	9.12 ns	1.22	0.06	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Perf_Ascii*'

Payloads

Baseline
Compare

System.Text.Perf_Ascii.Equals_Chars(Size: 6)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.954001896465551 > 5.5092610127252915.
IsChangePoint: Marked as a change because one of 5/11/2023 1:19:27 PM, 7/6/2023 4:13:54 AM, 7/10/2023 4:47:56 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 4:47:56 PM.
IsRegressionStdDev: Marked as regression because -79.60986360261955 (T) = (0 -6.966106108776006) / Math.Sqrt((0.014756468692197986 / (29)) + (0.0001741369565574083 / (12))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (12) - 2, .025) and -0.35398946283682264 = (5.144874683279225 - 6.966106108776006) / 5.144874683279225 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Ascii.EqualsIgnoreCase_DifferentCase_Chars(Size: 6)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.116938381326122 > 7.946328539515318.
IsChangePoint: Marked as a change because one of 5/11/2023 1:19:27 PM, 6/5/2023 10:43:22 AM, 6/9/2023 2:21:24 AM, 6/22/2023 7:55:14 AM, 7/5/2023 3:16:33 PM, 7/10/2023 4:47:56 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 4:47:56 PM.
IsRegressionStdDev: Marked as regression because -166.45439924629133 (T) = (0 -9.121758824651808) / Math.Sqrt((0.0025314916021539464 / (26)) + (9.334391391871753E-06 / (15))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (15) - 2, .025) and -0.22045718440682033 = (7.47405066002808 - 9.121758824651808) / 7.47405066002808 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	9ea317f17d1ef5988921f611f80834d15a3e8070
Compare	0224f86a886e0aaf72acb6a4f5573236ba929292
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
IsMatch - Duration of single invocation	96.20 ns	117.56 ns	1.22	0.21	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple*'

Payloads

Baseline
Compare

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 6, Options: None)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 117.56298837651069 > 116.13514929069757.
IsChangePoint: Marked as a change because one of 6/2/2023 6:53:36 PM, 7/6/2023 4:13:54 AM, 7/10/2023 4:47:56 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 4:47:56 PM.
IsRegressionStdDev: Marked as regression because -5.188070999352076 (T) = (0 -114.29915709348228) / Math.Sqrt((64.06502654185418 / (29)) + (47.017069984060534 / (12))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (12) - 2, .025) and -0.12657732387725548 = (101.45700137129298 - 114.29915709348228) / 101.45700137129298 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[cincuranet]

Probably caused by #87141. cc @BrennanConroy

[BrennanConroy]

The Equals_Chars tests shouldn't be slower, if anything they should be slightly faster due to one less vector or operation. The code is basically the same unless the JIT is producing worse byte code due to moving the comparison into a method even if it's supposed to be inlined.

Tagging subscribers to this area: @dotnet/area-system-text-encoding
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	9ea317f17d1ef5988921f611f80834d15a3e8070
Compare	0224f86a886e0aaf72acb6a4f5573236ba929292
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Perf_Ascii

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
Equals_Chars - Duration of single invocation	5.00 ns	6.95 ns	1.39	0.09	False
EqualsIgnoreCase_DifferentCase_Chars - Duration of single invocation	7.45 ns	9.12 ns	1.22	0.06	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Perf_Ascii*'

Payloads

Baseline
Compare

System.Text.Perf_Ascii.Equals_Chars(Size: 6)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.954001896465551 > 5.5092610127252915.
IsChangePoint: Marked as a change because one of 5/11/2023 1:19:27 PM, 7/6/2023 4:13:54 AM, 7/10/2023 4:47:56 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 4:47:56 PM.
IsRegressionStdDev: Marked as regression because -79.60986360261955 (T) = (0 -6.966106108776006) / Math.Sqrt((0.014756468692197986 / (29)) + (0.0001741369565574083 / (12))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (12) - 2, .025) and -0.35398946283682264 = (5.144874683279225 - 6.966106108776006) / 5.144874683279225 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Ascii.EqualsIgnoreCase_DifferentCase_Chars(Size: 6)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.116938381326122 > 7.946328539515318.
IsChangePoint: Marked as a change because one of 5/11/2023 1:19:27 PM, 6/5/2023 10:43:22 AM, 6/9/2023 2:21:24 AM, 6/22/2023 7:55:14 AM, 7/5/2023 3:16:33 PM, 7/10/2023 4:47:56 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 4:47:56 PM.
IsRegressionStdDev: Marked as regression because -166.45439924629133 (T) = (0 -9.121758824651808) / Math.Sqrt((0.0025314916021539464 / (26)) + (9.334391391871753E-06 / (15))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (15) - 2, .025) and -0.22045718440682033 = (7.47405066002808 - 9.121758824651808) / 7.47405066002808 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	9ea317f17d1ef5988921f611f80834d15a3e8070
Compare	0224f86a886e0aaf72acb6a4f5573236ba929292
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
IsMatch - Duration of single invocation	96.20 ns	117.56 ns	1.22	0.21	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple*'

Payloads

Baseline
Compare

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 6, Options: None)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 117.56298837651069 > 116.13514929069757.
IsChangePoint: Marked as a change because one of 6/2/2023 6:53:36 PM, 7/6/2023 4:13:54 AM, 7/10/2023 4:47:56 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 4:47:56 PM.
IsRegressionStdDev: Marked as regression because -5.188070999352076 (T) = (0 -114.29915709348228) / Math.Sqrt((64.06502654185418 / (29)) + (47.017069984060534 / (12))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (12) - 2, .025) and -0.12657732387725548 = (101.45700137129298 - 114.29915709348228) / 101.45700137129298 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Author:	performanceautofiler[bot]
Assignees:	-
Labels:	area-System.Text.Encoding, os-windows, arch-x64, untriaged, runtime-coreclr, needs-area-label
Milestone:	-