Skip to content

[Perf] Windows/x64: 13 Improvements on 9/9/2022 1:01:45 PM #8539

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline bc183b8208b7a2b3310715ad2181d6a4720a159c
Compare 071fe7ce868a63113143b949c3c4185f9e5578cf
Diff Diff

Improvements in System.Collections.TryGetValueFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedList - Duration of single invocation 21.23 μs 19.01 μs 0.90 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryGetValueFalse&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryGetValueFalse<Int32, Int32>.SortedList(Size: 512)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.011199062754688 < 19.15450332831996.
IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 8/11/2022 4:03:37 AM, 8/16/2022 10:54:40 PM, 9/9/2022 3:12:19 AM, 9/13/2022 2:30:06 AM falls between 9/4/2022 8:45:57 AM and 9/13/2022 2:30:06 AM.
IsImprovementStdDev: Marked as improvement because 50.601880400876894 (T) = (0 -19026.172544772657) / Math.Sqrt((478617.8604773148 / (299)) + (7189.797678139116 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.10685739454683411 = (21302.502454374673 - 19026.172544772657) / 21302.502454374673 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.19042
Baseline bc183b8208b7a2b3310715ad2181d6a4720a159c
Compare 071fe7ce868a63113143b949c3c4185f9e5578cf
Diff Diff

Improvements in System.Tests.Perf_Version

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToStringL - Duration of single invocation 92.19 ns 76.67 ns 0.83 0.17 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Version*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Version.ToStringL


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 76.67134893692442 < 82.13801481369018.
IsChangePoint: Marked as a change because one of 7/10/2022 1:04:01 AM, 7/13/2022 4:52:15 AM, 7/23/2022 3:48:41 PM, 8/15/2022 4:29:54 PM, 8/29/2022 8:26:03 AM, 9/9/2022 3:12:19 AM, 9/13/2022 2:30:06 AM falls between 9/4/2022 8:45:57 AM and 9/13/2022 2:30:06 AM.
IsImprovementStdDev: Marked as improvement because 18.668931537978967 (T) = (0 -76.60023847080453) / Math.Sqrt((42.083237133683745 / (299)) + (1.3986353678667582 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.10321629282021999 = (85.41662594618072 - 76.60023847080453) / 85.41662594618072 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline bc183b8208b7a2b3310715ad2181d6a4720a159c
Compare 071fe7ce868a63113143b949c3c4185f9e5578cf
Diff Diff

Improvements in System.Collections.CreateAddAndClear<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IDictionary - Duration of single invocation 17.18 μs 15.80 μs 0.92 0.25 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CreateAddAndClear&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CreateAddAndClear<String>.IDictionary(Size: 512)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.801989717505856 < 16.297757257298223.
IsChangePoint: Marked as a change because one of 9/6/2022 11:34:40 PM, 9/9/2022 3:12:19 AM, 9/13/2022 2:30:06 AM falls between 9/4/2022 8:45:57 AM and 9/13/2022 2:30:06 AM.
IsImprovementStdDev: Marked as improvement because 22.292137311066625 (T) = (0 -15786.23242126952) / Math.Sqrt((585922.2734468534 / (299)) + (9185.500080119871 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.06594892889400535 = (16900.823637595426 - 15786.23242126952) / 16900.823637595426 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline bc183b8208b7a2b3310715ad2181d6a4720a159c
Compare 071fe7ce868a63113143b949c3c4185f9e5578cf
Diff Diff

Improvements in System.Text.Json.Reader.Tests.Perf_Base64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadBase64EncodedByteArray_HeavyEscaping - Duration of single invocation 114.51 ns 95.61 ns 0.83 0.11 False
ReadBase64EncodedByteArray_NoEscaping - Duration of single invocation 114.54 ns 96.26 ns 0.84 0.12 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Reader.Tests.Perf_Base64*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Reader.Tests.Perf_Base64.ReadBase64EncodedByteArray_HeavyEscaping(NumberOfBytes: 100)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 95.61326030630984 < 116.75628396158835.
IsChangePoint: Marked as a change because one of 7/24/2022 8:36:55 PM, 7/28/2022 3:42:13 AM, 8/15/2022 4:29:54 PM, 9/9/2022 3:12:19 AM, 9/13/2022 2:30:06 AM falls between 9/4/2022 8:45:57 AM and 9/13/2022 2:30:06 AM.
IsImprovementStdDev: Marked as improvement because 9.081013280632105 (T) = (0 -93.01769224800734) / Math.Sqrt((110.16809786547843 / (299)) + (27.68495409721262 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.12123513436851815 = (105.85049071251235 - 93.01769224800734) / 105.85049071251235 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Reader.Tests.Perf_Base64.ReadBase64EncodedByteArray_NoEscaping(NumberOfBytes: 100)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 96.25900019262957 < 116.69562581526124.
IsChangePoint: Marked as a change because one of 7/24/2022 8:36:55 PM, 7/28/2022 3:42:13 AM, 8/15/2022 4:29:54 PM, 9/9/2022 3:12:19 AM, 9/13/2022 2:30:06 AM falls between 9/4/2022 8:45:57 AM and 9/13/2022 2:30:06 AM.
IsImprovementStdDev: Marked as improvement because 9.260218280123029 (T) = (0 -93.96348517155158) / Math.Sqrt((113.10288855630493 / (299)) + (19.44230715965226 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.10840022942729988 = (105.38751609502562 - 93.96348517155158) / 105.38751609502562 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline bc183b8208b7a2b3310715ad2181d6a4720a159c
Compare 071fe7ce868a63113143b949c3c4185f9e5578cf
Diff Diff

Improvements in System.Reflection.Invoke

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Ctor0_NoParams - Duration of single invocation 17.24 ns 14.73 ns 0.85 0.03 False
Property_Get_int - Duration of single invocation 16.63 ns 14.86 ns 0.89 0.04 False
Method0_NoParms - Duration of single invocation 14.02 ns 12.48 ns 0.89 0.01 False
Property_Get_class - Duration of single invocation 14.23 ns 12.53 ns 0.88 0.03 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Reflection.Invoke*'

Payloads

Baseline
Compare

Histogram

System.Reflection.Invoke.Ctor0_NoParams


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.728812047536394 < 15.377410933406189.
IsChangePoint: Marked as a change because one of 9/9/2022 3:12:19 AM, 9/13/2022 2:30:06 AM falls between 9/4/2022 8:45:57 AM and 9/13/2022 2:30:06 AM.
IsImprovementStdDev: Marked as improvement because 60.23124375058947 (T) = (0 -14.719479032193147) / Math.Sqrt((0.15996836369612205 / (100)) + (0.00021184445444024714 / (17))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (100) + (17) - 2, .975) and 0.1411131275393859 = (17.13785540815579 - 14.719479032193147) / 17.13785540815579 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Invoke.Property_Get_int

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.860228471792867 < 15.095689781334727.
IsChangePoint: Marked as a change because one of 9/9/2022 3:12:19 AM, 9/13/2022 2:30:06 AM falls between 9/4/2022 8:45:57 AM and 9/13/2022 2:30:06 AM.
IsImprovementStdDev: Marked as improvement because 47.08464852171177 (T) = (0 -14.866313406302737) / Math.Sqrt((0.125317251900912 / (100)) + (0.0002380073795787119 / (17))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (100) + (17) - 2, .975) and 0.1013208158892442 = (16.542403194765186 - 14.866313406302737) / 16.542403194765186 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Invoke.Method0_NoParms

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.482493141074713 < 13.27570420185975.
IsChangePoint: Marked as a change because one of 9/9/2022 3:12:19 AM, 9/13/2022 2:30:06 AM falls between 9/4/2022 8:45:57 AM and 9/13/2022 2:30:06 AM.
IsImprovementStdDev: Marked as improvement because 146.54362600561302 (T) = (0 -12.47978616474777) / Math.Sqrt((0.009492458307316208 / (101)) + (1.425891163804635E-05 / (17))) is greater than 1.9806260024587088 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (101) + (17) - 2, .975) and 0.10261197336800842 = (13.906789253235251 - 12.47978616474777) / 13.906789253235251 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Invoke.Property_Get_class

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.530255697923742 < 12.560022892858843.
IsChangePoint: Marked as a change because one of 9/9/2022 3:12:19 AM, 9/13/2022 2:30:06 AM falls between 9/4/2022 8:45:57 AM and 9/13/2022 2:30:06 AM.
IsImprovementStdDev: Marked as improvement because 42.32863420584271 (T) = (0 -12.533131893765832) / Math.Sqrt((0.13539502362593275 / (99)) + (1.8113091996962542E-05 / (17))) is greater than 1.9809922979755963 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (99) + (17) - 2, .975) and 0.11106961298875188 = (14.099115157829837 - 12.533131893765832) / 14.099115157829837 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline bc183b8208b7a2b3310715ad2181d6a4720a159c
Compare 071fe7ce868a63113143b949c3c4185f9e5578cf
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Email_IsNotMatch - Duration of single invocation 136.10 ns 123.49 ns 0.91 0.04 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsNotMatch(Options: Compiled)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 123.4875463954128 < 126.75401072114617.
IsChangePoint: Marked as a change because one of 7/13/2022 7:22:00 AM, 7/21/2022 3:55:55 PM, 8/5/2022 10:45:33 AM, 8/10/2022 5:14:51 PM, 8/15/2022 4:29:54 PM, 8/30/2022 3:47:49 PM, 9/9/2022 3:12:19 AM, 9/13/2022 2:30:06 AM falls between 9/4/2022 8:45:57 AM and 9/13/2022 2:30:06 AM.
IsImprovementStdDev: Marked as improvement because 21.473305695380397 (T) = (0 -123.9834606800752) / Math.Sqrt((46.39008027146844 / (299)) + (0.16487207885782812 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.06569992345123363 = (132.7019699474506 - 123.9834606800752) / 132.7019699474506 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline bc183b8208b7a2b3310715ad2181d6a4720a159c
Compare 071fe7ce868a63113143b949c3c4185f9e5578cf
Diff Diff

Improvements in System.Collections.IterateFor<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableList - Duration of single invocation 6.58 μs 5.97 μs 0.91 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateFor&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IterateFor<String>.ImmutableList(Size: 512)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.965743222808754 < 6.053269730347538.
IsChangePoint: Marked as a change because one of 7/13/2022 7:22:00 AM, 7/28/2022 6:38:38 AM, 8/5/2022 10:45:33 AM, 9/9/2022 3:12:19 AM, 9/13/2022 2:30:06 AM falls between 9/4/2022 8:45:57 AM and 9/13/2022 2:30:06 AM.
IsImprovementStdDev: Marked as improvement because 14.160191481197083 (T) = (0 -6014.0024593301805) / Math.Sqrt((104568.71479688845 / (299)) + (3792.291019008732 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.053345766493851796 = (6352.902935907191 - 6014.0024593301805) / 6352.902935907191 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline bc183b8208b7a2b3310715ad2181d6a4720a159c
Compare 071fe7ce868a63113143b949c3c4185f9e5578cf
Diff Diff

Improvements in System.Text.Json.Tests.Perf_Reader

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadReturnBytes - Duration of single invocation 112.11 ns 102.31 ns 0.91 0.20 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Reader*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: True, TestCase: HelloWorld)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 102.30795945272953 < 106.39237967436303.
IsChangePoint: Marked as a change because one of 8/17/2022 6:09:21 PM, 9/9/2022 3:12:19 AM, 9/13/2022 2:30:06 AM falls between 9/4/2022 8:45:57 AM and 9/13/2022 2:30:06 AM.
IsImprovementStdDev: Marked as improvement because 20.442490346278092 (T) = (0 -100.78780776818476) / Math.Sqrt((16.358934311227053 / (299)) + (0.8086522230717035 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.06091522052923728 = (107.32556843801201 - 100.78780776818476) / 107.32556843801201 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline bc183b8208b7a2b3310715ad2181d6a4720a159c
Compare 071fe7ce868a63113143b949c3c4185f9e5578cf
Diff Diff

Improvements in System.Text.Json.Tests.Perf_Deep

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteDeepUtf16 - Duration of single invocation 5.71 ms 4.37 ms 0.77 0.40 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Deep*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Deep.WriteDeepUtf16(Formatted: False, SkipValidation: False)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.372494666666667 < 5.3483204574136005.
IsChangePoint: Marked as a change because one of 8/1/2022 12:35:11 AM, 9/9/2022 3:12:19 AM, 9/13/2022 2:30:06 AM falls between 9/4/2022 8:45:57 AM and 9/13/2022 2:30:06 AM.
IsImprovementStdDev: Marked as improvement because 17.41135749396537 (T) = (0 -4370570.000638987) / Math.Sqrt((364934821563.373 / (299)) + (1322298369.8807297 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.1255248124618388 = (4997934.833283374 - 4370570.000638987) / 4997934.833283374 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions