Skip to content

[Perf] Changes at 7/14/2022 8:11:01 PM #6964

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 0cc170c7532a2f23f2ef393b0eac11a1fb07c36f
Compare 81a3dcdad5235c38a329b372af132a0053dc1708
Diff Diff

Improvements in System.Reflection.Attributes

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetCustomAttributesMethodBaseHitInherit - Duration of single invocation 1.03 μs 892.96 ns 0.87 0.21 False
GetCustomAttributesMethodBaseMissInherit - Duration of single invocation 168.73 ns 128.09 ns 0.76 0.04 False
GetCustomAttributesClassMiss - Duration of single invocation 150.41 ns 102.41 ns 0.68 0.05 False
GetCustomAttributesMethodBaseMiss - Duration of single invocation 144.82 ns 100.88 ns 0.70 0.09 False
GetCustomAttributesMethodBaseHit - Duration of single invocation 985.73 ns 883.62 ns 0.90 0.22 False
GetCustomAttributesClassMissInherit - Duration of single invocation 248.29 ns 207.84 ns 0.84 0.04 False
GetCustomAttributesMethodOverrideMiss - Duration of single invocation 983.65 ns 921.18 ns 0.94 0.20 False

graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Reflection.Attributes*'

Payloads

Baseline
Compare

Histogram

System.Reflection.Attributes.GetCustomAttributesMethodBaseHitInherit


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 892.9573528081199 < 0.9759683322992079.
IsChangePoint: Marked as a change because one of 6/7/2022 6:20:49 PM, 6/22/2022 3:28:08 PM, 7/14/2022 4:09:15 PM, 7/18/2022 2:17:16 AM falls between 7/8/2022 10:31:34 AM and 7/18/2022 2:17:16 AM.
IsImprovementStdDev: Marked as improvement because 8.941558347902744 (T) = (0 -920.621265622382) / Math.Sqrt((1680.5994190030585 / (299)) + (930.1913688442986 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.06421358631401626 = (983.7942207304897 - 920.621265622382) / 983.7942207304897 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Attributes.GetCustomAttributesMethodBaseMissInherit

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 128.0886316196095 < 160.66390923307156.
IsChangePoint: Marked as a change because one of 6/27/2022 8:54:23 AM, 7/14/2022 4:09:15 PM, 7/18/2022 2:17:16 AM falls between 7/8/2022 10:31:34 AM and 7/18/2022 2:17:16 AM.
IsImprovementStdDev: Marked as improvement because 59.70145721447511 (T) = (0 -127.22912544404474) / Math.Sqrt((17.76564765044706 / (299)) + (11.663013057301807 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.26896824820972803 = (174.04049158256797 - 127.22912544404474) / 174.04049158256797 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Attributes.GetCustomAttributesClassMiss

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 102.40596863157394 < 142.72229472544515.
IsChangePoint: Marked as a change because one of 7/14/2022 4:09:15 PM, 7/18/2022 2:17:16 AM falls between 7/8/2022 10:31:34 AM and 7/18/2022 2:17:16 AM.
IsImprovementStdDev: Marked as improvement because 113.43859756699707 (T) = (0 -104.55031897494622) / Math.Sqrt((23.28629562261446 / (299)) + (2.4191259065710833 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.3228424070704642 = (154.3958453196073 - 104.55031897494622) / 154.3958453196073 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Attributes.GetCustomAttributesMethodBaseMiss

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 100.88309972345523 < 138.0428116608551.
IsChangePoint: Marked as a change because one of 7/14/2022 4:09:15 PM, 7/18/2022 2:17:16 AM falls between 7/8/2022 10:31:34 AM and 7/18/2022 2:17:16 AM.
IsImprovementStdDev: Marked as improvement because 79.2388406063229 (T) = (0 -99.6519049537305) / Math.Sqrt((26.610651858540717 / (299)) + (6.322338551371557 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.3318253925866846 = (149.14051484163693 - 99.6519049537305) / 149.14051484163693 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Attributes.GetCustomAttributesMethodBaseHit

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 883.620630988676 < 932.3876316640841.
IsChangePoint: Marked as a change because one of 6/7/2022 3:01:09 PM, 6/21/2022 11:38:18 AM, 7/14/2022 4:09:15 PM, 7/18/2022 2:17:16 AM falls between 7/8/2022 10:31:34 AM and 7/18/2022 2:17:16 AM.
IsImprovementStdDev: Marked as improvement because 9.44001815713657 (T) = (0 -883.9965924043499) / Math.Sqrt((1500.0941959357676 / (299)) + (997.3159679570757 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.07182351750175832 = (952.4014118791515 - 883.9965924043499) / 952.4014118791515 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Attributes.GetCustomAttributesClassMissInherit

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 207.84329038471026 < 235.89004458207754.
IsChangePoint: Marked as a change because one of 6/27/2022 8:54:23 AM, 7/14/2022 4:09:15 PM, 7/18/2022 2:17:16 AM falls between 7/8/2022 10:31:34 AM and 7/18/2022 2:17:16 AM.
IsImprovementStdDev: Marked as improvement because 58.88638188699514 (T) = (0 -204.1808385562251) / Math.Sqrt((61.30544410420931 / (299)) + (10.53526515229774 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.19513849831047386 = (253.6844390340681 - 204.1808385562251) / 253.6844390340681 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Attributes.GetCustomAttributesMethodOverrideMiss

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 921.1792981235941 < 931.6816608291417.
IsChangePoint: Marked as a change because one of 6/7/2022 8:56:31 PM, 6/22/2022 3:28:08 PM, 7/2/2022 7:35:06 PM, 7/14/2022 8:11:01 PM, 7/18/2022 2:17:16 AM falls between 7/8/2022 10:31:34 AM and 7/18/2022 2:17:16 AM.
IsImprovementStdDev: Marked as improvement because 11.241895329259751 (T) = (0 -883.5566306803973) / Math.Sqrt((1557.1654182824823 / (299)) + (697.7446964023234 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.07455895357689256 = (954.7411302917715 - 883.5566306803973) / 954.7411302917715 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 0cc170c7532a2f23f2ef393b0eac11a1fb07c36f
Compare 81a3dcdad5235c38a329b372af132a0053dc1708
Diff Diff

Improvements in System.Tests.Perf_Enum

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 96.98 ns 79.29 ns 0.82 0.02 False
TryParseGeneric - Duration of single invocation 46.55 ns 30.49 ns 0.66 0.06 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Enum*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Enum.Parse(text: "Red")


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 79.2899043506839 < 92.06880010020726.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 4:09:15 PM, 7/18/2022 2:17:16 AM falls between 7/8/2022 10:31:34 AM and 7/18/2022 2:17:16 AM.
IsImprovementStdDev: Marked as improvement because 25.811890327129678 (T) = (0 -79.92527625562484) / Math.Sqrt((170.04165197021157 / (299)) + (0.34911648162926817 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.19812568860409313 = (99.67307235031699 - 79.92527625562484) / 99.67307235031699 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Enum.TryParseGeneric(text: "Red")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 30.49080980320826 < 44.229804306344.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 4:09:15 PM, 7/18/2022 2:17:16 AM falls between 7/8/2022 10:31:34 AM and 7/18/2022 2:17:16 AM.
IsImprovementStdDev: Marked as improvement because 23.18138712698066 (T) = (0 -30.223982096551882) / Math.Sqrt((198.81826687233095 / (299)) + (0.34906260259069755 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.3877055613502104 = (49.36184323868882 - 30.223982096551882) / 49.36184323868882 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 0cc170c7532a2f23f2ef393b0eac11a1fb07c36f
Compare 81a3dcdad5235c38a329b372af132a0053dc1708
Diff Diff

Improvements in System.Tests.Perf_Environment

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetFolderPath - Duration of single invocation 334.68 ns 307.44 ns 0.92 0.51 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Environment*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Environment.GetFolderPath(folder: System, option: None)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 307.44036496564564 < 323.87221741201324.
IsChangePoint: Marked as a change because one of 7/9/2022 7:00:36 PM, 7/14/2022 4:09:15 PM, 7/18/2022 2:17:16 AM falls between 7/8/2022 10:31:34 AM and 7/18/2022 2:17:16 AM.
IsImprovementStdDev: Marked as improvement because 22.41834133316654 (T) = (0 -282.7798756158937) / Math.Sqrt((466.01201743603644 / (299)) + (205.8666481414507 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.21087429230212773 = (358.3457906102837 - 282.7798756158937) / 358.3457906102837 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions