Benchmarks for #116045 (EgorBo)

[EgorBot]

Processing dotnet/runtime#116045 (comment) command:

Command

-amd -arm

using System;
using System.Collections.Generic;
using System.Linq;
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;

BenchmarkSwitcher.FromAssembly(typeof(Bench).Assembly).Run(args);

public class Bench
{
    static Dictionary<string, int> Dictionary_OC = Enumerable.Range(0, 1000)
        .ToDictionary(i => i.ToString(), i => i);

    static Dictionary<string, int> Dictionary_OCIC = Enumerable.Range(0, 1000)
        .ToDictionary(i => i.ToString(), i => i, StringComparer.OrdinalIgnoreCase);

    [Benchmark]
    [Arguments("1")]
    [Arguments("999")]
    public int Lookup_best(string key) => Dictionary_OC[key];

    [Benchmark]
    [Arguments("1000000")]
    public bool Contains_best(string key) => Dictionary_OC.ContainsKey(key);


    [Benchmark]
    [Arguments("1")]
    [Arguments("999")]
    public int Lookup_worst(string key) => Dictionary_OCIC[key];

    [Benchmark]
    [Arguments("1000000")]
    public bool Contains_worst(string key) => Dictionary_OCIC.ContainsKey(key);
}

(EgorBot will reply in this issue)

[EgorBot]

Benchmark results on linux_azure_genoa

BenchmarkDotNet v0.14.0, Ubuntu 24.04.2 LTS (Noble Numbat)
AMD EPYC 9V74, 1 CPU, 16 logical and 8 physical cores
  Job-AECFSV : .NET 10.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
  Job-YKVEDB : .NET 10.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI

Method	Toolchain	key	Mean	Error	Ratio
Lookup_best	Main	1	11.428 ns	0.0038 ns	1.00
Lookup_best	PR	1	8.423 ns	0.0027 ns	0.74
Lookup_worst	Main	1	11.601 ns	0.0058 ns	1.00
Lookup_worst	PR	1	11.491 ns	0.0055 ns	0.99
Contains_best	Main	1000000	6.154 ns	0.0040 ns	1.00
Contains_best	PR	1000000	4.548 ns	0.0007 ns	0.74
Contains_worst	Main	1000000	9.690 ns	0.0050 ns	1.00
Contains_worst	PR	1000000	9.724 ns	0.0038 ns	1.00
Lookup_best	Main	999	9.785 ns	0.0043 ns	1.00
Lookup_best	PR	999	6.730 ns	0.0012 ns	0.69
Lookup_worst	Main	999	13.776 ns	0.0164 ns	1.00
Lookup_worst	PR	999	13.870 ns	0.0097 ns	1.01

BDN_Artifacts.zip

cc @EgorBo (agent_logs.txt). EgorBot manual: link.

[EgorBot]

Benchmark results on linux_azure_cobalt100

BenchmarkDotNet v0.14.0, Ubuntu 24.04.2 LTS (Noble Numbat)
azure_cobalt100
  Job-TDHHFD : .NET 10.0.0 (42.42.42.42424), Arm64 RyuJIT AdvSIMD
  Job-WRGFEH : .NET 10.0.0 (42.42.42.42424), Arm64 RyuJIT AdvSIMD

Method	Toolchain	key	Mean	Error	Ratio
Lookup_best	Main	1	15.858 ns	0.0064 ns	1.00
Lookup_best	PR	1	10.970 ns	0.0033 ns	0.69
Lookup_worst	Main	1	15.779 ns	0.0357 ns	1.00
Lookup_worst	PR	1	16.357 ns	0.0031 ns	1.04
Contains_best	Main	1000000	6.766 ns	0.0007 ns	1.00
Contains_best	PR	1000000	4.473 ns	0.0018 ns	0.66
Contains_worst	Main	1000000	11.290 ns	0.0022 ns	1.00
Contains_worst	PR	1000000	11.770 ns	0.0026 ns	1.04
Lookup_best	Main	999	12.920 ns	0.0071 ns	1.00
Lookup_best	PR	999	8.606 ns	0.0012 ns	0.67
Lookup_worst	Main	999	15.624 ns	0.0055 ns	1.00
Lookup_worst	PR	999	16.382 ns	0.0057 ns	1.05

BDN_Artifacts.zip

cc @EgorBo (agent_logs.txt). EgorBot manual: link.

[EgorBot]

Benchmark results on linux_azure_genoa

BenchmarkDotNet v0.14.0, Ubuntu 24.04.2 LTS (Noble Numbat)
AMD EPYC 9V74, 1 CPU, 16 logical and 8 physical cores
  Job-ICFULL : .NET 10.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
  Job-LKBKHJ : .NET 10.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
RatioSD=?

Method	Toolchain	key	Mean	Error	Ratio
Lookup_best	Main	Bundl(...)rFile [28]	NA	NA	?
Lookup_best	PR	Bundl(...)rFile [28]	NA	NA	?
Lookup_worst	Main	Bundl(...)rFile [28]	NA	NA	?
Lookup_worst	PR	Bundl(...)rFile [28]	NA	NA	?
Lookup_best	Main	C:\pr(...)lease [61]	NA	NA	?
Lookup_best	PR	C:\pr(...)lease [61]	NA	NA	?
Lookup_worst	Main	C:\pr(...)lease [61]	NA	NA	?
Lookup_worst	PR	C:\pr(...)lease [61]	NA	NA	?
Lookup_best	Main	You (...)ted. [185]	NA	NA	?
Lookup_best	PR	You (...)ted. [185]	NA	NA	?
Lookup_worst	Main	You (...)ted. [185]	NA	NA	?
Lookup_worst	PR	You (...)ted. [185]	NA	NA	?
Benchmarks with issues:
Bench.Lookup_best: Job-ICFULL(Toolchain=Main) [key=Bundl(...)rFile [28]]
Bench.Lookup_best: Job-LKBKHJ(Toolchain=PR) [key=Bundl(...)rFile [28]]
Bench.Lookup_worst: Job-ICFULL(Toolchain=Main) [key=Bundl(...)rFile [28]]
Bench.Lookup_worst: Job-LKBKHJ(Toolchain=PR) [key=Bundl(...)rFile [28]]
Bench.Lookup_best: Job-ICFULL(Toolchain=Main) [key=C:\pr(...)lease [61]]
Bench.Lookup_best: Job-LKBKHJ(Toolchain=PR) [key=C:\pr(...)lease [61]]
Bench.Lookup_worst: Job-ICFULL(Toolchain=Main) [key=C:\pr(...)lease [61]]
Bench.Lookup_worst: Job-LKBKHJ(Toolchain=PR) [key=C:\pr(...)lease [61]]
Bench.Lookup_best: Job-ICFULL(Toolchain=Main) [key=You (...)ted. [185]]
Bench.Lookup_best: Job-LKBKHJ(Toolchain=PR) [key=You (...)ted. [185]]
Bench.Lookup_worst: Job-ICFULL(Toolchain=Main) [key=You (...)ted. [185]]
Bench.Lookup_worst: Job-LKBKHJ(Toolchain=PR) [key=You (...)ted. [185]]

BDN_Artifacts.zip

cc @EgorBo (agent_logs.txt). EgorBot manual: link.

[EgorBot]

Benchmark results on linux_azure_cobalt100

BenchmarkDotNet v0.14.0, Ubuntu 24.04.2 LTS (Noble Numbat)
azure_cobalt100
  Job-FXMCZK : .NET 10.0.0 (42.42.42.42424), Arm64 RyuJIT AdvSIMD
  Job-LGVYYS : .NET 10.0.0 (42.42.42.42424), Arm64 RyuJIT AdvSIMD
RatioSD=?

Method	Toolchain	key	Mean	Error	Ratio
Lookup_best	Main	Bundl(...)rFile [28]	NA	NA	?
Lookup_best	PR	Bundl(...)rFile [28]	NA	NA	?
Lookup_worst	Main	Bundl(...)rFile [28]	NA	NA	?
Lookup_worst	PR	Bundl(...)rFile [28]	NA	NA	?
Lookup_best	Main	C:\pr(...)lease [61]	NA	NA	?
Lookup_best	PR	C:\pr(...)lease [61]	NA	NA	?
Lookup_worst	Main	C:\pr(...)lease [61]	NA	NA	?
Lookup_worst	PR	C:\pr(...)lease [61]	NA	NA	?
Lookup_best	Main	You (...)ted. [185]	NA	NA	?
Lookup_best	PR	You (...)ted. [185]	NA	NA	?
Lookup_worst	Main	You (...)ted. [185]	NA	NA	?
Lookup_worst	PR	You (...)ted. [185]	NA	NA	?
Benchmarks with issues:
Bench.Lookup_best: Job-FXMCZK(Toolchain=Main) [key=Bundl(...)rFile [28]]
Bench.Lookup_best: Job-LGVYYS(Toolchain=PR) [key=Bundl(...)rFile [28]]
Bench.Lookup_worst: Job-FXMCZK(Toolchain=Main) [key=Bundl(...)rFile [28]]
Bench.Lookup_worst: Job-LGVYYS(Toolchain=PR) [key=Bundl(...)rFile [28]]
Bench.Lookup_best: Job-FXMCZK(Toolchain=Main) [key=C:\pr(...)lease [61]]
Bench.Lookup_best: Job-LGVYYS(Toolchain=PR) [key=C:\pr(...)lease [61]]
Bench.Lookup_worst: Job-FXMCZK(Toolchain=Main) [key=C:\pr(...)lease [61]]
Bench.Lookup_worst: Job-LGVYYS(Toolchain=PR) [key=C:\pr(...)lease [61]]
Bench.Lookup_best: Job-FXMCZK(Toolchain=Main) [key=You (...)ted. [185]]
Bench.Lookup_best: Job-LGVYYS(Toolchain=PR) [key=You (...)ted. [185]]
Bench.Lookup_worst: Job-FXMCZK(Toolchain=Main) [key=You (...)ted. [185]]
Bench.Lookup_worst: Job-LGVYYS(Toolchain=PR) [key=You (...)ted. [185]]

BDN_Artifacts.zip

cc @EgorBo (agent_logs.txt). EgorBot manual: link.

[EgorBo]

@EgorBot -arm -amd

using System;
using System.Collections.Generic;
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;

BenchmarkSwitcher.FromAssembly(typeof(Bench).Assembly).Run(args);

public class Bench
{
    private static Dictionary<string, int> Dictionary =
        new()
        {
            { "1", 1 },
            { "222", 2 },
            { "3333", 3 },
            { "44444", 4 },
            { "555555", 5 },
            { @"C:\prj\runtime-main\src\libraries\System.Private.CoreLib\src\System\Collections\Generic\Dictionary.cs", 6 },
            { "You can contribute to .NET Runtime with issues and PRs.", 7 },
            { "https://github.com/", 8 },
        };

    [Benchmark]
    [Arguments("1")]
    [Arguments("44444")]
    [Arguments("You can contribute to .NET Runtime with issues and PRs.")]
    [Arguments(@"C:\prj\runtime-main\src\libraries\System.Private.CoreLib\src\System\Collections\Generic\Dictionary.cs")]
    public int Lookup(string key) => Dictionary[key];
}

[EgorBot]

Benchmark results on linux_azure_cobalt100

BenchmarkDotNet v0.14.0, Ubuntu 24.04.2 LTS (Noble Numbat)
azure_cobalt100
  Job-OPZSWJ : .NET 10.0.0 (42.42.42.42424), Arm64 RyuJIT AdvSIMD
  Job-JIYQBU : .NET 10.0.0 (42.42.42.42424), Arm64 RyuJIT AdvSIMD

Method	Toolchain	key	Mean	Error	Ratio
Lookup	Main	1	11.645 ns	0.0044 ns	1.00
Lookup	PR	1	9.582 ns	0.0014 ns	0.82
Lookup	Main	44444	10.676 ns	0.0009 ns	1.00
Lookup	PR	44444	8.748 ns	0.0008 ns	0.82
Lookup	Main	C:\p(...)y.cs [101]	34.298 ns	0.0125 ns	1.00
Lookup	PR	C:\p(...)y.cs [101]	29.486 ns	0.0028 ns	0.86
Lookup	Main	You c(...) PRs. [55]	22.747 ns	0.0039 ns	1.00
Lookup	PR	You c(...) PRs. [55]	18.537 ns	0.0018 ns	0.81

BDN_Artifacts.zip

cc @EgorBo (agent_logs.txt). EgorBot manual: link.

[EgorBot]

Benchmark results on linux_azure_genoa

BenchmarkDotNet v0.14.0, Ubuntu 24.04.2 LTS (Noble Numbat)
AMD EPYC 9V74, 1 CPU, 16 logical and 8 physical cores
  Job-ZBQVHC : .NET 10.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
  Job-NRIUQG : .NET 10.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI

Method	Toolchain	key	Mean	Error	Ratio
Lookup	Main	1	8.377 ns	0.0012 ns	1.00
Lookup	PR	1	7.211 ns	0.0018 ns	0.86
Lookup	Main	44444	8.094 ns	0.0013 ns	1.00
Lookup	PR	44444	5.935 ns	0.0095 ns	0.73
Lookup	Main	C:\p(...)y.cs [101]	25.312 ns	0.0095 ns	1.00
Lookup	PR	C:\p(...)y.cs [101]	24.023 ns	0.0066 ns	0.95
Lookup	Main	You c(...) PRs. [55]	16.224 ns	0.0057 ns	1.00
Lookup	PR	You c(...) PRs. [55]	14.870 ns	0.0033 ns	0.92

BDN_Artifacts.zip

cc @EgorBo (agent_logs.txt). EgorBot manual: link.