- 
                Notifications
    You must be signed in to change notification settings 
- Fork 4
Closed
Description
Run Information
| Name | Value | 
|---|---|
| Architecture | x64 | 
| OS | ubuntu 22.04 | 
| Queue | TigerUbuntu | 
| Baseline | a2b7214af6fe4c002847b8dcc68aabafc5523fa2 | 
| Compare | 8ba8249272917366e2382bb4c67a2347d19d2fb6 | 
| Diff | Diff | 
| Configs | CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:false, RunKind:micro_mono | 
Improvements in System.Numerics.Tensors.Tests.Perf_NumberTensorPrimitives<Single>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | 
|---|---|---|---|---|---|---|---|---|
| 3.00 μs | 2.36 μs | 0.79 | 0.01 | False | ||||
| 3.20 μs | 2.97 μs | 0.93 | 0.01 | True | ||||
| 176.09 ns | 148.20 ns | 0.84 | 0.01 | False | ||||
| 207.37 ns | 174.39 ns | 0.84 | 0.01 | False | ||||
| 2.78 μs | 2.16 μs | 0.78 | 0.01 | False | 
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Repro Steps
Prerequisites (Build files either built locally or downloaded from payload above)
- Libraries build extracted to runtime/artifactsor build instructions: Libraries README args:-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
- CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args:-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
- Mono Runtime build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args:-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release
- Dotnet SDK installed for dotnet commands
- Running commands from the runtime folder
Linux
# Set $RunDir to the runtime directory
RunDir=`pwd`
# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'
# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf
cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r
cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun
# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release
# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance
# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tensors.Tests.Perf_NumberTensorPrimitives<Single>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" 
# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Numerics.Tensors.Tests.Perf_NumberTensorPrimitives<Single>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packagesWindows
# Set $RunDir to the runtime directory
$RunDir="FullPathHere"
# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'
# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y
xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y
xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y
# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release
# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance
# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tensors.Tests.Perf_NumberTensorPrimitives<Single>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" 
# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Numerics.Tensors.Tests.Perf_NumberTensorPrimitives<Single>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packagesSystem.Numerics.Tensors.Tests.Perf_NumberTensorPrimitives<Single>.Max_Vector(BufferLength: 3079)
ETL Files
Histogram
JIT Disasms
System.Numerics.Tensors.Tests.Perf_NumberTensorPrimitives<Single>.MaxMagnitude_Vector(BufferLength: 3079)
ETL Files
Histogram
JIT Disasms
System.Numerics.Tensors.Tests.Perf_NumberTensorPrimitives<Single>.Max_Scalar(BufferLength: 128)
ETL Files
Histogram
JIT Disasms
System.Numerics.Tensors.Tests.Perf_NumberTensorPrimitives<Single>.Max_Vector(BufferLength: 128)
ETL Files
Histogram
JIT Disasms
System.Numerics.Tensors.Tests.Perf_NumberTensorPrimitives<Single>.Max_Scalar(BufferLength: 3079)
ETL Files
Histogram
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value | 
|---|---|
| Architecture | x64 | 
| OS | ubuntu 22.04 | 
| Queue | TigerUbuntu | 
| Baseline | 4a1a07666b55ed425a5c43896e7869227fca500c | 
| Compare | 2fe4afa81e5986124a1fef91e6f35f8994b92ef7 | 
| Diff | Diff | 
| Configs | CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:false, RunKind:micro_mono | 
Improvements in System.Numerics.Tensors.Tests.Perf_NumberTensorPrimitives<Double>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | 
|---|---|---|---|---|---|---|---|---|
| 5.56 μs | 4.25 μs | 0.76 | 0.01 | False | ||||
| 296.94 ns | 231.10 ns | 0.78 | 0.01 | False | ||||
| 324.94 ns | 265.25 ns | 0.82 | 0.01 | False | ||||
| 6.32 μs | 5.77 μs | 0.91 | 0.00 | True | ||||
| 296.14 ns | 279.23 ns | 0.94 | 0.00 | True | 
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Repro Steps
Prerequisites (Build files either built locally or downloaded from payload above)
- Libraries build extracted to runtime/artifactsor build instructions: Libraries README args:-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
- CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args:-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
- Mono Runtime build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args:-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release
- Dotnet SDK installed for dotnet commands
- Running commands from the runtime folder
Linux
# Set $RunDir to the runtime directory
RunDir=`pwd`
# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'
# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf
cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r
cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun
# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release
# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance
# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tensors.Tests.Perf_NumberTensorPrimitives<Double>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" 
# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Numerics.Tensors.Tests.Perf_NumberTensorPrimitives<Double>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packagesWindows
# Set $RunDir to the runtime directory
$RunDir="FullPathHere"
# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'
# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y
xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y
xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y
# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release
# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance
# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tensors.Tests.Perf_NumberTensorPrimitives<Double>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" 
# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Numerics.Tensors.Tests.Perf_NumberTensorPrimitives<Double>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packagesSystem.Numerics.Tensors.Tests.Perf_NumberTensorPrimitives<Double>.Max_Scalar(BufferLength: 3079)
ETL Files
Histogram
JIT Disasms
System.Numerics.Tensors.Tests.Perf_NumberTensorPrimitives<Double>.Max_Scalar(BufferLength: 128)
ETL Files
Histogram
JIT Disasms
System.Numerics.Tensors.Tests.Perf_NumberTensorPrimitives<Double>.Max_Vector(BufferLength: 128)
ETL Files
Histogram
JIT Disasms
System.Numerics.Tensors.Tests.Perf_NumberTensorPrimitives<Double>.MaxMagnitude_Vector(BufferLength: 3079)
ETL Files
Histogram
JIT Disasms
System.Numerics.Tensors.Tests.Perf_NumberTensorPrimitives<Double>.MaxMagnitude_Scalar(BufferLength: 128)
ETL Files
Histogram
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository









