[Perf] Linux/arm64: 56 Improvements on 7/6/2023 2:07:14 AM

[performanceautofiler]

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	cd26e63274102c8130109c9ce1301851a550433c
Compare	5d54b08d5fc40d0b1c156f430a487a94c1e34f79
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:true, RunKind:micro_mono

Improvements in System.Numerics.Tests.Perf_Vector4

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
LerpBenchmark - Duration of single invocation 📝 - Benchmark Source	118.89 ns	70.58 ns	0.59	0.23	False
DistanceSquaredBenchmark - Duration of single invocation 📝 - Benchmark Source	55.40 ns	27.96 ns	0.50	0.22	True
MultiplyOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source	35.41 ns	25.24 ns	0.71	0.39	False
SubtractOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source	31.61 ns	11.09 ns	0.35	0.41	True
AddOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source	32.35 ns	13.04 ns	0.40	0.37	True
TransformByMatrix4x4Benchmark - Duration of single invocation 📝 - Benchmark Source	355.98 ns	152.14 ns	0.43	0.21	False
SquareRootBenchmark - Duration of single invocation 📝 - Benchmark Source	40.34 ns	32.55 ns	0.81	0.24	False
NegateOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source	30.90 ns	9.20 ns	0.30	0.45	True
SubtractFunctionBenchmark - Duration of single invocation 📝 - Benchmark Source	33.45 ns	12.26 ns	0.37	0.38	True
DivideByScalarOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source	38.55 ns	29.03 ns	0.75	0.30	False
CreateFromScalarXYZWBenchmark - Duration of single invocation 📝 - Benchmark Source	16.42 ns	3.22 ns	0.20	0.40	False
OneBenchmark - Duration of single invocation 📝 - Benchmark Source	14.68 ns	2.89 ns	0.20	0.27	False
UnitWBenchmark - Duration of single invocation 📝 - Benchmark Source	16.11 ns	5.14 ns	0.32	0.37	False
UnitZBenchmark - Duration of single invocation 📝 - Benchmark Source	15.46 ns	3.23 ns	0.21	0.42	False
DistanceSquaredJitOptimizeCanaryBenchmark - Duration of single invocation 📝 - Benchmark Source	55.63 ns	29.93 ns	0.54	0.18	True
NegateBenchmark - Duration of single invocation 📝 - Benchmark Source	30.42 ns	8.88 ns	0.29	0.42	True
DistanceBenchmark - Duration of single invocation 📝 - Benchmark Source	56.01 ns	33.96 ns	0.61	0.18	True
MultiplyByScalarOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source	37.73 ns	27.60 ns	0.73	0.22	False
TransformVector3ByMatrix4x4Benchmark - Duration of single invocation 📝 - Benchmark Source	372.03 ns	151.61 ns	0.41	0.21	False
MultiplyFunctionBenchmark - Duration of single invocation 📝 - Benchmark Source	31.40 ns	25.82 ns	0.82	0.36	False
UnitXBenchmark - Duration of single invocation 📝 - Benchmark Source	15.45 ns	4.76 ns	0.31	0.40	False
TransformVector2ByMatrix4x4Benchmark - Duration of single invocation 📝 - Benchmark Source	269.32 ns	117.62 ns	0.44	0.17	False
AddFunctionBenchmark - Duration of single invocation 📝 - Benchmark Source	32.06 ns	13.04 ns	0.41	0.48	True
UnitYBenchmark - Duration of single invocation 📝 - Benchmark Source	15.84 ns	4.86 ns	0.31	0.37	False
MultiplyByScalarBenchmark - Duration of single invocation 📝 - Benchmark Source	38.44 ns	30.25 ns	0.79	0.25	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps

Prerequisites (Build files either built locally or downloaded from payload above)

• Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
• CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
• Mono Runtime build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release
• Dotnet SDK installed for dotnet commands
• Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf
cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r
cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector4*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" 

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Numerics.Tests.Perf_Vector4*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y
xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y
xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector4*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" 

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Numerics.Tests.Perf_Vector4*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages

Payloads

Baseline
Compare

System.Numerics.Tests.Perf_Vector4.LerpBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 10/7/2022 12:48:42 AM, 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 40.89025425919532 (T) = (0 -73.80810395165595) / Math.Sqrt((73.04781430657692 / (229)) + (11.39241075607783 / (7))) is greater than 1.9701536426234678 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (229) + (7) - 2, .975) and 0.4359618412650754 = (130.85657913145343 - 73.80810395165595) / 130.85657913145343 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector4.DistanceSquaredBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 6/6/2023 5:17:50 PM, 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 41.60866250461653 (T) = (0 -30.082138216147303) / Math.Sqrt((11.636583639319642 / (229)) + (4.5714106352659405 / (7))) is greater than 1.9701536426234678 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (229) + (7) - 2, .975) and 0.5371311254856159 = (64.99062666010494 - 30.082138216147303) / 64.99062666010494 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector4.MultiplyOperatorBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 6/16/2023 12:55:40 AM, 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 29.27453539623601 (T) = (0 -26.22602991529457) / Math.Sqrt((8.186619759324362 / (229)) + (1.2432371411866676 / (7))) is greater than 1.9701536426234678 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (229) + (7) - 2, .975) and 0.34019307736537313 = (39.74803691142452 - 26.22602991529457) / 39.74803691142452 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector4.SubtractOperatorBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 70.94372293692983 (T) = (0 -11.906743748266196) / Math.Sqrt((8.733497433428234 / (225)) + (0.8044996935069028 / (7))) is greater than 1.9703317732749426 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (225) + (7) - 2, .975) and 0.7002626781041952 = (39.723927847748094 - 11.906743748266196) / 39.723927847748094 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector4.AddOperatorBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 80.79922265341747 (T) = (0 -12.147197913805854) / Math.Sqrt((9.008345731506253 / (229)) + (0.5320569210346613 / (7))) is greater than 1.9701536426234678 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (229) + (7) - 2, .975) and 0.6931651963100114 = (39.58872255599404 - 12.147197913805854) / 39.58872255599404 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector4.TransformByMatrix4x4Benchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/6/2023 2:21:07 AM, 2/2/2023 2:17:49 AM, 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 15.77563148888068 (T) = (0 -162.386831902834) / Math.Sqrt((11551.146626067126 / (228)) + (50.10073325270952 / (7))) is greater than 1.970197598972363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (228) + (7) - 2, .975) and 0.42486189022084997 = (282.34406508932204 - 162.386831902834) / 282.34406508932204 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector4.SquareRootBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 5/14/2023 1:47:19 AM, 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 19.04432879183876 (T) = (0 -33.2236417590305) / Math.Sqrt((5.79090330201539 / (229)) + (2.0814193101133376 / (7))) is greater than 1.9701536426234678 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (229) + (7) - 2, .975) and 0.24561980328546687 = (44.040978148320534 - 33.2236417590305) / 44.040978148320534 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector4.NegateOperatorBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 54.7158699849617 (T) = (0 -10.528550166722798) / Math.Sqrt((8.906337886742415 / (222)) + (1.581904805005422 / (7))) is greater than 1.9704695126513152 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (222) + (7) - 2, .975) and 0.7283230619593924 = (38.753934149350194 - 10.528550166722798) / 38.753934149350194 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector4.SubtractFunctionBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 85.09246280974996 (T) = (0 -11.271392160095647) / Math.Sqrt((7.9723346771039685 / (229)) + (0.5403221377464912 / (7))) is greater than 1.9701536426234678 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (229) + (7) - 2, .975) and 0.7164365668590283 = (39.749103173300036 - 11.271392160095647) / 39.749103173300036 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector4.DivideByScalarOperatorBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 10/7/2022 12:48:42 AM, 5/25/2023 2:16:53 AM, 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 39.76841154927348 (T) = (0 -29.488780809638723) / Math.Sqrt((15.716410335740944 / (228)) + (0.7448703379453344 / (7))) is greater than 1.970197598972363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (228) + (7) - 2, .975) and 0.36090315210020135 = (46.14133351861273 - 29.488780809638723) / 46.14133351861273 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector4.CreateFromScalarXYZWBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 42.639902167371716 (T) = (0 -3.4081364875637274) / Math.Sqrt((2.7434849404432597 / (229)) + (0.7352847594867661 / (7))) is greater than 1.9701536426234678 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (229) + (7) - 2, .975) and 0.8106015663183284 = (17.99453364694609 - 3.4081364875637274) / 17.99453364694609 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector4.OneBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 10/7/2022 12:48:42 AM, 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 46.84948187356942 (T) = (0 -2.8916984960325425) / Math.Sqrt((11.658539625488084 / (229)) + (0.051415867729658005 / (7))) is greater than 1.9701536426234678 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (229) + (7) - 2, .975) and 0.796350640049123 = (14.199398891948697 - 2.8916984960325425) / 14.199398891948697 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector4.UnitWBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/2/2023 2:17:49 AM, 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 34.82032598723199 (T) = (0 -3.756309115499857) / Math.Sqrt((22.557202855735827 / (229)) + (1.0487425481978654 / (7))) is greater than 1.9701536426234678 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (229) + (7) - 2, .975) and 0.822043303428626 = (21.107995303752418 - 3.756309115499857) / 21.107995303752418 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector4.UnitZBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/2/2023 2:17:49 AM, 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 41.07880801988439 (T) = (0 -3.220267537894329) / Math.Sqrt((22.336767309763832 / (229)) + (0.6687111530800322 / (7))) is greater than 1.9701536426234678 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (229) + (7) - 2, .975) and 0.8486020227185874 = (21.27021506970747 - 3.220267537894329) / 21.27021506970747 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector4.DistanceSquaredJitOptimizeCanaryBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 88.76873110430756 (T) = (0 -30.180308710633064) / Math.Sqrt((10.914120748177364 / (229)) + (0.7438238818365869 / (7))) is greater than 1.9701536426234678 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (229) + (7) - 2, .975) and 0.5357352573944293 = (65.00667817515836 - 30.180308710633064) / 65.00667817515836 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector4.NegateBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 41.95851769470183 (T) = (0 -10.013537414131479) / Math.Sqrt((10.19035428561103 / (229)) + (3.0302003819647054 / (7))) is greater than 1.9701536426234678 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (229) + (7) - 2, .975) and 0.7432687142390902 = (39.00396238990889 - 10.013537414131479) / 39.00396238990889 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector4.DistanceBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 6/6/2023 5:17:50 PM, 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 88.07273501290017 (T) = (0 -33.75809081624904) / Math.Sqrt((13.15329330236437 / (220)) + (0.5130185081966913 / (7))) is greater than 1.9705633902925863 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (220) + (7) - 2, .975) and 0.4876339361101428 = (65.88666423369128 - 33.75809081624904) / 65.88666423369128 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector4.MultiplyByScalarOperatorBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 10/7/2022 12:48:42 AM, 5/25/2023 2:16:53 AM, 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 47.483015980830466 (T) = (0 -27.993278280718016) / Math.Sqrt((16.899067902361438 / (229)) + (0.5345228988680665 / (7))) is greater than 1.9701536426234678 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (229) + (7) - 2, .975) and 0.39660424858259474 = (46.39289921243309 - 27.993278280718016) / 46.39289921243309 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector4.TransformVector3ByMatrix4x4Benchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/6/2023 2:21:07 AM, 2/2/2023 2:17:49 AM, 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 15.194803178088234 (T) = (0 -163.5871570286338) / Math.Sqrt((11336.118356867468 / (224)) + (97.27789034414832 / (7))) is greater than 1.9703772833260038 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (224) + (7) - 2, .975) and 0.4272634948590859 = (285.623765135043 - 163.5871570286338) / 285.623765135043 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector4.MultiplyFunctionBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 6/16/2023 12:55:40 AM, 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 35.98586635839514 (T) = (0 -26.180795842016558) / Math.Sqrt((8.52545338168867 / (228)) + (0.7407105532624909 / (7))) is greater than 1.970197598972363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (228) + (7) - 2, .975) and 0.34217243870654684 = (39.798873416824584 - 26.180795842016558) / 39.798873416824584 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector4.UnitXBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/2/2023 2:17:49 AM, 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 29.76107537328738 (T) = (0 -3.970531262761549) / Math.Sqrt((22.13583228899527 / (228)) + (1.685378134125478 / (7))) is greater than 1.970197598972363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (228) + (7) - 2, .975) and 0.8133205946310088 = (21.269251714796198 - 3.970531262761549) / 21.269251714796198 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector4.TransformVector2ByMatrix4x4Benchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/6/2023 2:21:07 AM, 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 17.22347003817652 (T) = (0 -122.13561223929823) / Math.Sqrt((6707.357863987436 / (229)) + (17.255406639594636 / (7))) is greater than 1.9701536426234678 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (229) + (7) - 2, .975) and 0.44279246376979964 = (219.1923193745895 - 122.13561223929823) / 219.1923193745895 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector4.AddFunctionBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 75.01786941902684 (T) = (0 -12.159028422137917) / Math.Sqrt((8.139248831554168 / (229)) + (0.6956785796027313 / (7))) is greater than 1.9701536426234678 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (229) + (7) - 2, .975) and 0.6938407552904767 = (39.71471916085407 - 12.159028422137917) / 39.71471916085407 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector4.UnitYBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/2/2023 2:17:49 AM, 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 40.4219181011406 (T) = (0 -3.3818198869286342) / Math.Sqrt((21.43413487159 / (228)) + (0.6956925742898733 / (7))) is greater than 1.970197598972363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (228) + (7) - 2, .975) and 0.8401635346632748 = (21.157999708040357 - 3.3818198869286342) / 21.157999708040357 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector4.MultiplyByScalarBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 10/7/2022 12:48:42 AM, 5/25/2023 2:16:53 AM, 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 27.868484117602755 (T) = (0 -29.125215882804696) / Math.Sqrt((17.338409894019943 / (228)) + (2.1218182477428233 / (7))) is greater than 1.970197598972363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (228) + (7) - 2, .975) and 0.3707494495701118 = (46.28556282216532 - 29.125215882804696) / 46.28556282216532 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	cd26e63274102c8130109c9ce1301851a550433c
Compare	5d54b08d5fc40d0b1c156f430a487a94c1e34f79
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:true, RunKind:micro_mono

Improvements in System.Numerics.Tests.Perf_Matrix4x4

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
AddBenchmark - Duration of single invocation 📝 - Benchmark Source	395.21 ns	118.25 ns	0.30	0.19	False
CreateOrthographicOffCenterBenchmark - Duration of single invocation 📝 - Benchmark Source	116.98 ns	65.91 ns	0.56	0.21	False
CreateFromYawPitchRollBenchmarkBenchmark - Duration of single invocation 📝 - Benchmark Source	280.50 ns	234.16 ns	0.83	0.14	False
TranslationBenchmark - Duration of single invocation 📝 - Benchmark Source	126.17 ns	60.06 ns	0.48	0.26	False
SubtractBenchmark - Duration of single invocation 📝 - Benchmark Source	389.48 ns	124.24 ns	0.32	0.24	False
CreateScaleFromScalarBenchmark - Duration of single invocation 📝 - Benchmark Source	88.87 ns	31.41 ns	0.35	0.34	False
CreateTranslationFromScalarXYZ - Duration of single invocation 📝 - Benchmark Source	92.10 ns	25.15 ns	0.27	0.25	False
IdentityBenchmark - Duration of single invocation 📝 - Benchmark Source	92.74 ns	23.94 ns	0.26	0.25	False
SubtractOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source	444.91 ns	120.22 ns	0.27	0.19	False
CreateOrthographicBenchmark - Duration of single invocation 📝 - Benchmark Source	104.29 ns	58.06 ns	0.56	0.29	False
NegateBenchmark - Duration of single invocation 📝 - Benchmark Source	275.94 ns	91.30 ns	0.33	0.16	False
CreateFromScalars - Duration of single invocation 📝 - Benchmark Source	90.45 ns	62.27 ns	0.69	0.26	False
CreateRotationYWithCenterBenchmark - Duration of single invocation 📝 - Benchmark Source	121.92 ns	75.84 ns	0.62	0.17	False
CreateFromQuaternionBenchmark - Duration of single invocation 📝 - Benchmark Source	187.21 ns	144.03 ns	0.77	0.18	False
DecomposeBenchmark - Duration of single invocation 📝 - Benchmark Source	1.01 μs	826.05 ns	0.82	0.19	False
GetDeterminantBenchmark - Duration of single invocation 📝 - Benchmark Source	237.27 ns	159.52 ns	0.67	0.31	False
LerpBenchmark - Duration of single invocation 📝 - Benchmark Source	627.43 ns	380.67 ns	0.61	0.21	False
CreateRotationZWithCenterBenchmark - Duration of single invocation 📝 - Benchmark Source	122.00 ns	71.36 ns	0.58	0.21	False
MultiplyByScalarBenchmark - Duration of single invocation 📝 - Benchmark Source	303.92 ns	185.03 ns	0.61	0.22	False
NegationOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source	275.33 ns	92.63 ns	0.34	0.16	False
Transpose - Duration of single invocation 📝 - Benchmark Source	227.87 ns	99.04 ns	0.43	0.29	False
AddOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source	442.62 ns	122.07 ns	0.28	0.22	False
CreateFromAxisAngleBenchmark - Duration of single invocation 📝 - Benchmark Source	181.51 ns	133.07 ns	0.73	0.18	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps

Prerequisites (Build files either built locally or downloaded from payload above)

• Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
• CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
• Mono Runtime build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release
• Dotnet SDK installed for dotnet commands
• Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf
cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r
cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix4x4*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" 

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Numerics.Tests.Perf_Matrix4x4*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y
xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y
xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix4x4*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" 

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Numerics.Tests.Perf_Matrix4x4*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages

Payloads

Baseline
Compare

System.Numerics.Tests.Perf_Matrix4x4.AddBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/6/2023 2:21:07 AM, 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 24.633036514772158 (T) = (0 -124.83494959434317) / Math.Sqrt((16152.621055855068 / (228)) + (41.11932139256721 / (7))) is greater than 1.970197598972363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (228) + (7) - 2, .975) and 0.6334789562830535 = (340.5942216260564 - 124.83494959434317) / 340.5942216260564 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.CreateOrthographicOffCenterBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/6/2023 2:21:07 AM, 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 13.60914905654034 (T) = (0 -72.43463534988751) / Math.Sqrt((858.2666670396292 / (229)) + (22.108539068638688 / (7))) is greater than 1.9701536426234678 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (229) + (7) - 2, .975) and 0.3305433204607204 = (108.19913754499703 - 72.43463534988751) / 108.19913754499703 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.CreateFromYawPitchRollBenchmarkBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/6/2023 2:21:07 AM, 3/16/2023 1:27:09 AM, 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 22.673949104291264 (T) = (0 -242.00205100914167) / Math.Sqrt((390.08642498045214 / (225)) + (26.16194468749788 / (7))) is greater than 1.9703317732749426 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (225) + (7) - 2, .975) and 0.17975824978817087 = (295.0374702904895 - 242.00205100914167) / 295.0374702904895 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.TranslationBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/6/2023 2:21:07 AM, 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 14.949799846161 (T) = (0 -59.316352617621035) / Math.Sqrt((2029.4336475662942 / (229)) + (30.765314780483923 / (7))) is greater than 1.9701536426234678 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (229) + (7) - 2, .975) and 0.4785340297456756 = (113.74923005750851 - 59.316352617621035) / 113.74923005750851 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.SubtractBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/6/2023 2:21:07 AM, 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 19.80252317304567 (T) = (0 -136.26189698737497) / Math.Sqrt((16288.514533662925 / (229)) + (255.31059887873724 / (7))) is greater than 1.9701536426234678 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (229) + (7) - 2, .975) and 0.6011956387465842 = (341.67604526468267 - 136.26189698737497) / 341.67604526468267 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromScalarBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/6/2023 2:21:07 AM, 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 22.736578564388477 (T) = (0 -25.781898140577233) / Math.Sqrt((1173.766082031399 / (229)) + (6.502842904920758 / (7))) is greater than 1.9701536426234678 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (229) + (7) - 2, .975) and 0.6845391011183084 = (81.72771405893417 - 25.781898140577233) / 81.72771405893417 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.CreateTranslationFromScalarXYZ

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/6/2023 2:21:07 AM, 2/2/2023 2:17:49 AM, 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 24.245922469247652 (T) = (0 -24.984929132083487) / Math.Sqrt((1432.0557863183928 / (229)) + (0.6779802977502108 / (7))) is greater than 1.9701536426234678 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (229) + (7) - 2, .975) and 0.7097628244171661 = (86.08452408589805 - 24.984929132083487) / 86.08452408589805 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.IdentityBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/6/2023 2:21:07 AM, 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 18.363942842493206 (T) = (0 -24.932123631116344) / Math.Sqrt((2044.759715182379 / (229)) + (2.198933158118836 / (7))) is greater than 1.9701536426234678 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (229) + (7) - 2, .975) and 0.6912942318787738 = (80.76338768417733 - 24.932123631116344) / 80.76338768417733 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.SubtractOperatorBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/6/2023 2:21:07 AM, 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 21.28873534627305 (T) = (0 -132.22615220080365) / Math.Sqrt((17391.838143445897 / (229)) + (139.21366241509187 / (7))) is greater than 1.9701536426234678 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (229) + (7) - 2, .975) and 0.6118216733341278 = (340.6324957308048 - 132.22615220080365) / 340.6324957308048 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.CreateOrthographicBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/6/2023 2:21:07 AM, 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 15.222816064259161 (T) = (0 -61.14672698294422) / Math.Sqrt((971.7600157915412 / (229)) + (5.754372348632805 / (7))) is greater than 1.9701536426234678 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (229) + (7) - 2, .975) and 0.3591052003541298 = (95.4083681389382 - 61.14672698294422) / 95.4083681389382 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.NegateBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/6/2023 2:21:07 AM, 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 23.35300755304484 (T) = (0 -97.16158279939319) / Math.Sqrt((8270.276111859674 / (222)) + (23.009491727885404 / (7))) is greater than 1.9704695126513152 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (222) + (7) - 2, .975) and 0.6047990629496446 = (245.85362455001754 - 97.16158279939319) / 245.85362455001754 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.CreateFromScalars

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/6/2023 2:21:07 AM, 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 25.466908336785828 (T) = (0 -63.12098841126723) / Math.Sqrt((185.6414228577089 / (228)) + (0.5449341904903587 / (7))) is greater than 1.970197598972363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (228) + (7) - 2, .975) and 0.2759218103395286 = (87.1742711113359 - 63.12098841126723) / 87.1742711113359 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.CreateRotationYWithCenterBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/6/2023 2:21:07 AM, 3/16/2023 1:27:09 AM, 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 20.38971171317227 (T) = (0 -77.29248281566073) / Math.Sqrt((862.2046241855901 / (228)) + (4.7564066026139455 / (7))) is greater than 1.970197598972363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (228) + (7) - 2, .975) and 0.3578127429159332 = (120.35816961958591 - 77.29248281566073) / 120.35816961958591 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.CreateFromQuaternionBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/6/2023 2:21:07 AM, 3/17/2023 1:34:47 AM, 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 24.11643302683062 (T) = (0 -146.64077735668712) / Math.Sqrt((639.9273391623575 / (229)) + (15.322272415401578 / (7))) is greater than 1.9701536426234678 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (229) + (7) - 2, .975) and 0.2685400516277397 = (200.47683770384316 - 146.64077735668712) / 200.47683770384316 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.DecomposeBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 1/6/2023 2:21:07 AM, 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 8.874412391528699 (T) = (0 -869.6079176453568) / Math.Sqrt((7701.036430985331 / (229)) + (1620.1844607347587 / (7))) is greater than 1.9701536426234678 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (229) + (7) - 2, .975) and 0.1424794797577595 = (1014.0957529502638 - 869.6079176453568) / 1014.0957529502638 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.GetDeterminantBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/6/2023 2:21:07 AM, 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 14.833723857194645 (T) = (0 -158.64533917761855) / Math.Sqrt((2614.25233684488 / (226)) + (26.50169523346092 / (7))) is greater than 1.970286659282671 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (226) + (7) - 2, .975) and 0.26813647231276516 = (216.76901932653263 - 158.64533917761855) / 216.76901932653263 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.LerpBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/6/2023 2:21:07 AM, 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 15.00973987435756 (T) = (0 -394.59762120856414) / Math.Sqrt((23004.48074665123 / (229)) + (346.6504561988806 / (7))) is greater than 1.9701536426234678 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (229) + (7) - 2, .975) and 0.3177949729421496 = (578.4150007078482 - 394.59762120856414) / 578.4150007078482 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.CreateRotationZWithCenterBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/6/2023 2:21:07 AM, 3/16/2023 1:27:09 AM, 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 18.89263010320043 (T) = (0 -76.6451554896729) / Math.Sqrt((865.7607554867417 / (229)) + (10.970391008121515 / (7))) is greater than 1.9701536426234678 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (229) + (7) - 2, .975) and 0.3630684947773911 = (120.33500440975244 - 76.6451554896729) / 120.33500440975244 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.MultiplyByScalarBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/6/2023 2:21:07 AM, 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 11.417144139112283 (T) = (0 -193.66571806707307) / Math.Sqrt((7349.669010481404 / (225)) + (172.94248000994784 / (7))) is greater than 1.9703317732749426 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (225) + (7) - 2, .975) and 0.30869109169316794 = (280.1435302510467 - 193.66571806707307) / 280.1435302510467 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.NegationOperatorBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/6/2023 2:21:07 AM, 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 22.22666226842593 (T) = (0 -96.7293409704793) / Math.Sqrt((9069.217754936783 / (229)) + (19.35601296182516 / (7))) is greater than 1.9701536426234678 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (229) + (7) - 2, .975) and 0.5993075952614686 = (241.40547668628565 - 96.7293409704793) / 241.40547668628565 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.Transpose

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/6/2023 2:21:07 AM, 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 17.087441215226843 (T) = (0 -107.1439431614411) / Math.Sqrt((5850.451792167428 / (222)) + (46.37483155887847 / (7))) is greater than 1.9704695126513152 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (222) + (7) - 2, .975) and 0.4780382395087198 = (205.27163342501416 - 107.1439431614411) / 205.27163342501416 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.AddOperatorBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/6/2023 2:21:07 AM, 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 23.762348825578883 (T) = (0 -126.72795029666858) / Math.Sqrt((17360.616139409718 / (228)) + (30.485066303429576 / (7))) is greater than 1.970197598972363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (228) + (7) - 2, .975) and 0.6271889761874844 = (339.925437291788 - 126.72795029666858) / 339.925437291788 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.CreateFromAxisAngleBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/6/2023 2:21:07 AM, 3/16/2023 1:27:09 AM, 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 20.50045095761424 (T) = (0 -136.78368040032873) / Math.Sqrt((835.8134815659984 / (229)) + (13.652245752496873 / (7))) is greater than 1.9701536426234678 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (229) + (7) - 2, .975) and 0.2618150404787222 = (185.29730067791502 - 136.78368040032873) / 185.29730067791502 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	cd26e63274102c8130109c9ce1301851a550433c
Compare	5d54b08d5fc40d0b1c156f430a487a94c1e34f79
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:true, RunKind:micro_mono

Improvements in System.Numerics.Tests.Perf_Plane

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
DotBenchmark - Duration of single invocation 📝 - Benchmark Source	54.67 ns	32.27 ns	0.59	0.30	False
CreateFromVector4Benchmark - Duration of single invocation 📝 - Benchmark Source	39.94 ns	24.63 ns	0.62	0.37	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps

Prerequisites (Build files either built locally or downloaded from payload above)

• Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
• CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
• Mono Runtime build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release
• Dotnet SDK installed for dotnet commands
• Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf
cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r
cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Plane*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" 

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Numerics.Tests.Perf_Plane*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y
xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y
xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Plane*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" 

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Numerics.Tests.Perf_Plane*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages

Payloads

Baseline
Compare

System.Numerics.Tests.Perf_Plane.DotBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/2/2023 2:17:49 AM, 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 30.768829614721472 (T) = (0 -34.18910572737452) / Math.Sqrt((37.87997921993795 / (228)) + (3.5274590010661533 / (7))) is greater than 1.970197598972363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (228) + (7) - 2, .975) and 0.4241899003322956 = (59.37566177999446 - 34.18910572737452) / 59.37566177999446 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Plane.CreateFromVector4Benchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/2/2023 2:17:49 AM, 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 53.84781972128581 (T) = (0 -25.17827032189866) / Math.Sqrt((39.66815486383519 / (220)) + (0.12609119565535074 / (7))) is greater than 1.9705633902925863 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (220) + (7) - 2, .975) and 0.4878151635074372 = (49.15856255003414 - 25.17827032189866) / 49.15856255003414 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	cd26e63274102c8130109c9ce1301851a550433c
Compare	5d54b08d5fc40d0b1c156f430a487a94c1e34f79
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:true, RunKind:micro_mono

Improvements in System.Numerics.Tests.Perf_Vector3

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
TransformByMatrix4x4Benchmark - Duration of single invocation 📝 - Benchmark Source	351.44 ns	154.62 ns	0.44	0.16	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps

Prerequisites (Build files either built locally or downloaded from payload above)

• Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
• CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
• Mono Runtime build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release
• Dotnet SDK installed for dotnet commands
• Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf
cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r
cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector3*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" 

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Numerics.Tests.Perf_Vector3*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y
xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y
xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector3*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" 

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Numerics.Tests.Perf_Vector3*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages

Payloads

Baseline
Compare

System.Numerics.Tests.Perf_Vector3.TransformByMatrix4x4Benchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/6/2023 2:21:07 AM, 2/2/2023 2:17:49 AM, 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 13.46276379156282 (T) = (0 -164.71074788677961) / Math.Sqrt((14295.63700693383 / (228)) + (33.56631347753691 / (7))) is greater than 1.970197598972363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (228) + (7) - 2, .975) and 0.40173673761687745 = (275.31482917849684 - 164.71074788677961) / 275.31482917849684 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	cd26e63274102c8130109c9ce1301851a550433c
Compare	5d54b08d5fc40d0b1c156f430a487a94c1e34f79
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:true, RunKind:micro_mono

Improvements in System.Numerics.Tests.Perf_Quaternion

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
InverseBenchmark - Duration of single invocation 📝 - Benchmark Source	104.97 ns	84.01 ns	0.80	0.16	False
ConjugateBenchmark - Duration of single invocation 📝 - Benchmark Source	58.48 ns	39.28 ns	0.67	0.25	False
CreateFromRotationMatrixBenchmark - Duration of single invocation 📝 - Benchmark Source	182.45 ns	86.67 ns	0.48	0.25	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps

Prerequisites (Build files either built locally or downloaded from payload above)

• Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
• CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
• Mono Runtime build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release
• Dotnet SDK installed for dotnet commands
• Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf
cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r
cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Quaternion*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" 

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Numerics.Tests.Perf_Quaternion*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y
xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y
xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Quaternion*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" 

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Numerics.Tests.Perf_Quaternion*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages

Payloads

Baseline
Compare

System.Numerics.Tests.Perf_Quaternion.InverseBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/2/2023 2:17:49 AM, 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 11.45504047053019 (T) = (0 -91.51838232815813) / Math.Sqrt((75.23225996385862 / (229)) + (14.37919539691012 / (7))) is greater than 1.9701536426234678 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (229) + (7) - 2, .975) and 0.16192237019478362 = (109.20036411117258 - 91.51838232815813) / 109.20036411117258 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Quaternion.ConjugateBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/2/2023 2:17:49 AM, 6/17/2023 12:20:29 AM, 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 26.553098319202444 (T) = (0 -43.564221783019214) / Math.Sqrt((15.176878200899427 / (229)) + (3.811869007933701 / (7))) is greater than 1.9701536426234678 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (229) + (7) - 2, .975) and 0.3226630465917349 = (64.31691282132198 - 43.564221783019214) / 64.31691282132198 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Quaternion.CreateFromRotationMatrixBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/6/2023 2:21:07 AM, 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 16.19923575404111 (T) = (0 -92.56159654271151) / Math.Sqrt((2132.2260127378927 / (222)) + (17.43340806131744 / (7))) is greater than 1.9704695126513152 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (222) + (7) - 2, .975) and 0.37836125054920555 = (148.89933522401532 - 92.56159654271151) / 148.89933522401532 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	cd26e63274102c8130109c9ce1301851a550433c
Compare	5d54b08d5fc40d0b1c156f430a487a94c1e34f79
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:true, RunKind:micro_mono

Improvements in System.Linq.Tests.Perf_Enumerable

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
EmptyTakeSelectToArray - Duration of single invocation 📝 - Benchmark Source	237.72 ns	140.55 ns	0.59	0.10	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps

Prerequisites (Build files either built locally or downloaded from payload above)

• Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
• CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
• Mono Runtime build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release
• Dotnet SDK installed for dotnet commands
• Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf
cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r
cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" 

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Linq.Tests.Perf_Enumerable*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y
xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y
xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" 

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Linq.Tests.Perf_Enumerable*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages

Payloads

Baseline
Compare

System.Linq.Tests.Perf_Enumerable.EmptyTakeSelectToArray

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/16/2023 1:27:09 AM, 7/1/2023 1:53:48 AM, 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 55.81399918016975 (T) = (0 -141.10797704484773) / Math.Sqrt((130.81669560643496 / (229)) + (0.4417441640021378 / (7))) is greater than 1.9701536426234678 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (229) + (7) - 2, .975) and 0.2395639943955392 = (185.56193552760934 - 141.10797704484773) / 185.56193552760934 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	cd26e63274102c8130109c9ce1301851a550433c
Compare	5d54b08d5fc40d0b1c156f430a487a94c1e34f79
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:true, RunKind:micro_mono

Improvements in System.Numerics.Tests.Perf_VectorOf<Single>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
OneBenchmark - Duration of single invocation 📝 - Benchmark Source	46.12 ns	0.75 ns	0.02	0.42	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps

Prerequisites (Build files either built locally or downloaded from payload above)

• Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
• CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
• Mono Runtime build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release
• Dotnet SDK installed for dotnet commands
• Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf
cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r
cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf<Single>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" 

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Numerics.Tests.Perf_VectorOf<Single>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y
xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y
xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf<Single>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" 

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Numerics.Tests.Perf_VectorOf<Single>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages

Payloads

Baseline
Compare

System.Numerics.Tests.Perf_VectorOf<Single>.OneBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 7/5/2023 12:45:20 AM, 7/14/2023 2:14:06 AM falls between 7/5/2023 12:45:20 AM and 7/14/2023 2:14:06 AM.
IsImprovementStdDev: Marked as improvement because 158.04787275598503 (T) = (0 -1.1984934657632107) / Math.Sqrt((12.126315428938575 / (220)) + (0.22714492922712692 / (7))) is greater than 1.9705633902925863 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (220) + (7) - 2, .975) and 0.9750147737348992 = (47.96808534158684 - 1.1984934657632107) / 47.96808534158684 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[kotlarmilos]

@BrzVlad more Vector4 improvements by dotnet/runtime#87822.