Skip to content

[PERF][Release/7.0] Update release/7.0 Perf infrastructure to use Perf repo release/7.0 branch #77699

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Nov 3, 2022
Merged

[PERF][Release/7.0] Update release/7.0 Perf infrastructure to use Perf repo release/7.0 branch #77699

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
42bd9fd
Update perf branch to clone to release/7.0 from main.
LoopedBard3 Oct 31, 2022
5b34fb0
Setup testing.
LoopedBard3 Oct 31, 2022
8b66246
Fix SOD and Startup builds for performance.
LoopedBard3 Oct 26, 2022
acd0492
[PERF] Pass local runtime timestamp from git into ci_setup (#76396)
LoopedBard3 Oct 3, 2022
bd6aee4
Updated the Nuget feed to fix net6.0 maui run and moved .net version …
LoopedBard3 Nov 1, 2022
9e79920
Switch back to using the release/7.0 perf branch.
LoopedBard3 Nov 1, 2022
0cca6c4
Remove user from pip3 install for musl.
LoopedBard3 Nov 2, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions eng/pipelines/coreclr/templates/build-perf-maui-apps-net6.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,10 +27,10 @@ steps:

# Get the current maui nuget config so all things can be found and darc based package sources are kept up to date.
- script: |
curl -o NuGet.config 'https://raw.githubusercontent.com/dotnet/maui/main/NuGet.config'
curl -o NuGet.config 'https://raw.githubusercontent.com/dotnet/maui/net6.0/NuGet.config'
curl -o dotnet-install.sh 'https://dotnet.microsoft.com/download/dotnet/scripts/v1/dotnet-install.sh'
chmod -R a+rx .
./dotnet-install.sh --channel 6.0.4xx --quality daily --install-dir .
./dotnet-install.sh --channel 6.0 --quality daily --install-dir .
./dotnet --info
./dotnet workload install maui --from-rollback-file https://aka.ms/dotnet/maui/net6.0.json --configfile NuGet.config
displayName: Install MAUI workload
Expand Down
6 changes: 3 additions & 3 deletions eng/pipelines/coreclr/templates/run-performance-job.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ jobs:
export PERFLAB_UPLOAD_TOKEN="$(HelixPerfUploadTokenValue)"
|| export PERF_PREREQS_INSTALL_FAILED=1;
test "x$PERF_PREREQS_INSTALL_FAILED" = "x1" && echo "** Error: Failed to install prerequites **"
- HelixPreCommandStemMusl: 'export ORIGPYPATH=$PYTHONPATH;sudo apk add icu-libs krb5-libs libgcc libintl libssl1.1 libstdc++ zlib cargo;sudo apk add libgdiplus --repository http://dl-cdn.alpinelinux.org/alpine/edge/testing; python3 -m venv $HELIX_WORKITEM_PAYLOAD/.venv;source $HELIX_WORKITEM_PAYLOAD/.venv/bin/activate;export PYTHONPATH=;python3 -m pip install --user -U pip;pip3 install --user azure.storage.blob==12.7.1;pip3 install --user azure.storage.queue==12.1.5;export PERFLAB_UPLOAD_TOKEN="$(HelixPerfUploadTokenValue)"'
- HelixPreCommandStemMusl: 'export ORIGPYPATH=$PYTHONPATH;sudo apk add icu-libs krb5-libs libgcc libintl libssl1.1 libstdc++ zlib cargo;sudo apk add libgdiplus --repository http://dl-cdn.alpinelinux.org/alpine/edge/testing; python3 -m venv $HELIX_WORKITEM_PAYLOAD/.venv;source $HELIX_WORKITEM_PAYLOAD/.venv/bin/activate;export PYTHONPATH=;python3 -m pip install -U pip;pip3 install azure.storage.blob==12.7.1;pip3 install azure.storage.queue==12.1.5;export PERFLAB_UPLOAD_TOKEN="$(HelixPerfUploadTokenValue)"'
- ExtraMSBuildLogsWindows: 'set MSBUILDDEBUGCOMM=1;set "MSBUILDDEBUGPATH=%HELIX_WORKITEM_UPLOAD_ROOT%"'
- ExtraMSBuildLogsLinux: 'export MSBUILDDEBUGCOMM=1;export "MSBUILDDEBUGPATH=$HELIX_WORKITEM_UPLOAD_ROOT"'
- HelixPreCommand: ''
Expand Down Expand Up @@ -155,11 +155,11 @@ jobs:
_Framework: ${{ framework }}
steps:
- ${{ parameters.steps }}
- powershell: $(Build.SourcesDirectory)\eng\testing\performance\performance-setup.ps1 $(IsInternal)$(Interpreter) -Framework $(_Framework) -Kind ${{ parameters.runKind }} -LogicalMachine ${{ parameters.logicalMachine }} ${{ parameters.pgoRunType }} ${{ parameters.extraSetupParameters }}
- powershell: $(Build.SourcesDirectory)\eng\testing\performance\performance-setup.ps1 $(IsInternal)$(Interpreter) -Framework $(_Framework) -Kind ${{ parameters.runKind }} -LogicalMachine ${{ parameters.logicalMachine }} ${{ parameters.pgoRunType }} -UseLocalCommitTime ${{ parameters.extraSetupParameters }}
displayName: Performance Setup (Windows)
condition: and(succeeded(), eq(variables['Agent.Os'], 'Windows_NT'))
continueOnError: ${{ parameters.continueOnError }}
- script: $(Build.SourcesDirectory)/eng/testing/performance/performance-setup.sh $(IsInternal)$(Interpreter) --framework $(_Framework) --kind ${{ parameters.runKind }} --logicalmachine ${{ parameters.logicalMachine }} ${{ parameters.extraSetupParameters }}
- script: $(Build.SourcesDirectory)/eng/testing/performance/performance-setup.sh $(IsInternal)$(Interpreter) --framework $(_Framework) --kind ${{ parameters.runKind }} --logicalmachine ${{ parameters.logicalMachine }} --uselocalcommittime ${{ parameters.extraSetupParameters }}
displayName: Performance Setup (Unix)
condition: and(succeeded(), ne(variables['Agent.Os'], 'Windows_NT'))
continueOnError: ${{ parameters.continueOnError }}
Expand Down
16 changes: 8 additions & 8 deletions eng/pipelines/coreclr/templates/run-scenarios-job.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -126,11 +126,11 @@ jobs:
steps:
- ${{ parameters.steps }}
# run performance-setup
- powershell: $(Build.SourcesDirectory)\eng\testing\performance\performance-setup.ps1 $(IsInternal) -Framework $(_Framework) -Kind ${{ parameters.runKind }} -LogicalMachine ${{ parameters.logicalMachine }} ${{ parameters.extraSetupParameters }} ${{ parameters.additionalSetupParameters }}
- powershell: $(Build.SourcesDirectory)\eng\testing\performance\performance-setup.ps1 $(IsInternal) -Framework $(_Framework) -Kind ${{ parameters.runKind }} -LogicalMachine ${{ parameters.logicalMachine }} -UseLocalCommitTime ${{ parameters.extraSetupParameters }} ${{ parameters.additionalSetupParameters }}
displayName: Performance Setup (Windows)
condition: and(succeeded(), eq(variables['Agent.Os'], 'Windows_NT'))
continueOnError: ${{ parameters.continueOnError }}
- script: $(Build.SourcesDirectory)/eng/testing/performance/performance-setup.sh $(IsInternal) --framework $(_Framework) --kind ${{ parameters.runKind }} --logicalmachine ${{ parameters.logicalMachine }} ${{ parameters.extraSetupParameters }} ${{ parameters.additionalSetupParameters }}
- script: $(Build.SourcesDirectory)/eng/testing/performance/performance-setup.sh $(IsInternal) --framework $(_Framework) --kind ${{ parameters.runKind }} --logicalmachine ${{ parameters.logicalMachine }} --uselocalcommittime ${{ parameters.extraSetupParameters }} ${{ parameters.additionalSetupParameters }}
displayName: Performance Setup (Linux/MAC)
condition: and(succeeded(), ne(variables['Agent.Os'], 'Windows_NT'))
continueOnError: ${{ parameters.continueOnError }}
Expand All @@ -157,33 +157,33 @@ jobs:
displayName: Copy scenario support files (Linux/MAC)
condition: and(succeeded(), ne(variables['Agent.Os'], 'Windows_NT'))
# build Startup
- script: $(PayloadDirectory)\dotnet\dotnet.exe publish -c Release -o $(WorkItemDirectory)\Startup -f net7.0 -r win-$(Architecture) $(PerformanceDirectory)\src\tools\ScenarioMeasurement\Startup\Startup.csproj
- script: $(PayloadDirectory)\dotnet\dotnet.exe publish -c Release -o $(WorkItemDirectory)\Startup -f net7.0 -r win-$(Architecture) $(PerformanceDirectory)\src\tools\ScenarioMeasurement\Startup\Startup.csproj -p:DisableTransitiveFrameworkReferenceDownloads=true
displayName: Build Startup tool (Windows)
env:
PERFLAB_TARGET_FRAMEWORKS: net7.0
condition: and(succeeded(), eq(variables['Agent.Os'], 'Windows_NT'))
- script: $(PayloadDirectory)/dotnet/dotnet publish -c Release -o $(WorkItemDirectory)/startup -f net7.0 -r linux-$(Architecture) $(PerformanceDirectory)/src/tools/ScenarioMeasurement/Startup/Startup.csproj
- script: $(PayloadDirectory)/dotnet/dotnet publish -c Release -o $(WorkItemDirectory)/startup -f net7.0 -r linux-$(Architecture) $(PerformanceDirectory)/src/tools/ScenarioMeasurement/Startup/Startup.csproj -p:DisableTransitiveFrameworkReferenceDownloads=true
displayName: Build Startup tool (Linux)
env:
PERFLAB_TARGET_FRAMEWORKS: net7.0
condition: and(succeeded(), eq(variables['Agent.Os'], 'Linux'))
- script: $(PayloadDirectory)/dotnet/dotnet publish -c Release -o $(WorkItemDirectory)/startup -f net7.0 -r osx-$(Architecture) $(PerformanceDirectory)/src/tools/ScenarioMeasurement/Startup/Startup.csproj
- script: $(PayloadDirectory)/dotnet/dotnet publish -c Release -o $(WorkItemDirectory)/startup -f net7.0 -r osx-$(Architecture) $(PerformanceDirectory)/src/tools/ScenarioMeasurement/Startup/Startup.csproj -p:DisableTransitiveFrameworkReferenceDownloads=true
displayName: Build Startup tool (MAC)
env:
PERFLAB_TARGET_FRAMEWORKS: net7.0
condition: and(succeeded(), eq(variables['Agent.Os'], 'Darwin'))
# build SizeOnDisk
- script: $(PayloadDirectory)\dotnet\dotnet.exe publish -c Release -o $(WorkItemDirectory)\SOD -f net7.0 -r win-$(Architecture) $(PerformanceDirectory)\src\tools\ScenarioMeasurement\SizeOnDisk\SizeOnDisk.csproj
- script: $(PayloadDirectory)\dotnet\dotnet.exe publish -c Release -o $(WorkItemDirectory)\SOD -f net7.0 -r win-$(Architecture) $(PerformanceDirectory)\src\tools\ScenarioMeasurement\SizeOnDisk\SizeOnDisk.csproj -p:DisableTransitiveFrameworkReferenceDownloads=true
displayName: Build SizeOnDisk tool (Windows)
env:
PERFLAB_TARGET_FRAMEWORKS: net7.0
condition: and(succeeded(), eq(variables['Agent.Os'], 'Windows_NT'))
- script: $(PayloadDirectory)/dotnet/dotnet publish -c Release -o $(WorkItemDirectory)/SOD -f net7.0 -r linux-$(Architecture) $(PerformanceDirectory)/src/tools/ScenarioMeasurement/SizeOnDisk/SizeOnDisk.csproj
- script: $(PayloadDirectory)/dotnet/dotnet publish -c Release -o $(WorkItemDirectory)/SOD -f net7.0 -r linux-$(Architecture) $(PerformanceDirectory)/src/tools/ScenarioMeasurement/SizeOnDisk/SizeOnDisk.csproj -p:DisableTransitiveFrameworkReferenceDownloads=true
displayName: Build SizeOnDisk tool (Linux)
env:
PERFLAB_TARGET_FRAMEWORKS: net7.0
condition: and(succeeded(), eq(variables['Agent.Os'], 'Linux'))
- script: $(PayloadDirectory)/dotnet/dotnet publish -c Release -o $(WorkItemDirectory)/SOD -f net7.0 -r osx-$(Architecture) $(PerformanceDirectory)/src/tools/ScenarioMeasurement/SizeOnDisk/SizeOnDisk.csproj
- script: $(PayloadDirectory)/dotnet/dotnet publish -c Release -o $(WorkItemDirectory)/SOD -f net7.0 -r osx-$(Architecture) $(PerformanceDirectory)/src/tools/ScenarioMeasurement/SizeOnDisk/SizeOnDisk.csproj -p:DisableTransitiveFrameworkReferenceDownloads=true
displayName: Build SizeOnDisk tool (MAC)
env:
PERFLAB_TARGET_FRAMEWORKS: net7.0
Expand Down
11 changes: 9 additions & 2 deletions eng/testing/performance/performance-setup.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,8 @@ Param(
[switch] $DynamicPGO,
[switch] $FullPGO,
[switch] $iOSLlvmBuild,
[string] $MauiVersion
[string] $MauiVersion,
[switch] $UseLocalCommitTime
)

$RunFromPerformanceRepo = ($Repository -eq "dotnet/performance") -or ($Repository -eq "dotnet-performance")
Expand Down Expand Up @@ -120,13 +121,19 @@ elseif($FullPGO)
$SetupArguments = "$SetupArguments --full-pgo"
}

if($UseLocalCommitTime)
{
$LocalCommitTime = (git show -s --format=%ci $CommitSha)
$SetupArguments = "$SetupArguments --commit-time `"$LocalCommitTime`""
}

if ($RunFromPerformanceRepo) {
$SetupArguments = "--perf-hash $CommitSha $CommonSetupArguments"

robocopy $SourceDirectory $PerformanceDirectory /E /XD $PayloadDirectory $SourceDirectory\artifacts $SourceDirectory\.git
}
else {
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance $PerformanceDirectory
git clone --branch release/7.0 --depth 1 --quiet https://github.com/dotnet/performance $PerformanceDirectory
}

if($MonoDotnet -ne "")
Expand Down
13 changes: 12 additions & 1 deletion eng/testing/performance/performance-setup.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ javascript_engine="v8"
iosmono=false
iosllvmbuild=""
maui_version=""
use_local_commit_time=false
only_sanity=false

while (($# > 0)); do
Expand Down Expand Up @@ -160,6 +161,10 @@ while (($# > 0)); do
maui_version=$2
shift 2
;;
--uselocalcommittime)
use_local_commit_time=true
shift 1
;;
--perffork)
perf_fork=$2
shift 2
Expand Down Expand Up @@ -200,6 +205,7 @@ while (($# > 0)); do
echo " --iosmono Set for ios Mono/Maui runs"
echo " --iosllvmbuild Set LLVM for iOS Mono/Maui runs"
echo " --mauiversion Set the maui version for Mono/Maui runs"
echo " --uselocalcommittime Pass local runtime commit time to the setup script"
echo ""
exit 0
;;
Expand Down Expand Up @@ -314,6 +320,11 @@ if [[ "$internal" != true ]]; then
setup_arguments="$setup_arguments --not-in-lab"
fi

if [[ "$use_local_commit_time" == true ]]; then
local_commit_time=$(git show -s --format=%ci $commit_sha)
setup_arguments="$setup_arguments --commit-time \"$local_commit_time\""
fi

if [[ "$run_from_perf_repo" == true ]]; then
payload_directory=
workitem_directory=$source_directory
Expand All @@ -323,7 +334,7 @@ else
if [[ -n "$perf_fork" ]]; then
git clone --branch $perf_fork_branch --depth 1 --quiet $perf_fork $performance_directory
else
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $performance_directory
git clone --branch release/7.0 --depth 1 --quiet https://github.com/dotnet/performance.git $performance_directory
Copy link
Member

@hoyosjs hoyosjs Nov 2, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just as a future thing, do you think it's worth using the build agent variables here?

Copy link
Member Author

@LoopedBard3 LoopedBard3 Nov 2, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where do the build agent variables get set/added from? I like the idea, and it may be worth making something in this area that will use the variable if it is properly available. At the very least, it is worthwhile to look into it.

Copy link
Member Author

@LoopedBard3 LoopedBard3 Nov 2, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If there is a good Doc link to make sure I am thinking of the correct thing when you say build agent variable would be great.

Copy link
Member

@hoyosjs hoyosjs Nov 3, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See https://learn.microsoft.com/en-us/azure/devops/pipelines/build/variables?view=azure-devops&tabs=yaml, particularly something like Build.SourceBranchName if this doesn't run on PRs, and use System.PullRequest.TargetBranch for PRs, that way you don't have to patch when branching.

fi
# uncomment to use BenchmarkDotNet sources instead of nuget packages
# git clone https://github.com/dotnet/BenchmarkDotNet.git $benchmark_directory
Expand Down