Skip to content

[wasm] Fix perf pipeline errors #75406

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Sep 13, 2022
Merged

[wasm] Fix perf pipeline errors #75406

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
9e60417
[wasm] Fix perf-pipeline errors
radical Sep 11, 2022
20e010f
[wasm] Fix perf pipeline errors
radical Sep 11, 2022
bfd5cfd
[wasm] Use test-main.js from 7.0 branch for perf pipeline
radical Sep 11, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion eng/pipelines/coreclr/perf-wasm-jobs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,7 @@ jobs:
runJobTemplate: /eng/pipelines/coreclr/templates/run-performance-job.yml
logicalmachine: 'perftiger'
javascriptEngine: 'v8'
additionalSetupParameters: '--dotnet-versions 8.0.0' # passed to ci_setup.py
collectHelixLogsScript: ${{ parameters.collectHelixLogsScript }}
compare: ${{ parameters.compare }}
onlySanityCheck: ${{ parameters.onlySanityCheck }}
Expand All @@ -140,6 +141,7 @@ jobs:
runjobtemplate: /eng/pipelines/coreclr/templates/run-performance-job.yml
logicalmachine: 'perftiger'
javascriptEngine: 'v8'
additionalSetupParameters: '--dotnet-versions 8.0.0' # passed to ci_setup.py
collectHelixLogsScript: ${{ parameters.collectHelixLogsScript }}
compare: ${{ parameters.compare }}
onlySanityCheck: ${{ parameters.onlySanityCheck }}
Expand All @@ -162,7 +164,7 @@ jobs:
projectFile: blazor_perf.proj
runKind: blazor_scenarios
runJobTemplate: /eng/pipelines/coreclr/templates/run-scenarios-job.yml
#additionalSetupParameters: '--latestdotnet' - doesn't do anything, IIUC. see performance-setup.sh
additionalSetupParameters: '--dotnetversions 8.0.0' # passed to performance-setup.sh
logicalmachine: 'perftiger'
downloadSpecificBuild: ${{ parameters.downloadSpecificBuild }}
perfForkToUse: ${{ parameters.perfForkToUse }}
3 changes: 2 additions & 1 deletion eng/pipelines/coreclr/templates/perf-job.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -181,11 +181,12 @@ jobs:
displayName: BrowserWasm
${{ insert }}: ${{ parameters.downloadSpecificBuild }}

# Using test-main-7.0.js, since we are building with tfm:net7.0
- script: >-
mkdir -p $(librariesDownloadDir)/bin/wasm/wasm-data &&
mkdir -p $(librariesDownloadDir)/bin/wasm/dotnet &&
cp -r $(librariesDownloadDir)/BrowserWasm/staging/dotnet-net7+latest/* $(librariesDownloadDir)/bin/wasm/dotnet &&
cp src/mono/wasm/test-main.js $(librariesDownloadDir)/bin/wasm/wasm-data/test-main.js &&
cp src/mono/wasm/Wasm.Build.Tests/data/test-main-7.0.js $(librariesDownloadDir)/bin/wasm/wasm-data/test-main.js &&
find $(librariesDownloadDir)/bin/wasm -type d &&
find $(librariesDownloadDir)/bin/wasm -type f -exec chmod 664 {} \;
displayName: "Create wasm directory (Linux)"
Expand Down
3 changes: 2 additions & 1 deletion eng/pipelines/coreclr/templates/run-performance-job.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ parameters:
javascriptEngine: 'NoJS'
helixTypeSuffix: '' # optional -- appends to HelixType
collectHelixLogsScript: '' # optional -- script to collect the logs, and artifacts helpful for debugging failures
additionalSetupParameters: '' # optional -- additional setup parameters that are job-specific

jobs:
- template: xplat-pipeline-job.yml
Expand Down Expand Up @@ -162,7 +163,7 @@ jobs:
displayName: Performance Setup (Unix)
condition: and(succeeded(), ne(variables['Agent.Os'], 'Windows_NT'))
continueOnError: ${{ parameters.continueOnError }}
- script: $(Python) $(PerformanceDirectory)/scripts/ci_setup.py $(SetupArguments)
- script: $(Python) $(PerformanceDirectory)/scripts/ci_setup.py $(SetupArguments) ${{ parameters.additionalSetupParameters }}
Copy link
Member

@LoopedBard3 LoopedBard3 Sep 12, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is mentioned that the additionalSetupParameters is passed to performance-setup, although is being passed to ci_setup here. Along with the changes to performance-setup, it looks like this should be added to the lines above this one.

Copy link
Member Author

@radical radical Sep 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For run-perfomance-job the comments in perf-wasm-jobs.yml say:
additionalSetupParameters: '--dotnet-versions 8.0.0' # passed to ci_setup.py

For run-scenarios-job the comment says:
additionalSetupParameters: '--dotnetversions 8.0.0' # passed to performance-setup.sh

That matches what the ymls are doing. Please correct me if I'm missing something.

displayName: Run ci setup script
# Run perf testing in helix
- template: /eng/pipelines/coreclr/templates/perf-send-to-helix.yml
Expand Down
34 changes: 17 additions & 17 deletions eng/pipelines/coreclr/templates/run-scenarios-job.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -107,10 +107,9 @@ jobs:
- AdditionalHelixPreCommands: $(HelixPreCommandOSX)
- AdditionalHelixPostCommands: $(HelixPostCommandOSX)

- ${{ if ne(parameters.runtimeType, 'wasm') }}:
- ExtraSetupArguments: --install-dir $(PayloadDirectory)/dotnet
- ${{ if and(eq(parameters.runtimeType, 'wasm'), in(variables['Build.Reason'], 'PullRequest')) }}:
- ExtraSetupArguments: ''
- name: ExtraSetupArguments
${{ if ne(parameters.runtimeType, 'wasm') }}:
value: --install-dir $(PayloadDirectory)/dotnet

workspace:
clean: all
Expand Down Expand Up @@ -142,10 +141,11 @@ jobs:
displayName: Run ci setup script (Linux/MAC)
condition: and(succeeded(), ne(variables['Agent.Os'], 'Windows_NT'))
# copy wasm packs if running on wasm
# Using test-main-7.0.js, since we are building with tfm:net7.0
- script: >-
mkdir -p $(librariesDownloadDir)/bin/wasm/data &&
cp -r $(librariesDownloadDir)/BrowserWasm/staging/dotnet-net7+latest $(librariesDownloadDir)/bin/wasm &&
cp src/mono/wasm/test-main.js $(librariesDownloadDir)/bin/wasm/data/test-main.js &&
cp src/mono/wasm/Wasm.Build.Tests/data/test-main-7.0.js $(librariesDownloadDir)/bin/wasm/data/test-main.js &&
find $(librariesDownloadDir)/bin/wasm -type f -exec chmod 664 {} \;
displayName: "Create wasm directory (Linux)"
condition: and(succeeded(), eq('${{ parameters.runtimeType }}', 'wasm'))
Expand All @@ -157,36 +157,36 @@ jobs:
displayName: Copy scenario support files (Linux/MAC)
condition: and(succeeded(), ne(variables['Agent.Os'], 'Windows_NT'))
# build Startup
- script: $(PayloadDirectory)\dotnet\dotnet.exe publish -c Release -o $(WorkItemDirectory)\Startup -f net6.0 -r win-$(Architecture) $(PerformanceDirectory)\src\tools\ScenarioMeasurement\Startup\Startup.csproj
- script: $(PayloadDirectory)\dotnet\dotnet.exe publish -c Release -o $(WorkItemDirectory)\Startup -f net7.0 -r win-$(Architecture) $(PerformanceDirectory)\src\tools\ScenarioMeasurement\Startup\Startup.csproj
displayName: Build Startup tool (Windows)
env:
PERFLAB_TARGET_FRAMEWORKS: net6.0
PERFLAB_TARGET_FRAMEWORKS: net7.0
condition: and(succeeded(), eq(variables['Agent.Os'], 'Windows_NT'))
- script: $(PayloadDirectory)/dotnet/dotnet publish -c Release -o $(WorkItemDirectory)/startup -f net6.0 -r linux-$(Architecture) $(PerformanceDirectory)/src/tools/ScenarioMeasurement/Startup/Startup.csproj
- script: $(PayloadDirectory)/dotnet/dotnet publish -c Release -o $(WorkItemDirectory)/startup -f net7.0 -r linux-$(Architecture) $(PerformanceDirectory)/src/tools/ScenarioMeasurement/Startup/Startup.csproj
displayName: Build Startup tool (Linux)
env:
PERFLAB_TARGET_FRAMEWORKS: net6.0
PERFLAB_TARGET_FRAMEWORKS: net7.0
condition: and(succeeded(), eq(variables['Agent.Os'], 'Linux'))
- script: $(PayloadDirectory)/dotnet/dotnet publish -c Release -o $(WorkItemDirectory)/startup -f net6.0 -r osx-$(Architecture) $(PerformanceDirectory)/src/tools/ScenarioMeasurement/Startup/Startup.csproj
- script: $(PayloadDirectory)/dotnet/dotnet publish -c Release -o $(WorkItemDirectory)/startup -f net7.0 -r osx-$(Architecture) $(PerformanceDirectory)/src/tools/ScenarioMeasurement/Startup/Startup.csproj
displayName: Build Startup tool (MAC)
env:
PERFLAB_TARGET_FRAMEWORKS: net6.0
PERFLAB_TARGET_FRAMEWORKS: net7.0
condition: and(succeeded(), eq(variables['Agent.Os'], 'Darwin'))
# build SizeOnDisk
- script: $(PayloadDirectory)\dotnet\dotnet.exe publish -c Release -o $(WorkItemDirectory)\SOD -f net6.0 -r win-$(Architecture) $(PerformanceDirectory)\src\tools\ScenarioMeasurement\SizeOnDisk\SizeOnDisk.csproj
- script: $(PayloadDirectory)\dotnet\dotnet.exe publish -c Release -o $(WorkItemDirectory)\SOD -f net7.0 -r win-$(Architecture) $(PerformanceDirectory)\src\tools\ScenarioMeasurement\SizeOnDisk\SizeOnDisk.csproj
displayName: Build SizeOnDisk tool (Windows)
env:
PERFLAB_TARGET_FRAMEWORKS: net6.0
PERFLAB_TARGET_FRAMEWORKS: net7.0
condition: and(succeeded(), eq(variables['Agent.Os'], 'Windows_NT'))
- script: $(PayloadDirectory)/dotnet/dotnet publish -c Release -o $(WorkItemDirectory)/SOD -f net6.0 -r linux-$(Architecture) $(PerformanceDirectory)/src/tools/ScenarioMeasurement/SizeOnDisk/SizeOnDisk.csproj
- script: $(PayloadDirectory)/dotnet/dotnet publish -c Release -o $(WorkItemDirectory)/SOD -f net7.0 -r linux-$(Architecture) $(PerformanceDirectory)/src/tools/ScenarioMeasurement/SizeOnDisk/SizeOnDisk.csproj
displayName: Build SizeOnDisk tool (Linux)
env:
PERFLAB_TARGET_FRAMEWORKS: net6.0
PERFLAB_TARGET_FRAMEWORKS: net7.0
condition: and(succeeded(), eq(variables['Agent.Os'], 'Linux'))
- script: $(PayloadDirectory)/dotnet/dotnet publish -c Release -o $(WorkItemDirectory)/SOD -f net6.0 -r osx-$(Architecture) $(PerformanceDirectory)/src/tools/ScenarioMeasurement/SizeOnDisk/SizeOnDisk.csproj
- script: $(PayloadDirectory)/dotnet/dotnet publish -c Release -o $(WorkItemDirectory)/SOD -f net7.0 -r osx-$(Architecture) $(PerformanceDirectory)/src/tools/ScenarioMeasurement/SizeOnDisk/SizeOnDisk.csproj
displayName: Build SizeOnDisk tool (MAC)
env:
PERFLAB_TARGET_FRAMEWORKS: net6.0
PERFLAB_TARGET_FRAMEWORKS: net7.0
condition: and(succeeded(), eq(variables['Agent.Os'], 'Darwin'))

# Zip the workitem directory (for xharness based workitems)
Expand Down
12 changes: 11 additions & 1 deletion eng/testing/performance/performance-setup.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ iosmono=false
iosllvmbuild=""
maui_version=""
only_sanity=false
dotnet_versions=""

while (($# > 0)); do
lowerI="$(echo $1 | tr "[:upper:]" "[:lower:]")"
Expand Down Expand Up @@ -148,6 +149,10 @@ while (($# > 0)); do
use_latest_dotnet=true
shift 1
;;
--dotnetversions)
dotnet_versions="$2"
shift 2
;;
--iosmono)
iosmono=true
shift 1
Expand Down Expand Up @@ -196,12 +201,13 @@ while (($# > 0)); do
echo " --wasmbundle Path to the wasm bundle containing the dotnet, and data needed for helix payload"
echo " --wasmaot Indicate wasm aot"
echo " --latestdotnet --dotnet-versions will not be specified. --dotnet-versions defaults to LKG version in global.json "
echo " --dotnetversions Passed as '--dotnet-versions <value>' to the setup script"
echo " --alpine Set for runs on Alpine"
echo " --iosmono Set for ios Mono/Maui runs"
echo " --iosllvmbuild Set LLVM for iOS Mono/Maui runs"
echo " --mauiversion Set the maui version for Mono/Maui runs"
echo ""
exit 0
exit 1
Copy link
Member

@LoopedBard3 LoopedBard3 Sep 12, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did we change this to 1?

Copy link
Member Author

@radical radical Sep 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you pass an unknown argument to the script then it will show help, and exit. But the build will keep going, even though it is missing crucial setup configuration. With exit 1, the build will stop at that point.

Copy link
Member

@LoopedBard3 LoopedBard3 Sep 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes sense, this should be a good change.

;;
esac
done
Expand Down Expand Up @@ -354,6 +360,10 @@ if [[ -n "$mono_dotnet" && "$monoaot" == "false" ]]; then
mv $mono_dotnet $mono_dotnet_path
fi

if [[ -n "$dotnet_versions" ]]; then
setup_arguments="$setup_arguments --dotnet-versions $dotnet_versions"
Copy link
Member

@LoopedBard3 LoopedBard3 Sep 12, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a corresponding change to ci_setup in the perf repo to use this setup argument? I didn't see anything already implemented.

Copy link
Member Author

@radical radical Sep 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.com/dotnet/performance/blob/89e9ed2e5bf061d7bc32ce4c0b4773e117fc6304/scripts/dotnet.py#L868-L876

Copy link
Member

@LoopedBard3 LoopedBard3 Sep 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, I missed the dotnet parser args being added to the ci_setup parser.

fi

if [[ "$monoaot" == "true" ]]; then
monoaot_dotnet_path=$payload_directory/monoaot
mv $monoaot_path $monoaot_dotnet_path
Expand Down