Skip to content

Clean up the crossgen2_publish project and local/live packs references #92826

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 16 commits into from
Oct 3, 2023
Merged

Clean up the crossgen2_publish project and local/live packs references #92826

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
d4dffde
Inline some of the options for the new crossgen2_publish project.
jkoritzinsky Sep 29, 2023
47325f7
Resolve TODOs in targetingpacks.targets
jkoritzinsky Sep 29, 2023
1a4402d
Crossgen1 is long gone. Don't try to discover it in our override targ…
jkoritzinsky Sep 29, 2023
affe0fd
Move ReadyToRun.targets infra into the shared repo infrastructure and…
jkoritzinsky Sep 29, 2023
bfc601d
Remove extraneous property set (the same value is calculated automati…
jkoritzinsky Sep 29, 2023
0b69d8d
Remove outdated comment.
jkoritzinsky Sep 29, 2023
cffdb4d
Fix NativeAOT and installer legs
jkoritzinsky Sep 29, 2023
0e3c720
Condition turning off pack downloads based on opt-in to local pack us…
jkoritzinsky Sep 29, 2023
09eaab4
Use the LKG host instead of the 7.0 host as the fallback for NativeEx…
jkoritzinsky Sep 29, 2023
de2f5e2
Apply suggestions from code review
jkoritzinsky Sep 30, 2023
c47bb43
Merge branch 'main' of github.com:dotnet/runtime into crossgen2-publi…
jkoritzinsky Oct 2, 2023
6620cc6
Remove AdditionalProperties as they aren't needed (global properties …
jkoritzinsky Oct 2, 2023
6a29dc3
PR feedback
jkoritzinsky Oct 2, 2023
8d32750
Hook into the targets pipeline to avoid overridding targets for R2Rin…
jkoritzinsky Oct 2, 2023
aac7ba1
Don't set CoreCLRArtifactsPath manually.
jkoritzinsky Oct 2, 2023
fdf9b1b
PR feedback
jkoritzinsky Oct 2, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions eng/readytorun.targets
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
<Project>
<PropertyGroup>
<!-- Only use mibc files if UsingToolIbcOptimization is false. Allows enabling/disabling using ibc instead of mibc data -->
<IncludePgoFiles Condition="'$(UsingToolIbcOptimization)' != 'true' and '$(EnableNgenOptimization)' == 'true' and '$(DotNetBuildFromSource)' != 'true'">true</IncludePgoFiles>
</PropertyGroup>

<Target Name="ResolveReadyToRunCompilers" DependsOnTargets="ResolveRuntimeFilesFromLocalBuild">
<ItemGroup>
<OptimizationMibcFiles Include="$(CoreCLRArtifactsPath)StandardOptimizationData.mibc" Condition="'$(DotNetBuildFromSource)' != 'true'" />
</ItemGroup>

<ItemGroup>
<PublishReadyToRunPgoFiles Condition="'$(IncludePgoFiles)' == 'true'" Include="$(CoreCLRArtifactsPath)StandardOptimizationData.mibc"/>
<PublishReadyToRunCrossgen2ExtraArgsList Condition="'$(IncludePgoFiles)' == 'true'" Include="--embed-pgo-data"/>
</ItemGroup>

<PropertyGroup>
<Crossgen2Dll>$(CoreCLRArtifactsPath)\$(BuildArchitecture)\crossgen2\crossgen2.dll</Crossgen2Dll>
Copy link
Contributor

@ivdiazsa ivdiazsa Sep 29, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT/Question: What's the criteria to define whether we write paths with backslashes, forward slashes, or .NET's Path API? I think it would be best to gradually standardize it for maximum safety and uniformity. While I prefer forward slashes, I believe using the Path API would be best to avoid potential compatibility pitfalls.

Copy link
Member

@sbomer sbomer Sep 29, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ViktorHofer has been encouraging me to move to backslashes in other PRs - it sounds like the libraries subset has standardized on that already.

Copy link
Member

@ViktorHofer ViktorHofer Sep 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this case, it would be better to use MSBuild's normalization function though as it automatically takes care of using the right directory separators (which makes the path look correct in a binlog).

Suggested change
<Crossgen2Dll>$(CoreCLRArtifactsPath)\$(BuildArchitecture)\crossgen2\crossgen2.dll</Crossgen2Dll>
<Crossgen2Dll>$([MSBuild]::NormalizePath('$(CoreCLRArtifactsPath)', '$(BuildArchitecture)', 'crossgen2', 'crossgen2.dll'))</Crossgen2Dll>


<PublishReadyToRunUseCrossgen2>true</PublishReadyToRunUseCrossgen2>
<PublishReadyToRunCrossgen2ExtraArgs>@(PublishReadyToRunCrossgen2ExtraArgsList)</PublishReadyToRunCrossgen2ExtraArgs>
<PublishReadyToRunPerfmapFormatVersion Condition="'$(PublishReadyToRunPerfmapFormatVersion)' == ''">1</PublishReadyToRunPerfmapFormatVersion>

<ScriptExt>.sh</ScriptExt>
<ScriptExt Condition="'$(OS)' == 'Windows_NT'">.cmd</ScriptExt>
</PropertyGroup>

<ItemGroup>
<Crossgen2Tool Include="$(Crossgen2Dll)"
TargetArch="$(TargetArchitecture)"
TargetOS="$(TargetOS)"
DotNetHostPath="$(RepoRoot)/dotnet$(ScriptExt)"
PerfmapFormatVersion="$(PublishReadyToRunPerfmapFormatVersion)"/>
</ItemGroup>
</Target>
</Project>
42 changes: 2 additions & 40 deletions eng/targetingpacks.targets
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,8 @@
<UseLocalTargetingRuntimePack Condition="'$(UseLocalTargetingRuntimePack)' == ''">true</UseLocalTargetingRuntimePack>
<UseLocalAppHostPack Condition="'$(UseLocalAppHostPack)' == ''">$(UseLocalTargetingRuntimePack)</UseLocalAppHostPack>
<EnableTargetingPackDownload>false</EnableTargetingPackDownload>
<!-- TODO: Enable when a 8.0.100 SDK is consumed
<EnableRuntimePackDownload>false</EnableRuntimePackDownload>
<EnableAppHostPackDownload>false</EnableAppHostPackDownload>
-->
<GenerateErrorForMissingTargetingPacks>false</GenerateErrorForMissingTargetingPacks>
</PropertyGroup>

Expand Down Expand Up @@ -99,44 +97,6 @@
</PropertyGroup>
</Target>

<!--
SDK tries to download runtime packs when RuntimeIdentifier is set, remove them from PackageDownload item.
TODO: Remove this target when an 8.0.100 SDK is consumed that respects EnableRuntimePackDownload.
-->
<Target Name="RemoveRuntimePackFromDownloadItem"
Condition="'$(UseLocalTargetingRuntimePack)' == 'true'"
AfterTargets="ProcessFrameworkReferences">
<ItemGroup>
<PackageDownload Remove="@(PackageDownload)"
Condition="'$(UsePackageDownload)' == 'true' and $([System.String]::Copy('%(Identity)').StartsWith('$(LocalFrameworkOverrideName).Runtime'))" />
<PackageReference Remove="@(PackageReference)"
Condition="'$(UsePackageDownload)' != 'true' and $([System.String]::Copy('%(Identity)').StartsWith('$(LocalFrameworkOverrideName).Runtime'))" />
<PackageDownload Remove="@(PackageDownload)"
Condition="'$(UsePackageDownload)' == 'true' and $([System.String]::Copy('%(Identity)').StartsWith('$(LocalFrameworkOverrideName).Crossgen2'))" />
<PackageReference Remove="@(PackageReference)"
Condition="'$(UsePackageDownload)' != 'true' and $([System.String]::Copy('%(Identity)').StartsWith('$(LocalFrameworkOverrideName).Crossgen2'))" />
<PackageDownload Remove="@(PackageDownload)"
Condition="'$(UsePackageDownload)' == 'true' and $([System.String]::Copy('%(Identity)').EndsWith('Microsoft.DotNet.ILCompiler'))" />
<!-- Always remove the PackageReference items as some of the packages are only via PackageReference -->
<PackageReference Remove="@(PackageReference)"
Condition="$([System.String]::Copy('%(Identity)').EndsWith('Microsoft.DotNet.ILCompiler'))" />
</ItemGroup>
</Target>

<!--
TODO: Remove this target when an 8.0.100 SDK is consumed that respects EnableAppHostPackDownload.
-->
<Target Name="RemoveAppHostPackFromDownloadItem"
Condition="'$(UseLocalAppHostPack)' == 'true'"
AfterTargets="ProcessFrameworkReferences">
<ItemGroup>
<PackageDownload Remove="@(PackageDownload)"
Condition="'$(UsePackageDownload)' == 'true' and $([System.String]::Copy('%(Identity)').StartsWith('$(LocalFrameworkOverrideName).Host'))" />
<PackageReference Remove="@(PackageReference)"
Condition="'$(UsePackageDownload)' != 'true' and $([System.String]::Copy('%(Identity)').StartsWith('$(LocalFrameworkOverrideName).Host'))" />
</ItemGroup>
</Target>

<!-- Use local targeting/runtime pack for NetCoreAppCurrent. -->
<Target Name="UpdateLocalTargetingAndRuntimePack"
Condition="'$(UseLocalTargetingRuntimePack)' == 'true'"
Expand Down Expand Up @@ -185,4 +145,6 @@
Condition="'%(RuntimeFramework.FrameworkName)' == '$(LocalFrameworkOverrideName)'" />
</ItemGroup>
</Target>

<Import Project="$(MSBuildThisFileDirectory)readytorun.targets" Condition="'$(UseLocalTargetingRuntimePack)' == 'true'" />
</Project>
12 changes: 7 additions & 5 deletions src/coreclr/tools/aot/crossgen2/crossgen2_publish.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,16 +1,19 @@
<Project>
<Project DefaultTargets="Publish;PublishItemsOutputGroup">
<!-- This project is specifically to build and publish a build of crossgen2 for shipping. -->
<PropertyGroup>
<_IsPublishing>true</_IsPublishing>
</PropertyGroup>

<Import Project="Sdk.props" Sdk="Microsoft.NET.Sdk" />

<PropertyGroup>
<!-- Can't use NativeAOT in source build yet https://github.com/dotnet/runtime/issues/66859 -->
<NativeAotSupported Condition="'$(DotNetBuildFromSource)' == 'true'">false</NativeAotSupported>
<NativeAotSupported Condition="$(OutputRID.StartsWith('tizen')) == 'true'">false</NativeAotSupported>
<PublishTrimmed>true</PublishTrimmed>
<RuntimeIdentifiers Condition="'$(DotNetBuildFromSource)' != 'true'">linux-x64;linux-musl-x64;linux-arm;linux-musl-arm;linux-arm64;linux-musl-arm64;freebsd-x64;freebsd-arm64;osx-x64;osx-arm64;win-x64;win-x86;win-arm64</RuntimeIdentifiers>
<RuntimeIdentifiers Condition="'$(DotNetBuildFromSource)' == 'true'">$(PackageRID)</RuntimeIdentifiers>
<RuntimeIdentifier>$(PackageRID)</RuntimeIdentifier>
<SelfContained>true</SelfContained>
<TargetFramework>$(NetCoreAppCurrent)</TargetFramework>
<UseLocalAppHostPack>true</UseLocalAppHostPack>
</PropertyGroup>

<Import Project="crossgen2.props" />
Expand Down Expand Up @@ -52,7 +55,6 @@
<CustomLinkerArg Condition="'$(CrossBuild)' == 'true' and '$(_hostArchitecture)' == '$(_targetArchitecture)' and '$(_IsApplePlatform)' != 'true' and '$(_hostOS)' != 'windows'" Include="--gcc-toolchain=$(ROOTFS_DIR)/usr" />
</ItemGroup>

<Import Project="$(R2ROverridePath)" Condition="'$(R2ROverridePath)' != ''" />
<Import Project="$(CoreCLRBuildIntegrationDir)Microsoft.DotNet.ILCompiler.SingleEntry.targets"
Condition="'$(NativeAotSupported)' == 'true'" />
<Import Project="$(RepositoryEngineeringDir)nativeSanitizers.targets" />
Expand Down
9 changes: 2 additions & 7 deletions src/installer/pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Crossgen2.sfxproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,13 +31,8 @@
<ItemGroup>
<ProjectReference
Include="$(RepoRoot)src/coreclr/tools/aot/crossgen2/crossgen2_publish.csproj"
AdditionalProperties="_IsPublishing=true
;RuntimeIdentifier=$(PackageRID)
;NativeAotSupported=$(NativeAotSupported)
;CoreCLRArtifactsPath=$(CoreCLRArtifactsPath)
;ObjCopyName=$(ObjCopyName)
;R2ROverridePath=$(MSBuildThisFileDirectory)ReadyToRun.targets"
Targets="Publish;PublishItemsOutputGroup"
AdditionalProperties="CoreCLRArtifactsPath=$(CoreCLRArtifactsPath)
;ObjCopyName=$(ObjCopyName)"
OutputItemType="_RawCrossgenPublishFiles"
ReferenceOutputAssembly="false" />
</ItemGroup>
Expand Down
2 changes: 1 addition & 1 deletion src/installer/pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Runtime.sfxproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,5 +27,5 @@
</ItemGroup>

<Import Project="Microsoft.NETCore.App.Runtime.props" />
<Import Project="ReadyToRun.targets" />
<Import Project="$(RepositoryEngineeringDir)readytorun.targets" />
</Project>
51 changes: 0 additions & 51 deletions src/installer/pkg/sfx/Microsoft.NETCore.App/ReadyToRun.targets
View file
Open in desktop

This file was deleted.