Skip to content

Clean up the crossgen2_publish project and local/live packs references #92826

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 16 commits into from
Oct 3, 2023
Merged

Clean up the crossgen2_publish project and local/live packs references #92826

Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
d4dffde
Inline some of the options for the new crossgen2_publish project.
jkoritzinsky Sep 29, 2023
47325f7
Resolve TODOs in targetingpacks.targets
jkoritzinsky Sep 29, 2023
1a4402d
Crossgen1 is long gone. Don't try to discover it in our override targ…
jkoritzinsky Sep 29, 2023
affe0fd
Move ReadyToRun.targets infra into the shared repo infrastructure and…
jkoritzinsky Sep 29, 2023
bfc601d
Remove extraneous property set (the same value is calculated automati…
jkoritzinsky Sep 29, 2023
0b69d8d
Remove outdated comment.
jkoritzinsky Sep 29, 2023
cffdb4d
Fix NativeAOT and installer legs
jkoritzinsky Sep 29, 2023
0e3c720
Condition turning off pack downloads based on opt-in to local pack us…
jkoritzinsky Sep 29, 2023
09eaab4
Use the LKG host instead of the 7.0 host as the fallback for NativeEx…
jkoritzinsky Sep 29, 2023
de2f5e2
Apply suggestions from code review
jkoritzinsky Sep 30, 2023
c47bb43
Merge branch 'main' of github.com:dotnet/runtime into crossgen2-publi…
jkoritzinsky Oct 2, 2023
6620cc6
Remove AdditionalProperties as they aren't needed (global properties …
jkoritzinsky Oct 2, 2023
6a29dc3
PR feedback
jkoritzinsky Oct 2, 2023
8d32750
Hook into the targets pipeline to avoid overridding targets for R2Rin…
jkoritzinsky Oct 2, 2023
aac7ba1
Don't set CoreCLRArtifactsPath manually.
jkoritzinsky Oct 2, 2023
fdf9b1b
PR feedback
jkoritzinsky Oct 2, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions eng/codeOptimization.targets
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,4 +27,17 @@
</PropertyGroup>
</Target>

<Target Name="AddReadyToRunPgoOptions" DependsOnTargets="ResolveRuntimeFilesFromLocalBuild" BeforeTargets="ResolveReadyToRunCompilers">
Copy link
Member

@ViktorHofer ViktorHofer Oct 2, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we actually depend on that target ResolveRuntimeFilesFromLocalBuild here?

Copy link
Member Author

@jkoritzinsky jkoritzinsky Oct 2, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Depending on the target helps the build cleanly error out if CoreCLR hasn't been built yet.

<PropertyGroup>
<!-- Only use mibc files if UsingToolIbcOptimization is false. Allows enabling/disabling using ibc instead of mibc data -->
<IncludeMibcFilesInReadyToRun Condition="'$(UsingToolIbcOptimization)' != 'true' and '$(EnableNgenOptimization)' == 'true' and '$(DotNetBuildFromSource)' != 'true'">true</IncludeMibcFilesInReadyToRun>
</PropertyGroup>
<ItemGroup>
<PublishReadyToRunPgoFiles Condition="'$(IncludeMibcFilesInReadyToRun)' == 'true'" Include="$(CoreCLRArtifactsPath)StandardOptimizationData.mibc"/>
</ItemGroup>

<PropertyGroup Condition="'$(IncludeMibcFilesInReadyToRun)' == 'true'">
<PublishReadyToRunCrossgen2ExtraArgs>$(PublishReadyToRunCrossgen2ExtraArgs);--embed-pgo-data</PublishReadyToRunCrossgen2ExtraArgs>
</PropertyGroup>
</Target>
</Project>
22 changes: 22 additions & 0 deletions eng/targetingpacks.targets
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,17 @@
</ItemGroup>
</Target>

<!-- Use local Crossgen2 pack for NetCoreAppCurrent. -->
<Target Name="UpdateLocalCrossgen2Pack"
Condition="'$(UseLocalCrossgen2Pack)' == 'true'"
AfterTargets="ResolveFrameworkReferences">
<ItemGroup>
<ResolvedCrossgen2Pack PackageDirectory="$(CoreCLRArtifactsPath)\$(BuildArchitecture)\crossgen2\"
Condition="$([System.String]::new('%(Identity)').StartsWith('$(LocalFrameworkOverrideName).Crossgen2'))" />
</ItemGroup>
</Target>


<!-- Use local app host pack for NetCoreAppCurrent. -->
<Target Name="UpdateLocalAppHostPack"
Condition="'$(UseLocalAppHostPack)' == 'true'"
Expand All @@ -172,4 +183,15 @@
Condition="'%(RuntimeFramework.FrameworkName)' == '$(LocalFrameworkOverrideName)'" />
</ItemGroup>
</Target>

<Target Name="UpdateLocalCrossgen2Tool"
Condition="'$(UseLocalCrossgen2Pack)' == 'true'"
AfterTargets="ResolveReadyToRunCompilers">
<ItemGroup>
<OriginalCrossgen2Tool Include="@(Crossgen2Tool)" Crossgen2Dll="$([System.IO.Path]::ChangeExtension('%(Identity)', '.dll'))" />
<Crossgen2Tool Remove="@(Crossgen2Tool)" />
<Crossgen2Tool Include="@(OriginalCrossgen2Tool->'%(Crossgen2Dll)')"
DotNetHostPath="$(DotNetTool)" />
</ItemGroup>
</Target>
</Project>
3 changes: 1 addition & 2 deletions src/coreclr/tools/aot/crossgen2/crossgen2_inbuild.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<CrossHostArch Condition="'$(CrossBuild)' == 'true' or '$(TargetArchitecture)' != '$(BuildArchitecture)' or '$(HostOS)' != '$(TargetOS)' or '$(EnableNativeSanitizers)' != ''">$(BuildArchitecture)</CrossHostArch>
<OutputPath>$(RuntimeBinDir)/$(BuildArchitecture)/crossgen2</OutputPath>
<UseAppHost>false</UseAppHost>
<OutputPath>$(RuntimeBinDir)/$(BuildArchitecture)/crossgen2/tools</OutputPath>
<TargetFramework>$(NetCoreAppToolCurrent)</TargetFramework>
</PropertyGroup>
<Import Project="crossgen2.props" />
Expand Down
12 changes: 8 additions & 4 deletions src/coreclr/tools/aot/crossgen2/crossgen2_publish.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,17 @@
<Project>
<Project DefaultTargets="Publish;PublishItemsOutputGroup">
<!-- This project is specifically to build and publish a build of crossgen2 for shipping. -->
<PropertyGroup>
<_IsPublishing>true</_IsPublishing>
</PropertyGroup>

<Import Project="Sdk.props" Sdk="Microsoft.NET.Sdk" />

<PropertyGroup>
<!-- Can't use NativeAOT in source build yet https://github.com/dotnet/runtime/issues/66859 -->
<NativeAotSupported Condition="'$(DotNetBuildFromSource)' == 'true'">false</NativeAotSupported>
<NativeAotSupported Condition="$(OutputRID.StartsWith('tizen')) == 'true'">false</NativeAotSupported>
<PublishTrimmed>true</PublishTrimmed>
<RuntimeIdentifiers Condition="'$(DotNetBuildFromSource)' != 'true'">linux-x64;linux-musl-x64;linux-arm;linux-musl-arm;linux-arm64;linux-musl-arm64;freebsd-x64;freebsd-arm64;osx-x64;osx-arm64;win-x64;win-x86;win-arm64</RuntimeIdentifiers>
<RuntimeIdentifiers Condition="'$(DotNetBuildFromSource)' == 'true'">$(PackageRID)</RuntimeIdentifiers>
<RuntimeIdentifier>$(PackageRID)</RuntimeIdentifier>
<SelfContained>true</SelfContained>
<TargetFramework>$(NetCoreAppCurrent)</TargetFramework>
<UseLocalAppHostPack>true</UseLocalAppHostPack>
Expand All @@ -33,6 +37,7 @@
</ItemGroup>

<Import Project="$(RepositoryEngineeringDir)targetingpacks.targets" />
<Import Project="$(RepositoryEngineeringDir)codeOptimization.targets" />

<PropertyGroup Condition="'$(NativeAotSupported)' == 'true'">
<IlcToolsPath>$(CoreCLRILCompilerDir)</IlcToolsPath>
Expand All @@ -52,7 +57,6 @@
<CustomLinkerArg Condition="'$(CrossBuild)' == 'true' and '$(_hostArchitecture)' == '$(_targetArchitecture)' and '$(_IsApplePlatform)' != 'true' and '$(_hostOS)' != 'windows'" Include="--gcc-toolchain=$(ROOTFS_DIR)/usr" />
</ItemGroup>

<Import Project="$(R2ROverridePath)" Condition="'$(R2ROverridePath)' != ''" />
<Import Project="$(CoreCLRBuildIntegrationDir)Microsoft.DotNet.ILCompiler.SingleEntry.targets"
Condition="'$(NativeAotSupported)' == 'true'" />
<Import Project="$(RepositoryEngineeringDir)nativeSanitizers.targets" />
Expand Down
7 changes: 0 additions & 7 deletions src/installer/pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Crossgen2.sfxproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,13 +31,6 @@
<ItemGroup>
<ProjectReference
Include="$(RepoRoot)src/coreclr/tools/aot/crossgen2/crossgen2_publish.csproj"
AdditionalProperties="_IsPublishing=true
;RuntimeIdentifier=$(PackageRID)
;NativeAotSupported=$(NativeAotSupported)
;CoreCLRArtifactsPath=$(CoreCLRArtifactsPath)
;ObjCopyName=$(ObjCopyName)
;R2ROverridePath=$(MSBuildThisFileDirectory)ReadyToRun.targets"
Targets="Publish;PublishItemsOutputGroup"
OutputItemType="_RawCrossgenPublishFiles"
ReferenceOutputAssembly="false" />
</ItemGroup>
Expand Down
3 changes: 3 additions & 0 deletions src/installer/pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Runtime.props
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
<IncludeWerRelatedKeys>true</IncludeWerRelatedKeys>
<IncludeBreadcrumbStoreFolder>true</IncludeBreadcrumbStoreFolder>
<MacOSPackageDescription>The .NET Shared Framework</MacOSPackageDescription>
<UseLocalCrossgen2Pack>$(PublishReadyToRun)</UseLocalCrossgen2Pack>
</PropertyGroup>

<PropertyGroup Condition="'$(BuildNativeAOTRuntimePack)' == 'true'">
Expand Down Expand Up @@ -178,6 +179,8 @@
<Import Project="$(Crossgen2SdkOverridePropsPath)" Condition="'$(BuildNativeAOTRuntimePack)' != 'true' and '$(Crossgen2SdkOverridePropsPath)' != ''" />
<Import Project="Sdk.targets" Sdk="Microsoft.NET.Sdk" />
<Import Project="Sdk.targets" Sdk="Microsoft.DotNet.SharedFramework.Sdk" />
<Import Project="$(RepositoryEngineeringDir)targetingpacks.targets" />
<Import Project="$(RepositoryEngineeringDir)codeOptimization.targets" />
<Import Project="$(Crossgen2SdkOverrideTargetsPath)" Condition="'$(BuildNativeAOTRuntimePack)' != 'true' and '$(Crossgen2SdkOverrideTargetsPath)' != ''" />
<PropertyGroup>
<PublishReadyToRunComposite Condition="$(ForcePublishReadyToRunComposite) == 'true'">true</PublishReadyToRunComposite>
Expand Down
39 changes: 2 additions & 37 deletions src/installer/pkg/sfx/Microsoft.NETCore.App/ReadyToRun.targets
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,51 +1,16 @@
<Project>
<Target Name="ResolveReadyToRunCompilers" DependsOnTargets="ResolveRuntimeFilesFromLocalBuild">
<ItemGroup>
<OptimizationMibcFiles Include="$(CoreCLRArtifactsPath)StandardOptimizationData.mibc" Condition="'$(DotNetBuildFromSource)' != 'true'" />
</ItemGroup>

<ItemGroup>
<PublishReadyToRunCrossgen2ExtraArgsList Include="--targetarch:$(TargetArchitecture) --targetos:$(TargetOS)"/>

<!-- Only use mibc files if UsingToolIbcOptimization is false. Allows enabling/disabling using ibc instead of mibc data -->
<PublishReadyToRunCrossgen2ExtraArgsList Condition="'$(UsingToolIbcOptimization)' != 'true' and '$(EnableNgenOptimization)' == 'true'" Include="@(OptimizationMibcFiles->'-m:%(Identity)')"/>
<PublishReadyToRunCrossgen2ExtraArgsList Condition="'$(UsingToolIbcOptimization)' != 'true' and '$(EnableNgenOptimization)' == 'true'" Include="--embed-pgo-data"/>
</ItemGroup>

<!-- The following property group can be simplified once runtime repo switches over to SDK 6.0 drop -->
<PropertyGroup>
<Crossgen2Dll>$(CoreCLRArtifactsPath)\$(BuildArchitecture)\crossgen2\crossgen2.dll</Crossgen2Dll>

<PublishReadyToRunUseCrossgen2>true</PublishReadyToRunUseCrossgen2>
<PublishReadyToRunCrossgen2ExtraArgs>@(PublishReadyToRunCrossgen2ExtraArgsList)</PublishReadyToRunCrossgen2ExtraArgs>
<PublishReadyToRunPerfmapFormatVersion Condition="'$(PublishReadyToRunPerfmapFormatVersion)' == ''">1</PublishReadyToRunPerfmapFormatVersion>

<ScriptExt>.sh</ScriptExt>
<ScriptExt Condition="'$(OS)' == 'Windows_NT'">.cmd</ScriptExt>
</PropertyGroup>

<ItemGroup Condition="'$(RuntimeFlavor)' != 'Mono'">
<_crossTargetJit Include="@(CoreCLRCrossTargetFiles)" Condition="'%(FileName)' == '$(LibPrefix)clrjit' and '%(Extension)' == '$(LibSuffix)'" />
<_clrjit Include="@(RuntimeFiles)" Condition="'%(FileName)' == '$(LibPrefix)clrjit' and '%(Extension)' == '$(LibSuffix)'" />
<_crossTargetCrossgen Include="@(CoreCLRCrossTargetFiles)" Condition="'%(FileName)' == 'crossgen' and '%(Extension)' == '$(ExeSuffix)'" />
<_crossgen Include="@(RuntimeFiles)" Condition="'%(FileName)' == 'crossgen' and '%(Extension)' == '$(ExeSuffix)'" />
</ItemGroup>
<ItemGroup Condition="'@(_crossTargetJit)' != '' and '@(_crossTargetCrossgen)' != ''">
<CrossgenTool Include="@(_crossTargetCrossgen->ClearMetadata())"
JitPath="@(_crossTargetJit)"
DiaSymReader="$(_diaSymReaderPath)" />
</ItemGroup>
<ItemGroup Condition="'@(_crossTargetJit)' == '' and '@(_crossTargetCrossgen)' == ''">
<CrossgenTool Include="@(_crossgen->ClearMetadata())"
JitPath="@(_clrjit)"
DiaSymReader="$(_diaSymReaderPath)" />
</ItemGroup>
<ItemGroup>
<Crossgen2Tool Include="$(Crossgen2Dll)"
TargetArch="$(TargetArchitecture)"
TargetOS="$(TargetOS)"
DotNetHostPath="$(RepoRoot)/dotnet$(ScriptExt)"
DotNetHostPath="$(DotNetTool)"
PerfmapFormatVersion="$(PublishReadyToRunPerfmapFormatVersion)"/>
</ItemGroup>
</Target>
</Project>
</Project>
13 changes: 4 additions & 9 deletions ...raries/System.Runtime.InteropServices/tests/TestAssets/NativeExports/NativeExports.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,16 +23,11 @@
<Compile Include="$(CommonPath)DisableRuntimeMarshalling.cs" Link="Common\DisableRuntimeMarshalling.cs" />
</ItemGroup>

<!-- Until we use the live app host, use a prebuilt from the 7.0 SDK.
<!-- Until we use the live app host, use a prebuilt from the SDK.
Issue: https://github.com/dotnet/runtime/issues/58109. -->
<ItemGroup Condition="'$(UseLocalAppHostPack)' != 'true'">
<KnownAppHostPack Include="Microsoft.NETCore.App"
ExcludedRuntimeIdentifiers="android"
AppHostPackNamePattern="Microsoft.NETCore.App.Host.**RID**"
AppHostPackVersion="7.0.0"
AppHostRuntimeIdentifiers="linux-arm;linux-arm64;linux-musl-arm64;linux-musl-x64;linux-x64;osx-x64;rhel.6-x64;tizen.4.0.0-armel;tizen.5.0.0-armel;win-arm64;win-x64;win-x86;linux-musl-arm;osx-arm64;linux-s390x;linux-bionic-arm;linux-bionic-arm64;linux-bionic-x64;linux-bionic-x86"
TargetFramework="$(NetCoreAppCurrent)"
Condition="'@(KnownAppHostPack)' == '' or !@(KnownAppHostPack->AnyHaveMetadataValue('TargetFramework', '$(NetCoreAppCurrent)'))" />
<ItemGroup Condition="'$(UseLocalAppHostPack)' != 'true' and '$(NetCoreAppToolCurrent)' != '$(NetCoreAppCurrent)'">
<KnownAppHostPack Include="@(KnownAppHostPack->WithMetadataValue('Identity', 'Microsoft.NETCore.App')->WithMetadataValue('TargetFramework', '$(NetCoreAppToolCurrent)'))"
TargetFramework="$(NetCoreAppCurrent)" />
</ItemGroup>

<ItemGroup>
Expand Down