Skip to content

Add new Pipeline for Running Libs Tests with TestReadyToRun #91229

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 54 commits into from
Dec 14, 2023
Merged

Add new Pipeline for Running Libs Tests with TestReadyToRun #91229

Show file tree
Hide file tree
Changes from 53 commits
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
58471ac
New beginning for the TestReadyToRun for the libraries.
ivdiazsa Aug 28, 2023
d424419
EXPERIMENTING: Added script extensions, archive extensions, and other…
ivdiazsa Aug 28, 2023
cbae83d
EXPERIMENTING: Forgot to add some curly braces...
ivdiazsa Aug 28, 2023
bbcbda5
Recovered some variables that were being eaten and removed some other
ivdiazsa Aug 28, 2023
74c393d
EXPERIMENTING: Added OSX and Windows to see if the CI will deign run
ivdiazsa Aug 28, 2023
f39ec24
EXPERIMENT ONLY: Added some missing parameters for
ivdiazsa Aug 29, 2023
e76d7b2
Is this it?????
ivdiazsa Aug 29, 2023
800bf85
Now that we know the libraries tests can be built properly, let's add
ivdiazsa Aug 30, 2023
4888506
Discovered that when packs are involved, everything has to be in the
ivdiazsa Aug 30, 2023
f95be94
Disabled the R2R-incompatible tests when TestReadyToRun is present.
ivdiazsa Aug 31, 2023
ce0beb7
Changed 'TargetFrameworks' to use 'net8.0' instead of '$(NetCoreAppCu…
ivdiazsa Sep 5, 2023
084dcd2
CI versioning nonsense again...
ivdiazsa Sep 5, 2023
d6145d9
Merge branch 'main' into r2r-libs-take-3
ivdiazsa Sep 5, 2023
7a98a59
Still does not work. Just saving my changes.
ivdiazsa Sep 19, 2023
57a1ed0
Merge with main
ivdiazsa Sep 19, 2023
9c9845c
Merge with main
ivdiazsa Sep 27, 2023
201c744
Merge branch 'main' into r2r-libs-take-3
ivdiazsa Sep 29, 2023
0e5318e
Merge branch 'main' into r2r-libs-take-3
ivdiazsa Oct 4, 2023
94b209d
Merged with Main AND IT WORKED LOCALLY!
ivdiazsa Oct 19, 2023
baf03ff
Updated the 'extraSteps' and 'extraStepsTemplate' to the new
ivdiazsa Oct 20, 2023
bd083f9
Reenabled all the pipelines and moved the TestReadyToRun one to the
ivdiazsa Oct 20, 2023
70a5310
Removed repeated arguments from the post build steps.
ivdiazsa Oct 20, 2023
29bdc40
Reverted weird change in System.Runtime/Directory.Build.props
ivdiazsa Oct 23, 2023
0dafa7a
Removed unnecessary import in Build.proj
ivdiazsa Oct 23, 2023
0987abf
As per Jeremy's feedback, now trying to set the TestReadyToRun template
ivdiazsa Oct 23, 2023
c050621
Reenabled the problematic tests to see if they still fail. If yes, then
ivdiazsa Oct 23, 2023
21f95ef
Merge branch 'main' into r2r-libs-take-3
ivdiazsa Oct 24, 2023
59af6bb
Found a workaround for the libraries tests!
ivdiazsa Oct 30, 2023
52875a7
Forgot to specify the test group for the new pipelines.
ivdiazsa Oct 30, 2023
43eee05
Helix doesn't want to work so how about we try running them as we
ivdiazsa Oct 30, 2023
4666ac0
Disabled tests incompatible with TestReadyToRun.
ivdiazsa Oct 30, 2023
79a9028
Changed the test disabling mechanism to the already defined
ivdiazsa Oct 31, 2023
e95e20d
Moved the 'UseLocalAppHostPack' setting to eng/tests.singlefile.targets.
ivdiazsa Oct 31, 2023
e857281
Temporarily restored 'UseLocalAppHostPack' to
ivdiazsa Nov 1, 2023
d193ab5
Extended the TestReadyToRun pipelines timeout because building all
ivdiazsa Nov 2, 2023
7a3a1d5
Testing removing the .dbg files since those are responsible for most
ivdiazsa Nov 8, 2023
cd94d5a
Removed the debugging files from the "To Copy" lists, rather than
ivdiazsa Nov 10, 2023
3c54a84
Replaced Remove+Condition with Remove->WithMetadataValue for better
ivdiazsa Nov 10, 2023
879e5ea
Merge branch 'main' into r2r-libs-take-3
ivdiazsa Nov 14, 2023
11cdb6b
Tried adding to pretest setup as per Viktor's suggestion.
ivdiazsa Nov 14, 2023
19bb85f
Testing something...
ivdiazsa Nov 20, 2023
81cb9e3
Fixed the R2R-precompiled pack not built on time.
ivdiazsa Nov 21, 2023
47760b3
Moved the R2R-precompiled runtime generation from Subsets.props to pr…
ivdiazsa Nov 22, 2023
0728170
Added the ArchiveTests property to get the test artifacts packed
ivdiazsa Nov 27, 2023
4262460
Excluded dotnet-host installers in TestReadyToRun builds because we
ivdiazsa Nov 27, 2023
403a5f9
Disabled System.Runtime.Loader.Tests because they can't be run due to
ivdiazsa Nov 30, 2023
797f2c0
Disabled the other faulty System.Runtime.Loader* test, and linked to a
ivdiazsa Nov 30, 2023
048d35e
Fixed the live-ref-pack getting removed from publish files.
ivdiazsa Dec 5, 2023
9939472
Added a copy function
ivdiazsa Dec 6, 2023
033508d
Forgot to actually set the files to copy.
ivdiazsa Dec 6, 2023
54088ef
Merge branch 'main' into r2r-libs-take-3
ivdiazsa Dec 7, 2023
7a23788
Testing something...
ivdiazsa Dec 7, 2023
7c97aa7
Changed the TestReadyToRun pipeline dynamics to only enable the tests
ivdiazsa Dec 12, 2023
f54af82
Added comments to tests.singlefile.targets, and disabled the
ivdiazsa Dec 13, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions eng/pipelines/common/platform-matrix.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ parameters:
jobs:

# Linux arm

- ${{ if or(containsValue(parameters.platforms, 'linux_arm'), in(parameters.platformGroup, 'all', 'gcstress')) }}:
- template: xplat-setup.yml
parameters:
Expand All @@ -45,6 +46,7 @@ jobs:
${{ insert }}: ${{ parameters.jobParameters }}

# Linux armv6

- ${{ if containsValue(parameters.platforms, 'linux_armv6') }}:
- template: xplat-setup.yml
parameters:
Expand Down
23 changes: 23 additions & 0 deletions eng/pipelines/coreclr/crossgen2.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,29 @@ extends:
displayNameArgs: R2R_CG2
liveLibrariesBuildConfig: Release

- template: /eng/pipelines/common/platform-matrix.yml
parameters:
jobTemplate: /eng/pipelines/common/global-build-job.yml
helixQueuesTemplate: /eng/pipelines/libraries/helix-queues-setup.yml
buildConfig: Release
platforms:
- linux_x64
- osx_x64
- windows_x64
jobParameters:
testGroup: innerloop
buildArgs: -s clr+libs+libs.tests
-c $(_BuildConfig)
/p:TestReadyToRun=true
/p:ArchiveTests=true
nameSuffix: TestReadyToRun_Libraries
timeoutInMinutes: 360
postBuildSteps:
- template: /eng/pipelines/libraries/helix.yml
parameters:
creator: dotnet-bot
testRunNamePrefixSuffix: TestReadyToRun_$(_BuildConfig)

# Run pri0 tests with hot/cold splitting enabled (only supported on x64 at the moment)
# TODO: test on arm64 once supported
- template: /eng/pipelines/common/platform-matrix.yml
Expand Down
77 changes: 77 additions & 0 deletions eng/testing/tests.singlefile.targets
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,10 @@
<DefineConstants>$(DefineConstants);SINGLE_FILE_TEST_RUNNER</DefineConstants>
</PropertyGroup>

<PropertyGroup Condition="'$(TestReadyToRun)' == 'true'">
<DefineConstants>$(DefineConstants);TEST_READY_TO_RUN_COMPILED</DefineConstants>
</PropertyGroup>

<Import Project="$(CoreCLRBuildIntegrationDir)Microsoft.DotNet.ILCompiler.SingleEntry.targets" Condition="'$(TestNativeAot)' == 'true'" />

<ItemGroup Condition="'$(TestNativeAot)' == 'true'">
Expand Down Expand Up @@ -106,6 +110,79 @@
</ItemGroup>
</Target>

<Target Name="ExcludeExistingR2RBinaries"
Copy link
Member

@ViktorHofer ViktorHofer Dec 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please describe these targets via xml comments. I don't understand what the following targets are doing or why we need them so a comment would be appreciated: ExcludeExistingR2RBinaries, AddExistingR2RBinariesReferencesForCrossgen2, CopyExistingR2RBinaries and CopyLiveRefPackIfPresent.

Copy link
Member

@ericstj ericstj Dec 13, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree with @ViktorHofer here - copies like this can cause extra work and even create incremental build issues when they use wildcards like this. It'd be good to understand them.

Condition="'$(TestReadyToRun)' == 'true'"
BeforeTargets="_PrepareForReadyToRunCompilation">
<PropertyGroup>
<ArtifactsNetCoreAppBundlePath>$(ArtifactsObjDir)Microsoft.NETCore.App.Bundle/</ArtifactsNetCoreAppBundlePath>
<ArtifactsNetCoreAppBundlePath>$(ArtifactsNetCoreAppBundlePath)$(Configuration)/$(NetCoreAppCurrent)/$(OutputRID)/output/</ArtifactsNetCoreAppBundlePath>
<ArtifactsNetCoreAppBundlePath>$(ArtifactsNetCoreAppBundlePath)shared/$(MicrosoftNetCoreAppFrameworkName)/$(PackageVersion)/</ArtifactsNetCoreAppBundlePath>
</PropertyGroup>

<ItemGroup>
<_BundleAssembliesToCopy Include="$(ArtifactsNetCoreAppBundlePath)*.dll" />
<ResolvedFileToPublish Remove="@(_BundleAssembliesToCopy)" MatchOnMetadata="Filename" />
</ItemGroup>
</Target>

<Target Name="AddExistingR2RBinariesReferencesForCrossgen2"
Condition="'$(TestReadyToRun)' == 'true'"
AfterTargets="_PrepareForReadyToRunCompilation">
<ItemGroup>
<_ReadyToRunAssembliesToReference Include="@(_BundleAssembliesToCopy)" />
</ItemGroup>
</Target>

<Target Name="RemoveDbgBinsFromTestR2ROutput"
Condition="'$(TestReadyToRun)' == 'true'"
BeforeTargets="_CopyFilesMarkedCopyLocal">
<ItemGroup>
<ReferenceCopyLocalPaths
Remove="@(ReferenceCopyLocalPaths->WithMetadataValue('Extension', '.dbg'))" />
</ItemGroup>
</Target>

<Target Name="RemoveDbgBinsFromTestR2RPublish"
Condition="'$(TestReadyToRun)' == 'true'"
BeforeTargets="_CopyResolvedFilesToPublishPreserveNewest">
<ItemGroup>
<_ResolvedFileToPublishPreserveNewest
Remove="@(_ResolvedFileToPublishPreserveNewest->WithMetadataValue('Extension', '.dbg'))" />
</ItemGroup>
</Target>

<Target Name="CopyExistingR2RBinaries"
Condition="'$(TestReadyToRun)' == 'true'"
AfterTargets="_CopyResolvedFilesToPublishAlways">

<Copy SourceFiles="@(_BundleAssembliesToCopy)"
DestinationFolder="$(PublishDir)"
OverwriteReadOnlyFiles="$(OverwriteReadOnlyFiles)"
Retries="$(CopyRetryCount)"
RetryDelayMilliseconds="$(CopyRetryDelayMilliseconds)"
UseHardlinksIfPossible="$(CreateHardLinksForPublishFilesIfPossible)"
UseSymboliclinksIfPossible="$(CreateSymbolicLinksForPublishFilesIfPossible)" />

</Target>

<Target Name="CopyLiveRefPackIfPresent"
Condition="'$(TestReadyToRun)' == 'true'"
AfterTargets="CopyExistingR2RBinaries">

<ItemGroup>
<OutDirLiveRefPackFiles Include="$(OutDir)live-ref-pack/*" />
</ItemGroup>

<Copy SourceFiles="@(OutDirLiveRefPackFiles)"
DestinationFolder="$(PublishDir)live-ref-pack"
OverwriteReadOnlyFiles="$(OverwriteReadOnlyFiles)"
Retries="$(CopyRetryCount)"
RetryDelayMilliseconds="$(CopyRetryDelayMilliseconds)"
UseHardlinksIfPossible="$(CreateHardLinksForPublishFilesIfPossible)"
UseSymboliclinksIfPossible="$(CreateSymbolicLinksForPublishFilesIfPossible)" />

</Target>

<Target Name="__UpdateExcludedAssembliesFromSingleFile"
Inputs="ExcludeFromSingleFile"
Outputs="ResolvedFileToPublish"
Expand Down
4 changes: 2 additions & 2 deletions src/installer/pkg/sfx/bundle/Microsoft.NETCore.App.Bundle.bundleproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,8 @@

<ItemGroup>
<BundleComponentReference Include="../Microsoft.NETCore.App/Microsoft.NETCore.App.Runtime.sfxproj" />
<BundleComponentReference Include="../installers/dotnet-hostfxr.proj" />
<BundleComponentReference Include="../installers/dotnet-host.proj" />
<BundleComponentReference Include="../installers/dotnet-hostfxr.proj" Condition="'$(TestReadyToRun)' != 'true'" />
<BundleComponentReference Include="../installers/dotnet-host.proj" Condition="'$(TestReadyToRun)' != 'true'" />
</ItemGroup>

<Target Name="PublishToDisk">
Expand Down
11 changes: 11 additions & 0 deletions src/libraries/Common/tests/SingleFileTestRunner/SingleFileTestRunner.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,17 @@ public static int Main(string[] args)
// The current RemoteExecutor implementation is not compatible with the SingleFileTestRunner.
Environment.SetEnvironmentVariable("DOTNET_REMOTEEXECUTOR_SUPPORTED", "0");

// To detect ReadyToRun testing mode, we set a constant in
// eng/testing/tests.singlefile.targets, which we use in the following
// preprocessor directive. In the case that it is defined, we set an
// environment variable that we consume later to implement
// PlatformDetection.IsReadyToRunCompiled. This last value is used for the
// [ActiveIssue] annotations designed to exclude tests from running.

#if TEST_READY_TO_RUN_COMPILED
Environment.SetEnvironmentVariable("TEST_READY_TO_RUN_MODE" ,"1");
#endif

var diagnosticSink = new ConsoleDiagnosticMessageSink();
var testsFinished = new TaskCompletionSource();
var testSink = new TestMessageSink();
Expand Down
2 changes: 2 additions & 0 deletions src/libraries/Common/tests/TestUtilities/System/PlatformDetection.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -211,6 +211,8 @@ private static bool GetLinqExpressionsBuiltWithIsInterpretingOnly()
public static bool HasHostExecutable => HasAssemblyFiles; // single-file don't have a host
public static bool IsSingleFile => !HasAssemblyFiles;

public static bool IsReadyToRunCompiled => Environment.GetEnvironmentVariable("TEST_READY_TO_RUN_MODE") == "1";

private static volatile Tuple<bool> s_lazyNonZeroLowerBoundArraySupported;
public static bool IsNonZeroLowerBoundArraySupported
{
Expand Down
4 changes: 4 additions & 0 deletions src/libraries/Directory.Build.props
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,10 @@
<EnableCoverageSupport Condition="'$(ContinuousIntegrationBuild)' != 'true'">true</EnableCoverageSupport>
</PropertyGroup>

<PropertyGroup Condition="'$(TestReadyToRun)' == 'true'">
Copy link
Member

@MichalStrehovsky MichalStrehovsky Oct 31, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's zero references to TestReadyToRun in src/libraries. Should all of TestReadyToRun handling stay in eng/? tests.singlefile.targets looks more suitable.

Copy link
Contributor Author

@ivdiazsa ivdiazsa Oct 31, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried moving it, but then the build fails in other places because it can't find the apphost. Like for example, this one:

/path/to/runtime/.dotnet/sdk/8.0.100-rtm.23506.1/Microsoft.Common.CurrentVersion.targets(5189,5): error MSB3030: Could not copy the file "/path/to/runtime/artifacts/obj/System.Runtime.Tests/Release/net9.0-unix/apphost" because it was not found. [/path/to/runtime/src/libraries/System.Runtime/tests/System.Runtime.Tests.csproj::TargetFramework=net9.0-unix]
    0 Warning(s)
    1 Error(s)

In this case, this means UseLocalAppHostPack is set to false, i.e., it is not using it when building the tests and that's where we need it. We don't need it for clr or libs, just libs.tests. So I'd like to ask if you had another possible solution, if adding it to Directory.Build.props is not right.

<UseLocalAppHostPack>true</UseLocalAppHostPack>
</PropertyGroup>

<!-- To enable the interpreter for mono desktop, we need to pass an env switch -->
<PropertyGroup>
<MonoEnvOptions Condition="'$(MonoEnvOptions)' == '' and '$(TargetsMobile)' != 'true' and '$(MonoForceInterpreter)' == 'true'">--interpreter</MonoEnvOptions>
Expand Down
1 change: 1 addition & 0 deletions src/libraries/System.Runtime/tests/System.Runtime.Tests/System/EnumTests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1599,6 +1599,7 @@ public void GetValues_EnumTypeNotEnum_ThrowsArgumentException(Type enumType)
private class ClassWithEnumConstraint<T> where T : Enum { }

[Fact]
[ActiveIssue("https://github.com/dotnet/runtime/issues/94189", typeof(PlatformDetection), nameof(PlatformDetection.IsReadyToRunCompiled))]
public void EnumConstraint_ThrowsArgumentException()
{
Type genericArgumentWithEnumConstraint = typeof(ClassWithEnumConstraint<>).GetGenericArguments()[0];
Expand Down
1 change: 1 addition & 0 deletions src/libraries/System.Runtime/tests/System.Runtime.Tests/System/Runtime/JitInfoTests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,7 @@ public void JitInfoIsNotPopulated()

[ConditionalFact(typeof(PlatformDetection), nameof(PlatformDetection.IsReflectionEmitSupported))]
[SkipOnMono("Mono does not track thread specific JIT information")]
[ActiveIssue("https://github.com/dotnet/runtime/issues/94189", typeof(PlatformDetection), nameof(PlatformDetection.IsReadyToRunCompiled))]
public void JitInfoCurrentThreadIsPopulated()
{
TimeSpan t1_beforeCompilationTime = TimeSpan.Zero;
Expand Down
1 change: 1 addition & 0 deletions src/libraries/System.Runtime/tests/System.Runtime.Tests/System/Type/TypeTests.Get.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ public static IEnumerable<object[]> GetInterface_TestData()

[Theory]
[MemberData(nameof(GetInterface_TestData))]
[ActiveIssue("https://github.com/dotnet/runtime/issues/94189", typeof(PlatformDetection), nameof(PlatformDetection.IsReadyToRunCompiled))]
public void GetInterface_Invoke_ReturnsExpected(Type type, string name, bool ignoreCase, Type expected)
{
if (!ignoreCase)
Expand Down
9 changes: 8 additions & 1 deletion src/libraries/pretest.proj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,14 @@
<ProjectReference Include="$(MonoProjectRoot)wasm\symbolicator\WasmSymbolicator.csproj" Condition="'$(TargetOS)' == 'browser'" />

<!-- needed to test workloads for wasm -->
<ProjectReference Include="$(InstallerProjectRoot)pkg\sfx\Microsoft.NETCore.App\Microsoft.NETCore.App.Runtime.sfxproj" Pack="true" Condition="'$(TargetOS)' == 'browser' or '$(TargetOS)' == 'wasi'" />
<ProjectReference Include="$(InstallerProjectRoot)pkg\sfx\Microsoft.NETCore.App\Microsoft.NETCore.App.Runtime.sfxproj"
Pack="true"
Condition="'$(TargetOS)' == 'browser' or '$(TargetOS)' == 'wasi'" />
</ItemGroup>

<ItemGroup Condition="'$(TestReadyToRun)' == 'true'">
<ProjectReference Include="$(InstallerProjectRoot)pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Runtime.sfxproj" />
<ProjectReference Include="$(InstallerProjectRoot)pkg/sfx/bundle/Microsoft.NETCore.App.Bundle.bundleproj" />
Comment on lines +35 to +36
Copy link
Member

@jkoritzinsky jkoritzinsky Nov 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can use the PublishToDisk target with the OutputPath property pointing to where you want the project dumped on disk on just the bundleproj and that will dump the whole layout to disk with R2R'd runtime assemblies.

Copy link
Member

@ViktorHofer ViktorHofer Nov 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is that required? I see that workload tests only have the ProjectReference with the Pack metadata.

Copy link
Member

@jkoritzinsky jkoritzinsky Nov 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure how workload tests work, but I wouldn't be amazed if they need the NuGet packages or the installers or the archives.

This is another option that I built into the Shared Framework SDK to make it easier to dump the layout to disk wherever you want and not having to guess where the output will be.

Copy link
Member

@ViktorHofer ViktorHofer Nov 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it. What's the default output location of the bundle project? Is it put to disk somewhere without invoking the target?

Copy link
Member

@jkoritzinsky jkoritzinsky Nov 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Shared Framework SDK by default doesn't actually lay it out on disk. The Archives and Installers NuGet packages from dotnet/arcade use this target to put it into the intermediate output path for the project and then use that to produce their output.

Copy link
Member

@ViktorHofer ViktorHofer Nov 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. For the sake of static graph (which disallows calling custom targets in a P2P protocol), can we enable a switch so that the target is automatically sequenced into the build? The project's output directory would be a good default place.

Copy link
Member

@jkoritzinsky jkoritzinsky Dec 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should be able to add the target to the Bundle.bundleproj project and sequence it into the build, but then we still end up building the installers (which is what's causing the build failures that prompted this suggestion).

</ItemGroup>

<Import Project="$(RepositoryEngineeringDir)testing\wasm-provisioning.targets"
Expand Down
34 changes: 33 additions & 1 deletion src/libraries/tests.proj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -597,6 +597,31 @@
-->
</ItemGroup>

<!--
There is a decent number of hidden tests that fail with TestReadyToRun, and
it's proven to be a neverending task to flush all of them at once. So, we will
start by only enabling the tests we have confirmed work correctly, and we will
gradually enable the rest in subsequent PR's.
Tracking Issue for this work item: https://github.com/dotnet/runtime/issues/95928
-->
<ItemGroup Condition="'$(TestReadyToRun)' == 'true'">
<TestReadyToRunProjects Include="$(MSBuildThisFileDirectory)System.*/tests/**/*.Tests.csproj" />

<TestReadyToRunProjects Remove="$(MSBuildThisFileDirectory)System.ComponentModel.Composition/tests/System.ComponentModel.Composition.Tests.csproj" />
<TestReadyToRunProjects Remove="$(MSBuildThisFileDirectory)System.Composition/tests/System.Composition.Tests.csproj" />
<TestReadyToRunProjects Remove="$(MSBuildThisFileDirectory)System.Console/tests/System.Console.Tests.csproj" />
<TestReadyToRunProjects Remove="$(MSBuildThisFileDirectory)System.Runtime/tests/System.Dynamic.Runtime.Tests/System.Dynamic.Runtime.Tests.csproj" />
<TestReadyToRunProjects Remove="$(MSBuildThisFileDirectory)System.IO.FileSystem.Watcher/tests/System.IO.FileSystem.Watcher.Tests.csproj" />
<TestReadyToRunProjects Remove="$(MSBuildThisFileDirectory)System.Runtime/tests/System.Reflection.Tests/System.Reflection.Tests.csproj" />
<TestReadyToRunProjects Remove="$(MSBuildThisFileDirectory)System.Reflection.TypeExtensions/tests/System.Reflection.TypeExtensions.Tests.csproj" />
<TestReadyToRunProjects Remove="$(MSBuildThisFileDirectory)System.Runtime/tests/System.Runtime.Extensions.Tests/System.Runtime.Extensions.Tests.csproj" />
<TestReadyToRunProjects Remove="$(MSBuildThisFileDirectory)System.Runtime.Loader/tests/DefaultContext/System.Runtime.Loader.DefaultContext.Tests.csproj" />
<TestReadyToRunProjects Remove="$(MSBuildThisFileDirectory)System.Runtime.Loader/tests/System.Runtime.Loader.Tests.csproj" />
<TestReadyToRunProjects Remove="$(MSBuildThisFileDirectory)System.Text.RegularExpressions/tests/FunctionalTests/System.Text.RegularExpressions.Tests.csproj" />
<TestReadyToRunProjects Remove="$(MSBuildThisFileDirectory)System.Threading.Thread/tests/System.Threading.Thread.Tests.csproj" />
</ItemGroup>

<ItemGroup>
<ProjectReference Condition="'$(RunSmokeTestsOnly)' == 'true'"
Include="@(SmokeTestProject)" />
Expand All @@ -605,9 +630,16 @@
BuildInParallel="false" />
<ProjectReference Condition="'$(RunGrpcTestsOnly)' == 'true'"
Include="@(GrpcTestProject)" />
<ProjectReference Condition="'$(TestReadyToRun)' == 'true'"
Include="@(TestReadyToRunProjects)" />
</ItemGroup>

<ItemGroup Condition="'$(RunSmokeTestsOnly)' != 'true' and '$(RunGrpcTestsOnly)' != 'true' and '$(RunHighAOTResourceRequiringTestsOnly)' != 'true' and '$(RunLimitedSetOfTests)' != 'true'">
<ItemGroup Condition="'$(RunSmokeTestsOnly)' != 'true'
Copy link
Member

@MichalStrehovsky MichalStrehovsky Dec 13, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please do not make this more complicated than it already is.

The existing ProjectExclusions ItemGroup should be exactly what you need here.

Copy link
Contributor Author

@ivdiazsa ivdiazsa Dec 13, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok will change that. Just worth mentioning I thought you all didn't want to change that further because I originally disabled the bad tests with ProjectExclusions, and was told that was not the way to do it...

Copy link
Member

@MichalStrehovsky MichalStrehovsky Dec 14, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I originally disabled the bad tests with ProjectExclusions, and was told that was not the way to do it...

Can you point me to the comment asking not to use ProjectExclusion? I want to understand it.

Copy link
Contributor Author

@ivdiazsa ivdiazsa Dec 14, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My bad, it was about disabling them in System.Runtime.Tests.csproj, not about ProjectExclusions. So many loops and very similar changes since this work was started I got confused for a second there.

and '$(RunGrpcTestsOnly)' != 'true'
and '$(RunHighAOTResourceRequiringTestsOnly)' != 'true'
and '$(RunLimitedSetOfTests)' != 'true'
and '$(TestReadyToRun)' != 'true'">

<ProjectReference Include="$(MSBuildThisFileDirectory)*\tests\**\*.Tests.csproj"
Exclude="@(ProjectExclusions)"
Condition="'$(TestAssemblies)' == 'true'"
Expand Down
4 changes: 2 additions & 2 deletions src/tasks/Crossgen2Tasks/ShimFilesSimulatingLogicInSdkRepo/Microsoft.NET.CrossGen.props
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<!--
***********************************************************************************************
Microsoft.NET.CrossGen.targets
Microsoft.NET.CrossGen.props

WARNING: DO NOT MODIFY this file unless you are knowledgeable about MSBuild and have
created a backup copy. Incorrect changes to this file will make it
Expand All @@ -17,4 +17,4 @@ Copyright (c) .NET Foundation. All rights reserved.
<UsingTask TaskName="PrepareForReadyToRunCompilation" AssemblyFile="$(MSBuildThisFileDirectory)Crossgen2Tasks.dll" />
<UsingTask TaskName="ResolveReadyToRunCompilers" AssemblyFile="$(MSBuildThisFileDirectory)Crossgen2Tasks.dll" />
<UsingTask TaskName="RunReadyToRunCompiler" AssemblyFile="$(MSBuildThisFileDirectory)Crossgen2Tasks.dll" />
</Project>
</Project>