Reachability helper feature hangs tests in some scenario

[meziantou]

After updating to Coverlet 3.0.0, one of the test projects doesn't run anymore. The process is stuck at:

Test run for C:\Users\mezia\source\repos\Meziantou.Framework\tests\Meziantou.Framework.SingleInstance.Tests\bin\Debug\net461\Meziantou.Framework.SingleInstance.Tests.dll (.NETFramework,Version=v4.6.1)
Microsoft (R) Test Execution Command Line Tool Version 16.8.3
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...

The CPU usage is 25% for the .NET process (I have a 4 cores CPU).

I've generated a dump file, and it seems to be stuck somewhere in Mono.Cecil:

System.Private.CoreLib.dll!System.SpanHelpers.SequenceEqual(ref byte first, ref byte second, nuint length) Line 1561
	at /_/src/libraries/System.Private.CoreLib/src/System/SpanHelpers.Byte.cs(1561)
Mono.Cecil.dll!Mono.Cecil.MetadataResolver.GetType(Mono.Cecil.ModuleDefinition module, Mono.Cecil.TypeReference reference)
Mono.Cecil.dll!Mono.Cecil.MetadataResolver.Resolve(Mono.Cecil.MethodReference method)
Mono.Cecil.dll!Mono.Cecil.ModuleDefinition.Resolve(Mono.Cecil.MethodReference method)
Mono.Cecil.dll!Mono.Cecil.MethodReference.Resolve()
coverlet.core.dll!Coverlet.Core.Instrumentation.Reachability.ReachabilityHelper.CreateForModule(Mono.Cecil.ModuleDefinition module, string[] doesNotReturnAttributes, Coverlet.Core.Abstractions.ILogger logger) Line 299
	at /_/src/coverlet.core/Instrumentation/ReachabilityHelper.cs(299)
coverlet.core.dll!Coverlet.Core.Instrumentation.Instrumenter.CreateReachabilityHelper() Line 210
	at /_/src/coverlet.core/Instrumentation/Instrumenter.cs(210)
coverlet.core.dll!Coverlet.Core.Instrumentation.Instrumenter.InstrumentModule() Line 219
	at /_/src/coverlet.core/Instrumentation/Instrumenter.cs(219)
coverlet.core.dll!Coverlet.Core.Instrumentation.Instrumenter.Instrument() Line 159
	at /_/src/coverlet.core/Instrumentation/Instrumenter.cs(159)
coverlet.core.dll!Coverlet.Core.Coverage.PrepareModules() Line 146
	at /_/src/coverlet.core/Coverage.cs(146)
coverlet.collector.dll!Coverlet.Collector.DataCollection.CoverageWrapper.PrepareModules(Coverlet.Core.Coverage coverage) Line 63
	at /_/src/coverlet.collector/DataCollection/CoverageWrapper.cs(63)
coverlet.collector.dll!Coverlet.Collector.DataCollection.CoverageManager.InstrumentModules() Line 65
	at /_/src/coverlet.collector/DataCollection/CoverageManager.cs(65)
coverlet.collector.dll!Coverlet.Collector.DataCollection.CoverletCoverageCollector.OnSessionStart(object sender, Microsoft.VisualStudio.TestPlatform.ObjectModel.DataCollection.SessionStartEventArgs sessionStartEventArgs) Line 140
	at /_/src/coverlet.collector/DataCollection/CoverletCoverageCollector.cs(140)
[Native to Managed Transition]
[Managed to Native Transition]
System.Private.CoreLib.dll!System.Delegate.DynamicInvokeImpl(object[] args) Line 89
	at /_/src/coreclr/src/System.Private.CoreLib/src/System/Delegate.CoreCLR.cs(89)
System.Private.CoreLib.dll!System.Delegate.DynamicInvoke(object[] args) Line 64
	at /_/src/libraries/System.Private.CoreLib/src/System/Delegate.cs(64)
Microsoft.TestPlatform.CoreUtilities.dll!Microsoft.VisualStudio.TestPlatform.Utilities.MulticastDelegateUtilities.SafeInvoke(System.Delegate delegates, object sender, System.EventArgs args, string traceDisplayName)
Microsoft.VisualStudio.TestPlatform.Common.dll!Microsoft.VisualStudio.TestPlatform.Common.DataCollector.TestPlatformDataCollectionEvents.OnSessionStart(Microsoft.VisualStudio.TestPlatform.ObjectModel.DataCollection.DataCollectionEventArgs e)
Microsoft.VisualStudio.TestPlatform.Common.dll!Microsoft.VisualStudio.TestPlatform.Common.DataCollector.TestPlatformDataCollectionEvents.RaiseEvent(Microsoft.VisualStudio.TestPlatform.ObjectModel.DataCollection.DataCollectionEventArgs e)
Microsoft.VisualStudio.TestPlatform.Common.dll!Microsoft.VisualStudio.TestPlatform.Common.DataCollector.DataCollectionManager.SendEvent(Microsoft.VisualStudio.TestPlatform.ObjectModel.DataCollection.DataCollectionEventArgs args)
Microsoft.VisualStudio.TestPlatform.Common.dll!Microsoft.VisualStudio.TestPlatform.Common.DataCollector.DataCollectionManager.SessionStarted(Microsoft.VisualStudio.TestPlatform.ObjectModel.DataCollection.SessionStartEventArgs sessionStartEventArgs)
Microsoft.TestPlatform.CommunicationUtilities.dll!Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.DataCollection.DataCollectionRequestHandler.HandleBeforeTestRunStart(Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.Message message)
Microsoft.TestPlatform.CommunicationUtilities.dll!Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.DataCollection.DataCollectionRequestHandler.ProcessRequests()
datacollector.dll!Microsoft.VisualStudio.TestPlatform.DataCollector.DataCollectorMain.StartProcessing()
datacollector.dll!Microsoft.VisualStudio.TestPlatform.DataCollector.DataCollectorMain.Run(string[] args)
datacollector.dll!Microsoft.VisualStudio.TestPlatform.DataCollector.Program.Main(string[] args)

Environment:

• Windows 10
• .NET SDK 5.0.102

Repro steps:

• git clone https://github.com/meziantou/Meziantou.Framework.git
• cd Meziantou.Framework
• git checkout f96e0db0ab979a7d8344ec92391a1136b19a14fe
• dotnet build Meziantou.Framework.sln --configuration Debug /bl
• dotnet test .\tests\Meziantou.Framework.SingleInstance.Tests\Meziantou.Framework.SingleInstance.Tests.csproj --configuration Debug --no-build --logger trx --collect:"XPlat Code Coverage" -f net461

=> This is the only project of the solution that hangs, and it works with other target frameworks (-f net5.0)
=> It works with coverlet 1.3.0
=> It works without --collect:"XPlat Code Coverage"

[MarcoRossignoli]

Thanks for reporting this, we moved collector to netstandard support netfx, that could be the reason. I had similar issue in past with loop resolution inside cecil related to different platform type forwarding.

[danielp37]

I'm seeing the same thing in a project that I'm working on. With my projects, I've narrowed it down to a project that works with objects from System.Data (like DataTable or SqlConnection) AND the project also references one of these 3 nuget packages:

Microsoft.TeamFoundationServer.ExtendedClient Version="16.153.0"
Microsoft.VisualStudio.Services.Client Version="16.153.0"
Microsoft.VisualStudio.Services.InteractiveClient Version="16.153.0"

And these references need to be in a project that the unit test project references. If the System.Data references are in the test project, then it works fine. It has to be in a separate project. And if I don't reference any of these three projects it works fine and if I reference the projects but don't access system.data, it works fine.

I am targeting net461, am using SDK-style projects and running the tests with dotnet test targeting the .csproj. And just like above, it works with coverlet 1.3.0 and works without --collect:"XPlat Code Coverage".

[bthharper]

Am using nunit3-console with the coverlet console command line rather than dotnet test but get the same result. Have reverted back to 1.7.2 for now.

[MarcoRossignoli]

Can you all try with this packages?

Packages.zip

The issue is a loop fired by new reachability helper+our custom netstandard resolver
I removed local probing and @meziantou sample works on my local
cc: @kevin-montrose

[meziantou]

@MarcoRossignoli The CI completes with the preview package :)
https://github.com/meziantou/Meziantou.Framework/runs/1717298953

Thanks for the fix! I'll wait for the official release to update coverlet in my project

[MarcoRossignoli]

Unfortunately I fixed your issue but now I got a lot of noise from reachability helper, I need more time 😢

https://dev.azure.com/tonerdo/coverlet/_build/results?buildId=1324&view=logs&j=f0f0520d-467b-5d1d-a95d-4e3a2581fe8a&t=67725203-aed8-540e-d7ff-b357d1276ce7

A lot of warning [coverlet] Unable to resolve method reference...

[bthharper]

I've tried the updated coverlet, but it still hangs for me,

 	[External Code]	
 	[Managed to Native Transition]	
 	System.Private.CoreLib.dll!System.Collections.Generic.Dictionary<Mono.Cecil.Metadata.Row<string, string>, Mono.Cecil.TypeDefinition>.TryGetValue(Mono.Cecil.Metadata.Row<string, string> key, out Mono.Cecil.TypeDefinition value)	Unknown
 	Mono.Cecil.dll!Mono.Cecil.TypeDefinitionCollection.GetType(string namespace, string name)	Unknown
 	Mono.Cecil.dll!Mono.Cecil.MetadataResolver.GetType(Mono.Cecil.ModuleDefinition module, Mono.Cecil.TypeReference reference)	Unknown
 	Mono.Cecil.dll!Mono.Cecil.MetadataResolver.Resolve(Mono.Cecil.MethodReference method)	Unknown
 	Mono.Cecil.dll!Mono.Cecil.ModuleDefinition.Resolve(Mono.Cecil.MethodReference method)	Unknown
 	Mono.Cecil.dll!Mono.Cecil.MethodReference.Resolve()	Unknown
>	coverlet.core.dll!Coverlet.Core.Instrumentation.Reachability.ReachabilityHelper.CreateForModule(Mono.Cecil.ModuleDefinition module, string[] doesNotReturnAttributes, Coverlet.Core.Abstractions.ILogger logger) Line 299	C#
 	coverlet.core.dll!Coverlet.Core.Instrumentation.Instrumenter.CreateReachabilityHelper() Line 209	C#
 	coverlet.core.dll!Coverlet.Core.Instrumentation.Instrumenter.InstrumentModule() Line 218	C#
 	coverlet.core.dll!Coverlet.Core.Instrumentation.Instrumenter.Instrument() Line 159	C#
 	coverlet.core.dll!Coverlet.Core.Coverage.PrepareModules() Line 146	C#
 	coverlet.console.dll!Coverlet.Console.Program.Main.AnonymousMethod__1() Line 106	C#
 	McMaster.Extensions.CommandLineUtils.dll!McMaster.Extensions.CommandLineUtils.CommandLineApplication.Execute(string[] args)	Unknown
 	coverlet.console.dll!Coverlet.Console.Program.Main(string[] args) Line 261	C#
 	[Native to Managed Transition]	
 	[External Code]	

[MarcoRossignoli]

I think for the moment I'll go for disable "Methods that do not return" by default.
I need more time to understand what's happening, the type resolution part is a bit tricky and at the moment there is no workaround to unblock.
cc: @kevin-montrose

[zplan]

Unfortunately I fixed your issue but now I got a lot of noise from reachability helper, I need more time 😢

https://dev.azure.com/tonerdo/coverlet/_build/results?buildId=1324&view=logs&j=f0f0520d-467b-5d1d-a95d-4e3a2581fe8a&t=67725203-aed8-540e-d7ff-b357d1276ce7

A lot of warning [coverlet] Unable to resolve method reference...

Confirm @MarcoRossignoli .
I see a lot of these "Unable to resolve..." warnings comming from coverlet.msbuild.targets(39,5)
Installed version: coverlet.msbuild 3.0.1