AssemblyLoadContext crash when collectible assembly use XmlSerializer

[uffebjorklund]

[EDIT]
Investigation of this issue is complete and has resulted in 5 additional issues being opened.
See this comment below for links to each of those issues.

We will keep this issue open and update it as those other issues are fixed.
Please add any additional comments to those issues.

---

When creating a collectible AssemblyLoadContext and then use System.Xml.Serialization.XmlSerializer inside of a collectible assembly you will get a crash.

I have a repro project: https://github.com/uffebjorklund/AssemblyLoadContextBug

Have tried a few different things

• If creating the ALC as collectible = false there is no crash
• If the XmlSerializer code is removed from the collectible assembly there is no crash
• The crash will occur regardless of using netstandard2.0 or netstandard2.1 on the collectible assembly

Exception

A non-collectible assembly may not reference a collectible assembly

Stacktrace

at System.Reflection.Emit.ModuleBuilder.GetTypeRef(QCallModule module, String strFullName, QCallModule refedModule, String strRefedModuleFileName, Int32 tkResolution)
at System.Reflection.Emit.ModuleBuilder.GetTypeRefNested(Type type, Module refedModule, String strRefedModuleFileName)
at System.Reflection.Emit.ModuleBuilder.GetTypeTokenWorkerNoLock(Type type, Boolean getGenericDefinition)
at System.Reflection.Emit.ModuleBuilder.GetTypeTokenInternal(Type type, Boolean getGenericDefinition)
at System.Reflection.Emit.ILGenerator.Emit(OpCode opcode, Type cls)
at System.Xml.Serialization.CodeGenerator.Castclass(Type target)
at System.Xml.Serialization.CodeGenerator.InternalConvert(Type source, Type target, Boolean isAddress)
at System.Xml.Serialization.SourceInfo.InternalLoad(Type elementType, Boolean asAddress)
at System.Xml.Serialization.XmlSerializationWriterILGen.WriteElement(SourceInfo source, ElementAccessor element, String arrayName, Boolean writeAccessor)
at System.Xml.Serialization.XmlSerializationWriterILGen.WriteElements(SourceInfo source, String enumSource, ElementAccessor[] elements, TextAccessor text, ChoiceIdentifierAccessor choice, String arrayName, Boolean writeAccessors, Boolean isNullable)
at System.Xml.Serialization.XmlSerializationWriterILGen.WriteMember(SourceInfo source, String choiceSource, ElementAccessor[] elements, TextAccessor text, ChoiceIdentifierAccessor choice, TypeDesc memberTypeDesc, Boolean writeAccessors)
at System.Xml.Serialization.XmlSerializationWriterILGen.GenerateTypeElement(XmlTypeMapping xmlTypeMapping)
at System.Xml.Serialization.XmlSerializationWriterILGen.GenerateElement(XmlMapping xmlMapping)
at System.Xml.Serialization.TempAssembly.GenerateRefEmitAssembly(XmlMapping[] xmlMappings, Type[] types, String defaultNamespace)
at System.Xml.Serialization.TempAssembly..ctor(XmlMapping[] xmlMappings, Type[] types, String defaultNamespace, String location)
at System.Xml.Serialization.XmlSerializer.GenerateTempAssembly(XmlMapping xmlMapping, Type type, String defaultNamespace, String location)
at System.Xml.Serialization.XmlSerializer..ctor(Type type, String defaultNamespace)

dotnet --info

.NET Core SDK (reflecting any global.json):
 Version:   3.0.100
 Commit:    04339c3a26

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  10.13
 OS Platform: Darwin
 RID:         osx.10.13-x64
 Base Path:   /usr/local/share/dotnet/sdk/3.0.100/

Host (useful for support):
  Version: 3.0.0
  Commit:  7d57652f33

.NET Core SDKs installed:
  2.1.301 [/usr/local/share/dotnet/sdk]
  2.1.302 [/usr/local/share/dotnet/sdk]
  2.1.400 [/usr/local/share/dotnet/sdk]
  2.1.401 [/usr/local/share/dotnet/sdk]
  2.1.403 [/usr/local/share/dotnet/sdk]
  2.1.500 [/usr/local/share/dotnet/sdk]
  2.2.100-preview3-009430 [/usr/local/share/dotnet/sdk]
  2.2.101 [/usr/local/share/dotnet/sdk]
  2.2.102 [/usr/local/share/dotnet/sdk]
  2.2.103 [/usr/local/share/dotnet/sdk]
  2.2.105 [/usr/local/share/dotnet/sdk]
  2.2.106 [/usr/local/share/dotnet/sdk]
  2.2.203 [/usr/local/share/dotnet/sdk]
  2.2.300 [/usr/local/share/dotnet/sdk]
  3.0.100-alpha1-009708 [/usr/local/share/dotnet/sdk]
  3.0.100-preview-010184 [/usr/local/share/dotnet/sdk]
  3.0.100-preview3-010431 [/usr/local/share/dotnet/sdk]
  3.0.100-preview8-013656 [/usr/local/share/dotnet/sdk]
  3.0.100-preview9-014004 [/usr/local/share/dotnet/sdk]
  3.0.100 [/usr/local/share/dotnet/sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.1.1 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.2 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.3 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.5 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.6 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.0-preview3-35497 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.0 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.1 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.3 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.4 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.5 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 3.0.0-alpha1-10062 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.1 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.2 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.3 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.5 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.6 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.0-preview3-35497 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.0 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.1 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.3 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.4 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.5 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.0.0-alpha1-10062 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.0.0-preview-19075-0444 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.0.0-preview3-19153-02 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.0.0-preview8.19405.7 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.0.0-preview9.19424.4 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.0.0 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.1.1 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.2 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.3 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.5 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.6 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.0-preview3-27014-02 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.0 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.1 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.3 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.4 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.5 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.0.0-preview-27324-5 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.0.0-preview1-27029-03 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.0.0-preview3-27503-5 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.0.0-preview9-19423-09 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.0.0 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]

[csalat]

Hello,

Is any update on this?

Thanks,
Csaba

[StephenBonikowsky]

@uffebjorklund Thanks for reporting this, sorry for the slow response. It looks like all the information we need to do further investigation is here. We will look into it.

[mconnew]

I did a bit of digging in to this and I believe that the problem is due to XmlSerializer creating a dynamic assembly in the default load context which makes the dynamic assembly non-collectible. When we call ILGenerator.Emit(OpCodes.Castclass, type) with a type which is a collectible assembly, we get an exception. I did some digging around the api's and I can't see a way to specify the AssemblyLoadContext to generate a dynamic assembly into.
@sdmaclea, is this scenario supported? If so, can you point me to the api's to create a dynamic assembly in a specific AssemblyLoadContext?

[sdmaclea]

@sdmaclea, is this scenario supported? If so, can you point me to the api's to create a dynamic assembly in a specific AssemblyLoadContext?

@mconnew I think this was a feature we missed in 3.0. There is an API proposal to close this gap dotnet/corefx#38426

/cc @vitek-karas @elinor-fung @jeffschwMSFT

[curia-damiano]

Hi,
I share the same supposition, that it can be because of the dynamic generated assemblies.
But: I have tried using the XmlSerializerGenerator, with the idea that it wouldn't generated dynamic assemblies anymore.
But still I can't make it to work, see attached solution.
Do I miss anything?!?
Test_ALC with XmlSerializerGenerator.zip

[curia-damiano]

Btw this was my first project with the issue, without the XmlSerializerGenerator
Test_ALC.zip

[vitek-karas]

@curia-damiano How do you generate the serialization code? I tried to turning it on via VS on the CL.Implementation.Problem assembly and that failed (which is probably an issue on its own, but not directly related to this issue).

Also - I noticed that your "plugins" are built as netstandard2.0 - this is not recommended as SDK will not always produce the right assets. Ideally plugins should not target a standard but a specific framework - for example netcoreapp3.1 or so.

[curia-damiano]

Hi,
I have a ItemGroup in the project that "should" invoke the generation of the serialization code during build time. I haven't found any additional documentation saying I should do something more.
FYI I have used VS 2019 Preview (16.4.0 Preview 5 or 6).

[vitek-karas]

I tried again - Release build this time - and I spotted this in the output of the build:

3>Method 'System.Xml.Serialization.XmlSerializer.GenerateSerializer' was not found. This is likely because you are using an older version of the framework. Please update to .NET Core v2.1 or later.
3>Sgen utility failed to pregenerate serialization code for E:\AppModel\repro\Test_ALC\CL.Implementation.Problem\obj\Release\netstandard2.0\CL.Implementation.Problem.dll.

@StephenMolloy for the XML serializer specific portions of this question, like the above failure of sgen.

[mconnew]

@curia-damiano, I haven't tried running your code yet but I can see a problem in Test_ALC with XmlSerializerGenerator.zip. This is the problem code:

	var assembly = TLC.LoadFromAssemblyPath(dllPath);
	labelUnloaded.Text = "Loaded";
	var type = assembly.GetType("CL.Implementation.Easy.Multiplier");
	var constructor = type.GetConstructor(new Type[] { });
	ICompute compute = (ICompute)constructor.Invoke(new object[] { });

When using a different AssemblyLoadContext, the same type in the same assembly is a different runtime type. So this means the ICompute interface that CL.Implementation.Easy.Multiplier implements is a different runtime type that the same interface in the default load context. So the cast on the last line will fail.

[mconnew]

@sdmaclea, in this comment it says it uses the callers AssemblyLoadContext. You can see here that we do use AssemblyBuilder.DefineDynamicAssembly so I would expect the dynamically created assembly to use the custom AssemblyLoadContext. Is the problem that we pass AssemblyBuilderAccess.Run instead of AssemblyBuilderAccess.RunAndCollect? What's stopping the dynamically generated assembly from being created in the correct context? Also, is this going to be a problem for DispatchProxy too?