SNI.dll file locked when ASP.NET (Framework 4.8) App is running in IIS

[jakenuts]

Describe the bug

The SNI.dll file is loaded/locked by IIS when using a Microsft.Data.SqlClient connection in a ASP.NET 4.8 Framework website but is not unloaded when the app is shutdown. Subsequent attempts to build or deploy the site are blocked by the error Unable to delete file "bin\x86\SNI.dll". Access to the path '...\bin\x86\SNI.dll' is denied. Every other dll in the bin folder can be deleted even while the web app is still running save for this one.

To reproduce

1.) Create an ASP.NET MVC website on the 4.8 Framework
2.) Add the Microsoft.Data.SqlClient package
3.) In the HomeController.Index method open a SqlConnection and execute an arbitrary command

using (var connection = new SqlConnection("<connection string here>"))
{
    connection.Open();
    new SqlCommand("SELECT 1", connection).ExecuteNonQuery();
}

4.) Cause the site to shutdown by modifying the web.config or stopping the debugger.
5.) The SNI.dll file remains locked and cannot be replaced or modified without shutting down IIS or the application pool associated with the website

Expected behavior

The SNI.dll should be unlocked (like the rest of the files in the bin folder) so that a site can be updated through deployment or build.

Further technical details

Microsoft.Data.SqlClient version: v1.1.0
.NET target: Framework 4.8
SQL Server version: SQL Azure
Operating system: Windows 10 Professional / Windows Server 2016 Datacenter (Azure VM)

[jakenuts]

The SNI.dll library appears to be loaded in the SNINativeMethodWrapper static initializer but the handle is not kept or freed anywhere else in the code.

 IntPtr pDll = LoadLibrary(localFolder + subfolder + SNI);
 if (pDll == IntPtr.Zero)
 {
    throw new System.ComponentModel.Win32Exception("Failed to load " + localFolder + subfolder + SNI, 
   new System.ComponentModel.Win32Exception(Marshal.GetLastWin32Error()));
}

[cheenamalhotra]

@jakenuts

Could you also verify and confirm if same behaviour is observed with System.Data.SqlClient?

[jakenuts]

I tried removing the SqlClient packages, changing the using statement to using System.Data.SqlClient which creates a SqlConnection from the System.Data 4.0.0 framework assembly and no SNI.dll is copied to the bin folder so no issues there.

I tried installing the System.Data.SqlClient 4.8 package and it does not appear to rely on a SNI.dll for this 4.8 project and all the files in the bin folder can be deleted while the web app is running.

I then replaced that package with Microsoft.Data.SqlClient but kept the using statement as "using System.Data.SqlClient". This copied the SNI.dll to the bin/x86 folder but it is not loaded or locked when executing the command.

Changing the using statement to "using Microsoft.Data.SqlClient" and running the app caused the SNI.dll to be loaded and it is locked until I terminate the IIS process.

[jakenuts]

I tried some alternative setups where the data call DoData() was moved to a .NET Standard 2.0 library as that's the one my project uses. It demonstrated the "could not load file or assembly" error that shows up alot in Google searches, performed as expected when both project referenced System.Data.SqlClient 4.8, but moving to them both the Microsoft.Data.SqlClient replicates the file locking issue above.

[Setup 1]

ASP.NET NET 4.8 Project

• no data assemblies referenced

.NET Standard 2.0 Library

• System.Data.SqlClient 4.8 referenced
• Using System.Data.SqlClient

No SNI dll in bin
Calling DoData() from the webapp into the library generates

System.IO.FileNotFoundException: 'Could not load file or assembly 'System.Data.SqlClient, Version=4.6.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.'

[Setup 2]

ASP.NET NET 4.8 Project

• System.Data.SqlClient 4.8 referenced

.NET Standard 2.0 Library

• System.Data.SqlClient 4.8 referenced

No SNI dll in bin
Calling DoData() from the webapp into the library succeeds, all files can be deleted

[Setup 3]

ASP.NET NET 4.8 Project

• no data assemblies referenced

.NET Standard 2.0 Library

• Microsoft.Data.SqlClient 1.1.0 referenced
• using Microsoft.Data.SqlClient;

No SNI dll in bin
Calling DoData() from the webapp fails with the 'could not load' exception

[Alternate Setup 4]

ASP.NET NET 4.8 Project

• System.Data.SqlClient 4.8 referenced

.NET Standard 2.0 Library

• Microsoft.Data.SqlClient 1.1.0 referenced
• using Microsoft.Data.SqlClient;

No SNI dll in bin
Calling DoData() from the webapp fails with the 'could not load' exception

[Alternate Setup 4]

ASP.NET NET 4.8 Project

• Microsoft.Data.SqlClient 1.1.0 referenced

.NET Standard 2.0 Library

• Microsoft.Data.SqlClient 1.1.0 referenced
• using Microsoft.Data.SqlClient;

SNI dll copied to bin
Calling DoData() succeeds, SNI.dll is locked until IIS process killed

[jakenuts]

Looking at the System.Data.SqlClient repository, there are some references to the necessity of holding onto and freeing the handle from SNI.dll (can't dig up where they load it). It doesn't appear that the handle loaded in Microsoft.Data.SqlClient is kept or later released which might explain why it sticks around.

From LocalDBAPI.Common.cs in System.Data.SqlClient

    //This is copy of handle that SNI maintains, so we are responsible for freeing it - therefore there we are not using SafeHandle
    private static IntPtr s_userInstanceDLLHandle = IntPtr.Zero;

   internal static void ReleaseDLLHandles()
   {
      s_userInstanceDLLHandle = IntPtr.Zero;
      s_localDBFormatMessage = null;
   }

[cheenamalhotra]

Thanks for the details @jakenuts

I just opened up System.Data.SqlClient .NET framework library source code and I didn't see any call to "LoadLibrary" there (since our code is imported from there), here's the only way it's loaded for every API:

[DllImport("SNI.dll", CallingConvention = CallingConvention.Cdecl, EntryPoint = "SNIAddProviderWrapper")]
internal static extern uint SNIAddProvider(SNIHandle pConn, ProviderEnum ProvNum, [In] ref uint pInfo);

Which is also present in our case in SNINativeMethodWrapper.cs

I'm trying to understand why we added LoadLibrary call in there, which seems to be primary difference and cause of this behaviour.

[cheenamalhotra]

Hi @jakenuts

[edit]
Turns out it's necessary to load DLLs in Microsoft.Data.SqlClient as SNI.dll is external from NuGet package, unlike System.Data.SqlClient, I'll continue to investigate for possible solution, if any.

[cheenamalhotra]

Hi @jakenuts

I tried creating ASP.NET application to try reproduce issue but I cannot reproduce the problem using steps you mentioned above. I get file in use errors for "Roslyn" instead which I cannot delete, everything else I'm able to clean normally.

Application I tried:
WebApplication1.zip

Can you provide me a simple application which you're able to reproduce with if I missed something as I'm new to ASP.NET MVC architecture.

[jakenuts]

Sorry, I should have included that the Roslyn folder cannot be deleted while the app is running. Here is a minimal ASP.NET 4.8 project that demonstrates the issue (at least on my machine). You'll need to run it on IIS or IIS Express to reproduce the locking. I'm also connecting to Azure SQL in my connection string (stupidly posted that earlier, thus the deletion) but have replaced that with the localhost one from yours. Later I'll take a look at the two projects and see if there are any relevant differences.

LockingWebApp.zip

[jakenuts]

Here's another sample app that is even more 'minimal'. It uses owin and creates a connection to a non-existent LocalDb which is enough to use the SNI.dll imports. Once you open the project, restore the packages and run the app it should demonstrate the problem. Replacing the using statement for Microsoft.Data.SqlClient with System.Data.SqlClient resolves the issue.

LockingWebApp.zip

[cheenamalhotra]

Hi @jakenuts

I tried your samples but still cannot reproduce any problem.
I apparently have localdb installed, but I also tested with random server name in connection string just to trigger exception case. I don't see any locking problems with SNI.dll.

Just to confirm the steps:

1. Open application in VS and run with IIS
2. Stop debugging app
3. Delete bin and obj folders manually
   • I can delete totally fine!

My Environment details are:

• Windows 10 Enterprise
• Visual Studio Enterprise 2019 (16.4.1)
• .NET Framework 4.8.03752
• IIS Version 10.0.18362.1

Did I miss something?

P.S. I'm also checking with my team internally if someone can reproduce in their environment.

[jakenuts]

Thanks! I tried loading the project on a different machine and for whatever reason it differs in that "stop debugging" or closing the browser window also closes IISExpress which then releases the DLL. On my machine IISExpress stays active even after I've stopped debugging.

When I tried with "Start without debugging" on the new machine, IISExpress stays active and when you try to rebuild, or if you delete all the rest of the files the SNI.dll remains locked. This replicates what is happening in production where full IIS remains running and once the dll is loaded it cannot be moved or updated until you kill the application pool.

[jakenuts]

I just tried creating an Azure App Service for the sample and published it there. Unfortunately it's way better at detecting changes than my VM deployment so it doesn't try to update the SNI.dll as they match so deployment succeeded without an issue. I was able to go into the console there and confirm that all the files in the bin directory could be deleted save for the SNI.dll though.

[cheenamalhotra]

Hi @jakenuts

Yes I can reproduce the behavior now. The behavior with "Start without Debugging" happens because application is running on IIS in a detached mode and is Active on IIS Server from "bin" directory. What you mean to do is rebuild and redeploy on top of Active application's "bin" directory. SNI.dll is locked in the process since it's a loaded library, from where you pointed above. Since the application is "Active", the DLL will stay loaded there.

If you stop the site, that would free the lock and SNI.dll can be deleted.

This seems to be expected behavior with Microsoft.Data.SqlClient.

With System.Data.SqlClient, you would not see this behavior because SNI code is inbuilt in "System.Data.dll" hence there was no external loading done. The Native SNI code in there is shared between all ADO.NET providers. This is not the case with Microsoft.Data.SqlClient as SNI.dll needs to be loaded from an external NuGet package "Microsoft.Data.SqlClient.SNI" in current process.

You can say that SqlClient (for .NET Framework) is incomplete without this referenced "SNI.dll". The reason why it's referenced externally from Microsoft.Data.SqlClient is that SNI native implementation continues to be a closed source, and is not open sourced yet.

[cheenamalhotra]

I was wondering if SNI.dll stayed loaded even after App was Shutdown/Stopped, as mentioned in the first comment, that doesn't seem to be the case.