IIS Shadow Copied DLL accessing dependency DLLs from bin location - asp.net

I am trying to resolve an issue with ASP.Net Framework 4.8 site using EFCore 3.1.16 in IIS. Microsoft.Data.SqlClient has a process lock on SNI.dll which causes issues with xcopy deployment in IIS.
I have tried a strategy of copying the SNI.dll to the same shadow copy location as Microsoft.Data.SqlClient so it doesn't have to try and access the DLL in the bin folder as outlined in https://github.com/lscorcia/sqlclient.snishadowcopy.
// Look for the main Microsoft.Data.SqlClient assembly in the
// shadow copy path
var sqlClientShadowAssembly = Directory.GetFiles(
currentShadowPath, "Microsoft.Data.SqlClient.dll",
SearchOption.AllDirectories).FirstOrDefault();
// Extract the directory information from the shadow assembly path
var sqlClientShadowPath =
Path.GetDirectoryName(sqlClientShadowAssembly);
// Find out the process bitness and choose the appropriate native
// assembly
var moduleName = Environment.Is64BitProcess ? "x86\\SNI.dll"
: "x64\\SNI.dll";
// Compute the source and target paths for the native assembly
var sourceFile = Path.Combine(currentPrivatePath, moduleName);
var targetFile = Path.Combine(sqlClientShadowPath, "SNI.dll");
File.Copy(sourceFile, targetFile);
However, it still tries to access the bin location first instead of the sni.dll that is in the same folder location.
I have checked that the Microsoft.Data.SqlClient in the shadow location is being used correctly by deleting the DLL and confirming that a FileNotFound exception is thrown.I have also tried copying directly into the same folder and also copying into an x64 sub folder in the shadow location.

In my case, the error occured only when my IIS application is located on an UNC path (e.g. "\\myserver\myshare\myapplication"). Here is a workaround that worked in my scenario.
Use P/Invoke to SetDllDirectory:
[DllImport(#"kernel32.dll", CharSet = CharSet.Auto, SetLastError = true)]
private static extern bool SetDllDirectory(string lpPathName);
(See also this MSDN article about DLL load order)
Then, early in my Global.asax.cs Application_Start method (before any DB calls), call it like this:
SetDllDirectory(Server.MapPath(#"~/bin"))
After that, everything works as expected.
I still to consider this to be kind of a hack/workaround, but at least, it is working now in my scenario.
As I do understand, you can call SetDllDirectory multiple times to add additional directories (i.e. not overwrite the existing one).
So in case someone reading this might have other assemblies that refer to native DLLs in "x86" or "x64" folders, one might do something like this:
var bitness = Environment.Is64BitProcess ? #"x64" : #"x86";
SetDllDirectory(Server.MapPath($#"~/bin/{bitness}"));
I've also tried serving my test application from a local path (like "C:\Inetpub\wwwroot") and here, the error does not occur, even when not calling SetDllDirectory.
I'm still not sure why the error occurs for UNC paths only, and not for local paths, as I would expect that the shadow copied managed assemblies to fail the DllImports, too.
(I've also posted the above in this GitHub issue)

Related

Native DLL dependencies with ASP.NET MVC Projects

EDIT: I found a way to get it to work locally, but on Azure I still get System.IO.FileNotFoundException on that assembly.
My question might seem like a duplicate to this question here. But it is slightly different, I have already tried that solution and it did not work. Here are the details.
I have an ASP.NET MVC App that has a Reference added to a third party CLR DLL. That third-party DLL requires a native DLL which it invokes. Now if I had control over where the Shadow Copying occurs and what is copied, I would be in paradise. The Shadow Copying misses copying that native DLL despite it's Build Action set as Content and Copy To Output Dir set as Copy Always.
So I searched internet and ran into this discussion on SO, which is same as what was mentioned earlier. I tried adding the code that sets the PATH Environment Variable inside Application_Init and Application_Start of Global.asax, I set the breakpoints in both the methods and to my surprise I get the ASP.NET Error Page before it even hits the breakpoint. This leads me to believe that the referenced assembly at the time of binding hits the native DLL and invokes it. What can I do? Can I delay the reference binding somehow?
EDIT: Yes we can, I opened the Referenced DLL's code which was written in Managed C++, I adjusted the linker setting to Delay Load the Native DLL and now my Application_Start executes first. Yayy! but that does not solve the same problem I am having on Azure
Here is the test solution with DLLs
Here is the source code for the Native DLL
Here is the source code for the Referenced Assembly that uses the Native DLL
To download the Native DLL distribution, Go to their distribution page, choose the windows archive with the bitness you desire (I am using 32-bit), and you will find amzi.dll inside APIs/bin directory.
Actual problem was the wrapper DLL not recognized on Azure server because of lack of support of earlier frameworks and toolsets, as well as Debug CRT.
I used XDT/Application_Start to set the PATH environment variable to include the location of my native DLL
I upgraded my Managed C++ Wrapper DLL to use Toolset 14.0 and .NET 4.6.2
Used Linker Setting of /DELAYLOAD on Managed C++ Wrapper DLL
After downloaded the DLLs and source code which you provided, I found that the native DLL depends on x64 platform. Firstly, we need to change the Platform property of our web app to x64 using Azure portal. If the platform button is disabled, you need to scale up your web app plan to Basic level plan or higher level.
In addition, the original path may end with “;”, so we need to check whether it contains “;” and append right content to it. Code below is for your reference.
string path = Environment.GetEnvironmentVariable("PATH");
Trace.TraceError(path);
string binDir = Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "Bin");
Trace.TraceError(binDir);
if (path.EndsWith(";"))
{
Environment.SetEnvironmentVariable("PATH", path + binDir);
}
else
{
Environment.SetEnvironmentVariable("PATH", path + ";" + binDir);
}
To test whether the path is set successfully, you could add a page to test it.
public ActionResult GetPath()
{
string path = Environment.GetEnvironmentVariable("PATH");
return Content(path);
}
After path is set, the native DLL can be load successfully on my side.
On my end I added a throw new ApplicationException("Test") at the beginning of Application_Start and instead of getting my test exception, I was getting the DLL load error.
It means the setting path code will not executed. To fix it, you could remove the native DLL reference from your web application. Now your application could work fine and set the path environment variable. Then you could add the native DLL reference back.
Another way to do it is that we could create a webjobs and set the path environment variable in webjobs and deploy this webjobs before deploying your web application.
I am using 32-bit distributions, my native dlls depends on x86/32-bit.
If you use 32-bit distributions and the platform targets of your CLR DLL and your web application are set to "x86 or Any CPU", you won't need to change platform to x64 in web app. Please change it back to x86.

IIS Web application and class-library configuration - config file is lost when deployed

I have an ASP.NET web application written in C# 4.0. The application references a class library that comes with its own configuration file. At runtime, the class library uses similar to the following code to load this specific configuration:
var exeConfigPath = this.GetType().Assembly.Location;
var config = ConfigurationManager.OpenExeConfiguration(exeConfigPath);
This is done because the library has to load its bundled configuration rather than the application configuration. The application configuration should not be concerned of the library's settings and should not be able to alter them.
Now, there are a few other things that need to be done for this concept to work. I have to set the library's configuration file build operation as Content in the properties window and the Copy to be Copy Always or Copy If Newer. So far so good - the file gets automatically both into the class library's bin directory, and the web applications's bin directory, and is correctly renamed from App.config to CustomLibrary.dll.config (as supposed, the library's dll is CustomLibrary.dll).
Now I am facing two issues.
1) When I publish the web application to a filesystem location (mapped in IIS), the CustomLibrary.dll.config appears back as App.config in the bin folder of the published app. OK - I will rename it in the class library project to match the expected convention - and problem solved.
2) Even when published, the IIS compiles the application again and stores it in the ASP.NET Temporary Files. There is a fancy directory structure with a folder dedicated for each assembly referenced. The folder corresponding to the CustomLibrary.dll does not contain the config file in it. Since this.GetType().Assembly.Location will return the path to the temp folder, the application fails to load the configuration and crashes as it should.
I need to preserve the pattern of having the configuration in the class library, and be able to make it work in the web application. When manually copying the .config to the temp folder, the app works, but see, I really hate manual copying to randomly-named folders.
Is there a way to either prevent IIS from using the temp folders, or to make it copy along the config files? I believe the problem I am facing is configuration-related rather than conceptual since the application works as expected when the config file is in place. I'd prefer not to mess with using hard-coded physical paths to the config file either.
Edit:
To make it clearer, I will point out what and why I want to achieve. The idea is that the library and the web project will be developed as separate products - there will be no user or application specific information in the configuration of the library, so it will not change for different use scenarios. It is also rather specific to the class library functionality rather than the end application. It makes sense for me to keep the library's configuration information bundled within it (similar to Java, where a spring context xml file, or a properties file, get bundled with the jar of the library). I'd like to avoid having to copy the configuration in each app/web config of the consumer application. There will be cases where the consumer application is developed by third parties, and I do not want to rely on them doing their configuration right for my stuff to work. Again, the only issue here is not having the config file copied to the right place.
If those are static, internal settings that nobody should see or change, wouldn't you be better off having a file with the configuration included within the class library as an embedded resource? Either that or just a static class with the settings.
That way you'd be certain that nobody alters it, which in your scenario seems to be a plus.
I have come along a way to work arround the described issue, still not a very pleasant one to my requirements.
The solution is to take advantage of the application configuration (web.config in web apps, or app.config) which is always available. I have added as settings the absolute paths to the config file for each library. So I ended up with:
<!--
THIS IS IN THE WEB.CONFIG FILE
-->
<appSettings>
<add key ="ClassLibrary_ConfigPath"
value ="{My Publish Output Folder}\ClassLibrary.dll.config"/>
</appSettings>
and the class library now uses the following code to load its configuration:
Configuration config = null;
try
{
var exeConfigPath = this.GetType().Assembly.Location;
config = ConfigurationManager.OpenExeConfiguration(exeConfigPath);
}
catch (Exception e)
{
if (!IsConfigurationNotFoundError(e))
{
// IsConfigurationNotFoundError logic skipped for brevity
var exeConfigPath =
ConfigurationManager.AppSettings["ClassLibrary_ConfigPath"];
if (exeConfigPath != null)
{
config = ConfigurationManager.OpenExeConfiguration(exeConfigPath);
}
}
else
{
throw;
}
}
While this works, I will wait for a better solution if possible. Still, I do not have to copy the entire ClassLibrary.dll.config into the web.config file, but now I must manage filesystem locations and be aware of app-setting names. What I really want is the consumer app of the ClassLibrary.dll not to deal with its configuration in any way. If it were a desktop app, I have this covered, as Visual Studio copies the ClassLibary.dll.config appropriately. I hope there is a way to make it work smoothly for web apps.
The short answer is: you can't. You have to merge both configuration sections and place all settings in the main configuration file of your application. In case of the web application it would be the web.config. Read this

CruiseControl.net , SVN, web site and dependencies

I have an Asp.Net web site (there is no .csproj file).
This web site's source code is controlled by SVN.
I excluded the bin folder from the source control.
All the external assemblies are referenced from the DLL folder which is at the root of my SVN.
I try to deploy this website with cruisecontrol.net.
The problem :
Cruise control load all the files from subversion, it runs msbuild.exe against the .sln file. This results in an error : can't find the external assemblies (because the bin folder is excluded).
The solution I found so far :
Before my msbuild task , do a robocopy of the dll from my source control to the /bin folder.
Is there any other solution ? (I don't want to edit my configuration every time I add an external assembly to my project).
EDIT :
I finally used the "refresh file" technique here is the program I used to create them
class Program
{
private const string PATH_BIN = #"F:\WebSite\bin\";
private const string PATH_REFERENCE = #"C:\DLL\";
static void Main(string[] args)
{
foreach (string aFile in Directory.GetFiles(PATH_BIN))
{
if(!aFile.EndsWith(".dll"))
continue;
string pathRefreshFile = aFile + ".refresh";
if(File.Exists(pathRefreshFile))
continue;
string referenceFilePath = PATH_REFERENCE+Path.GetFileName(aFile);
if (!File.Exists(referenceFilePath))
continue;
using (StreamWriter sw = new StreamWriter(pathRefreshFile))
{
sw.Write(referenceFilePath);
}
}
}
}
I'm assuming that you have .dll files that are third party, rather than ones created by class library projects that you could just include in your solution and add as a project reference. We have a similar scenario with items like the Microsoft Anti-Xss library and other common third party .dlls that we reference in almost all our web apps.
Instead of excluding the \bin directory, exclude *.dll. This way, when you add a reference to the .dll, Visual Studio will add the .dll to the \bin directory AND a .dll.refresh file to the \bin directory.
That.Refresh file tells Visual Studio where to grab the .dll file from. As long as the original .dll is in that location then you should be able to build on the build server OR on a brand new developer's PC.
For example, we have a \shared\commonDlls directory in source control, where the .dlls are checked into source control.
All of our other apps reference these when needed. As long as these are checked in, when the build server builds your project, it will know to go tot he \shared\commonDlls directory to copy that dll in.
This enables us to have one master source for the .dll files so that we don't have to copy it into a few dozen different web apps when it's time to upgrade. We put it into our shared\commonDlls directory (overwriting the original), check it in, and all of our web apps are now using the latest version.
edit - added - links to answers on related topics that have bearing on this answer:
SVN and binaries
How manage references in SVN/Visual Studio?
Best Practice: Collaborative Environment, Bin Directory, SVN
Edit - added - section from our ccnet.config file pointing to a .sln file
This is probably not needed, but since I mentioned ointing ot the .sln file in my comment below I thought I'd add it.
<msbuild>
<description>DonationRequests</description>
<workingDirectory>D:\dev\svn\trunk\Intranet\DonationRequests</workingDirectory>
<executable>C:\WINDOWS\Microsoft.NET\Framework\v4.0.30319\MSBuild.exe</executable>
<projectFile>DonationRequests.sln</projectFile>
<buildArgs>/p:Configuration=Debug /v:diag</buildArgs>
<timeout>900</timeout>
<logger>C:\Program Files\CruiseControl.NET\server\ThoughtWorks.CruiseControl.MsBuild.dll</logger>
</msbuild>

How a native C++ plug-in DLL can load a private assembly located in a specific directory

I have a native C++ DLL which is used as a plug-in in another application. This DLL has an embedded manifest and depends on a private assembly located in a folder external to the application. The application fails to load my plug-in DLL because my DLL depends on a private assembly which is never found (it is not located in the application directory, nor in the winsxs folder but in my plug-in directory whose location is not controlled by the application). The question is: how is it possible to make the system find my private assembly which is located in my own specific directory ? As an analogy, I would need an equivalent of setDllDirectory() but for assemblies. Or another way so that the system find my private assembly.
Constraints:
Because my DLL is a plug-in, I cannot install anything in the directory and sub-directories of the application. I also cannot modify the behavior of the application.
I would like to avoid sharing the assembly in winsxs.
I also have to use an assembly and not a simple DLL (that I could load with LoadLibrary) to avoid version conflicts.
Thanks.
you need to have a global location where you could give a list of directories that form your (from java-terminology) class-path. Once you have this list just traverse the directories looking for dll's.
I have used the Registry in windows for this. An environment variable or the /etc folder under linux would also do.
This is how linkers do it, and this is one way that the .NET GAC does (it specifies a list of directories in a registry key).
-- edit 1 --
Your problem seems quite a uncomfortable mix, it's like trying to eat with a straight jacket on :P. Why don't you drop the assembly requirement and have a function in a native DLL that returns a version identifier (name-version), and then have some application logic to determine if it may use the DLL.
For example file-name (fn) = monkey-1-1 and a function call returns a struct:
struct version-id
{
const char * name;
int major_number;
int minor_number;
};
Then fn monkey-1-2 would not conflict on file-name or internal conversioning. and your programming conventions or application logic would make sure no adverse collisions occur.

Unmanaged DLLs fail to load on ASP.NET server

This question relates to an ASP.NET website, originally developed in VS 2005 and now in VS 2008.
This website uses two unmanaged external DLLs which are not .NET and I do not have the source code to compile them and have to use them as is.
This website runs fine from within Visual Studio, locating and accessing these external DLLs correctly. However, when the website is published on a webserver (runnning IIS6 and ASP.NET 2.0) rather than the development PC it cannot locate and access these external DLLs, and I get the following error:
Unable to load DLL 'XYZ.dll': The specified module could not be found. (Exception from HRESULT: 0x8007007E)
The external DLLs are located in the bin directory of the website, along with the managed DLLs that wrap them and all the other DLLs for the website.
Searching this problem reveals that many other people seem to have the same problem accessing external non.NET DLLs from ASP.NET websites, but I haven't found a solution that works.
I have tried the following:
Running DEPENDS to check the dependencies to establish that the first three
are in System32 directory in the path, the last is in the .NET 2
framework.
I put the two DLLs and their dependencies in
System32 and rebooted the server, but website still
couldn't load these external DLLs.
Gave full rights to ASPNET, IIS_WPG and IUSR (for that server) to
the website bin directory and rebooted, but website still couldn't
load these external DLLs.
Added the external DLLs as existing items to the projects and set
their "Copy to Output" property to "Copy Always", and website
still can't find the DLLs.
Also set their "Build Action" property to "Embedded resource" and
website still can't find the DLLs.
Any assistance with this problem would be greatly appreciated!
This happens because the managed dlls get shadow copied to a temporary location under the .NET Framework directory. See http://msdn.microsoft.com/en-us/library/ms366723.aspx for details.
Unfortunately, the unmanaged dlls do NOT get copied and the ASP.NET process won't be able to find them when it needs to load them.
One easy solution is to put the unmanaged dlls in a directory that is in the system path (type "path" at the command line to see the path on your machine) so that they can be found by the ASP.NET process. The System32 directory is always in the path, so putting the unmanaged dlls there always works, but I would recommend adding some other folder to the path and then adding the dlls there to prevent polluting the System32 directory. One big drawback to this method is you have to rename the unmanaged dlls for every version of your application and you can quickly have your own dll hell.
As an alternate to putting the dll in a folder that is already in the path (like system32) you can change the path value in your process by using the following code
System.Environment.SetEnvironmentVariable("Path", searchPath + ";" + oldPath)
Then when LoadLibrary tries to find the unmanaged DLL it will also scan searchPath. This may be preferable to making a mess in System32 or other folders.
Try putting the dlls in the \System32\Inetsrv directory. This is the working directory for IIS on Windows Server.
If this doesn't work try putting the dlls in the System32 directory and the dependency files in the Inetsrv directory.
Adding to Matt's answer, this is what finally worked for me for 64-bit server 2003 / IIS 6:
make sure your dlls / asp.net are the same version (32 / 64 bit)
Put the unmanaged dlls in inetsrv dir (note that in 64 bit windows, this is under syswow64, even though the sys32/inetsrv directory is created)
Leave the managed dlls in /bin
Make sure both sets of dll's have read/execute permissions
Take a look with FileMon or ProcMon and filter on the names of the troublesome DLLs. This will show you what directories are scanned in search of the DLLs, and any permission issues you might have.
Another option is embedding the native DLL as a resource in the managed DLL. This is more complicated in ASP.NET, as it requires writing to temporary folder at runtime. The technique is explained in another SO answer.
Always worth checking the path variable in your environment settings too.
I have come across the same issue. And I tried all above options, copying to system32, inetpub, setting path environment, etc nothing worked.
This issue is finally resolved by copying unmanaged dll to the bin directory of web application or web service.
Аfter struggling all day over this problem and finally I found a solution which suits me. It's just a test, but the method is working.
namespace TestDetNet
{
static class NativeMethods
{
[DllImport("kernel32.dll")]
public static extern IntPtr LoadLibrary(string dllToLoad);
[DllImport("kernel32.dll")]
public static extern IntPtr GetProcAddress(IntPtr hModule, string procedureName);
[DllImport("kernel32.dll")]
public static extern bool FreeLibrary(IntPtr hModule);
}
public partial class _Default : System.Web.UI.Page
{
[UnmanagedFunctionPointer(CallingConvention.StdCall)]
private delegate int GetRandom();
protected System.Web.UI.WebControls.Label Label1;
protected void Page_Load(object sender, EventArgs e)
{
Label1.Text = "Hell'ou";
Label1.Font.Italic = true;
}
protected void Button1_Click(object sender, EventArgs e)
{
if (File.Exists(System.Web.HttpContext.Current.Server.MapPath("html/bin")+"\\DelphiLibrary.dll")) {
IntPtr pDll = NativeMethods.LoadLibrary(System.Web.HttpContext.Current.Server.MapPath("html/bin")+"\\DelphiLibrary.dll");
if (pDll == IntPtr.Zero) { Label1.Text = "pDll is zero"; }
else
{
IntPtr pAddressOfFunctionToCall = NativeMethods.GetProcAddress(pDll, "GetRandom");
if (pAddressOfFunctionToCall == IntPtr.Zero) { Label1.Text += "IntPtr is zero"; }
else
{
GetRandom _getRandom = (GetRandom)Marshal.GetDelegateForFunctionPointer(pAddressOfFunctionToCall,typeof(GetRandom));
int theResult = _getRandom();
bool result = NativeMethods.FreeLibrary(pDll);
Label1.Text = theResult.ToString();
}
}
}
}
}
}
Run DEPENDS on XYZ.dll directly, in the location that you have deployed it to. If that doesn't reveal anything missing, use the fuslogvw tool in the platform SDK to trace loader errors. Also, the event logs sometimes contain information about failures to load DLLs.
On Application_start use this:
(customise /bin/x64 and bin/dll/x64 folders as needed)
String _path = String.Concat(System.Environment.GetEnvironmentVariable("PATH")
,";"
, System.Web.Hosting.HostingEnvironment.MapPath("~/bin/x64")
,";"
, System.Web.Hosting.HostingEnvironment.MapPath("~/bin/dll/x64")
,";"
);
System.Environment.SetEnvironmentVariable("PATH", _path, EnvironmentVariableTarget.Process);
Try putting the DLL in the Windows/SysWOW64 folder. It was the only thing that worked for me.

Resources