TeamCity using MSDEPLOY ERROR_FILE_IN_USE - asp.net

I have an ASP.NET 4.6.1 application that I am trying to deploy to a remote web server using TeamCity and MSBuild. Below is the MSBuild build step within TeamCity.
When I run the build in TeamCity, it always fails on this build step with a FILE_IN_USE_ERROR. The DLLs that are being updated before the error are usually the same, but sometimes they are different.
[MSBuild output] Info: Updating file (MyApp\bin\roslyn\Microsoft.VisualBasic.Core.targets).
[10:35:23]
[MSBuild output] Info: Updating file (MyApp\bin\roslyn\System.AppContext.dll).
[10:35:23]
[MSBuild output] Info: Updating file (MyApp\bin\roslyn\System.Collections.Immutable.dll).
[10:35:23]
[MSBuild output] MSDEPLOY : error Code: ERROR_FILE_IN_USE [C:\BuildAgent\work\b89cd3dfa447b\MyApp\MyApp.csproj]
I've already read through many articles (here, here, here, and here) suggesting that I drop an app_offline.htm file before deploying so that the IIS can re-route traffic and the file locking issues will stop. I tried that and it doesn't work. I still get the file in use issues during this build step.
I've also read in plenty of other discussions that another solution is to stop the application pool, deploy the app updates, and start the application pool. This is not a realistic solution because the deployment usually lasts between 15 and 20 minutes. Plus, this doesn't allow me to show a nice message to any current users.
I've been reading about this for a while now and it seems like this is a common use case for deployment to an IIS web server. Shouldn't this just work without locking files? Does anybody know of any other solutions to this problem?

I would try by stopping app pool to see if that`s a root cause of the issue.
But I'm not sure why it fails on .csproj file, because usually you don`t deploy those on server. So either it fails on different file (that is actually in use by web application) or it fails during build, not deploy (than app-pool stop will have no affect).

Related

Symbols not loading for remote debugging on ASP.Net app built and deployed from Azure DevOps Pipeline

I have an ASP.Net application who's code is sitting in an Azure Repo. The project has a build pipeline that builds on master branch merges. I then have a Deployment pipeline that takes the latest build and deploys local on my web server through a deployment pool I have running on my server. The web application builds with the VS Build task and deploys with the IIS Web App Deploy task. Both work fine.
I have one VM in with Visual Studio that I am trying to use to remote debug the web server. I have VS Remote Tools on the web server and it successfully runs. On my VM, I am able to open VS, attach debugger to a remote process on the web server successfully. The problem is that the symbols are not loading and I'm not sure what the correct sequence of items is here.
First, it doesn't appear that there are any .pdb files in the build produced by the Azure Pipeline. Second, I'm not sure what is the proper way to get the code onto the VM for debugging (Clone repo, vs download zip, etc). Third, I attempted to add a Publish Smybols task to my deploy pipeline, however its generating .pdb folders not files, and I'm not sure where to place these either on the web server, or on the vm.
My background is in classic local TFS setups, so working, building and deploying from Azure DevOps has me confused on how to get remote debugging to work.
OK this is not for the faint-hearted. It has taken me 3hrs to slowly work through this - but it's worth it. Many times has something worked locally, but then when you trigger an Build Agent with CI on a remote server you can't Step through the code with breakpoints.
So this info is if you are using the above situation - Azure build agent and Continuous Integration. If you are using a Publish Profile this doesn't apply.
First things first... The most important parts of this answer can be found in this blog:
https://willys-cave.ghost.io/i-have-a-dream-of-a-single-build-consistent-x-and-simple/
I've added that Url to the wayback machine at archive.org in case it disappears.
So yes the problem is the .PDB files - they need to be included by adding Publish symbols task. in your VSO pipeline.
Note: I had to change the BuildConfiguration parameter to debug (different from Willy's instructions). Otherwise when you eventually start to hit breakpoints the code is optimized and you won't see variable values in the hover-over etc.
In VS 2019 Willy's instruction for Link to the symbols during remote debugging sessions needs reading carefully. I didn't. There is a better image on:
https://devblogs.microsoft.com/devops/vsts-is-now-a-symbol-server/
I include the screen capture here:
Importantly you need to add your VSTS hostname into the list of Symbol Servers
Now mine still wasn't hitting the breakpoints and I found this page (which is generally about using the slightly different method of Publish Profiles), but I noticed some more components were loaded into IIS... Yes! You may need these too.
https://learn.microsoft.com/en-us/visualstudio/debugger/remote-debugging-azure?view=vs-2019
So the most important image I will paste here:
You need to add IIS Management Scripts and Tools to your IIS installation.
That should do it. Also I run my remote debugger as Administrator, attach it to the w3wp.exe (show All Users Processes) and if it doesn't appear - reload the remote page and Refresh as if the pool goes to sleep you won't see it in the list
Good luck!

ASP.NET 5 IIS8 dnx fail

I have a problem with hosting an ASP.NET RC1 application inside IIS.
I've read through similar issues here on the site, aswell as went through official Microsoft documentation, but to no avail. I simply cannot make the app start inside IIS8 which is a must in this scenario, as I need to use it with an SSL certificate, and no relevant logs are shown unfortunately.
App is packaged under dnx-clr-win-x86.1.0.0-rc1-update2
If I try to start it inside my local VS2015, it works flawlessly. If i run the Web.cmd on the server machine from command line, app starts without any problems. However, when I try to run it through IIS, event log catches the following.
Faulting application name: dnx.exe, version: 1.0.0.20309, time stamp: 0x56e05cbb
Faulting module name: ntdll.dll, version: 6.2.9200.21815, time stamp: 0x56eaff87
Exception code: 0xc0000374
Fault offset: 0x000db583
Faulting process id: 0xae4
Faulting application start time: 0x01d1aba4b38c38e7
Faulting application path: C:\inetpub\wwwroot\approot\runtimes\dnx-clr-win-x86.1.0.0-rc1-update2\bin\dnx.exe
Faulting module path: C:\Windows\SYSTEM32\ntdll.dll
Report Id: f1f534cd-1797-11e6-9404-000d3ab185aa
Faulting package full name:
Faulting package-relative application ID:
It seems to me like it is using a wrong dnx version, but this shouldn't be possible as i published the app with:
dnu publish --runtime active --no-source
First of all thanks Clint for pushing me in the right direction.
Unfortunately I ended up spinning up a new Windows Server 2012 instance, and installing everything from the grounds up, which seems to work great at this moment. I have probably messed something when initially setting up the server, but it was a test one so no harm is done. I have a few observations for DNX to IIS publishing, which could help others with similar problems or at least mitigate at least some of the problems.
Very important steps:
If publishing from command line, publish with dnu publish --runtime active. Add --no-source when you are sure you don't need to make any quick changes to make the app work so it compiles into NuGet packages.
Try to run the app manually by running web.cmd from the publishing folder from command line. If this doesn't work, IIS is least of your problems, and you need to resolve the app startup first.
Install IIS locally on your development machine, and try to make it first work with that one, which will help you understand which issues are happening to you and how to resolve them on the real web server.
Use Event Viewer -> Windows Logs -> Application to catch .NET and HttpPlatformHandler errors during startup
Enable stdout inside wwwroot/web.config for your solution.
Battling different issues:
1. Various http 4xx and 5xx errors.
Ensure that Http Handler 1.2 is installed. If you are not sure, check the add / remove programs section in control panel, where version is clearly visible. If using a x64 system, you need to install x64 runtime
Check that you have unlocked handlers in IIS
Check that Application pool is set to No Managed code (32 bit app support was not required in my case)
2. I'm not sure I published to a good folder
Create a folder for your web app inside your wwwroot folder e.g. C:\inetpub\wwwroot\myApplicationName
Copy publishing files to that folder. Structure should be as follows:
C:\inetpub\wwwroot\myApplicationName
approot
logs
wwwroot
Create a new website in IIS and attach it to your application pool that is set to no managed code
Set the application folder to be C:\inetpub\wwwroot\myApplicationName\wwwroot
3. Event Viewer - DNX Could not write to logs folder
This is an issue with IIS permissions on the logs folder from your publishing artefacts. Right click -> Properties -> Security on the logs folder and Add permissions for the following user.
IIS_IUSRS (permissions: read and execute, list, write)
Once you do this, restart the website, and check if this issue is gone. If Event Viewer is not logging anything, check if there is a file generated in the logs folder where you should get the next troubleshooting hint.
NOTE: If you publish an app update, and you overwrite the logs folder accidentally, you will kill its permissions and this issue will return. After the initial publish, just copy approot and wwwroot, and leave the logs intact to avoid this.
4. Unable to load application or execute command 'Microsoft.AspNet.Server.Kestrel'. Available commands: web
Try to run a dnu restore before publishing manually
Ensure that you have published with --runtime active
First make sure you have httpPlatformHandler v1.2 installed. Then follow the steps and issue resolutions here. If it still is not working, I've heard some people have resolved their issues by adding a Configure1() method in Startup.cs. See the details here.
I got one of my projects working with IIS but it was a struggle. So I decided to wait until the next release. I personally don't think it's worth bothering with right now. Because it relies on dnx which is being replaced in the next release. So whatever you do to get it working, probably won't be relevant after the next release.

Process.Start() and Windows permissions on IIS

Here's the situation: I've got an asp.net webpage who calls Process.Start() to do some work. When I do this on my development machine, it simply works. When I try it on a fresh Windows 7 install, I'm having issues running that other process (which is also a .NET application).
That program depends on, among other assemblies, the OOXML SDK DLL. When trying it on the deploy machine, I'm getting "Access Denied" exceptions when the program attempts to load the OOXML assembly. When trying to simply run the exe instead of doing it by Process.Start, there are no problems and the application works. I've tried settings permissions for IIS's Application Pool, IUSR account and all the basic solutions for usual Process.Start problems. I'm lost, what am I missing?
Some more information:
1) I can't find any differences between IIS configurations for the deploy/develop machines.
2) I've installed the OOXML SDK on the deploy machine but somehow the process still can't load it
When you say you have tried setting permissions, was it on the OOXML folder? You could try checking if the IIS account has read access to the folder where the OOXML dlls are placed.

Location of symbols for WCF remote debugging

This is more a 'why does this work this way' than a 'how do I make this work' sort of question.
I have a WCF web service I am debugging remotely. It is deployed to a staging server where the VS 2010 remote debugger is installed and running as a Windows service. The permissions are correct, I can attach to processes without any problem. The issue I ran into is I couldn't consistently get the symbols to load.
I have the WCF service deployed to C:\Webs\MyService, with assorted DLLs in C:\Webs\MyService\bin. It is set up as a separate site with its own app pool. What I found is even if I had the necessary .pdb file in the bin folder, Visual Studio wouldn't load any symbols when I attached to the w3wp.exe process from my local machine. What was happening is when IIS started and the worker process was spawned, my service DLL would get copied deep under the temp ASP.NET files directory, into something like C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\root\19f82539\e55fff8f\assembly\dl3\2926a261\f625d158_f62ecd01. I found that if I manually copied the .pdb file to this folder, then symbols would be loaded and I could do the debugging.
I'm wondering why the heck it works that way, and how I can avoid having to manually copy the symbol file to this other directory. What's worse is if I had to make changes and redeploy, the worker process wouldn't recognize them. I had to restart IIS which caused a different temp directory to be created, requiring me to copy the .pdb again.
I have a similar problem, with web applications. Apparently Microsoft are aware of this: http://go4answers.webhost4life.com/Example/remote-debugging-symbols-not-loaded-207525.aspx
Hopefully, they will release a fix soon.
There is also a suggestion by BrianR on a related question, Why are no Symbols loaded when remote debugging?, saying to create a folder with the debug files and on the remote server to point the environment variable _NT_SYMBOL_PATH to it.

Access denied when creating a virtual directory via Web Deployment Project

I’m trying to use a (VS 2008) Web Deployment project in a TFS solution to deploy the web site to the (TFS 2008) build server to run web based unit tests.
For some reason, that I can't yet figure out, it is failing to create the virtual directory:
Using "CreateVirtualDirectory" task from assembly "C:\Program Files\MSBuild\Microsoft\WebDeployment\v9.0\Microsoft.WebDeployment.Tasks.dll".
Task "CreateVirtualDirectory"
Initializing IIS Web Server...
C:\Program Files\MSBuild\Microsoft\WebDeployment\v9.0\Microsoft.WebDeployment.targets(667,5): error : Access is denied.
C:\Program Files\MSBuild\Microsoft\WebDeployment\v9.0\Microsoft.WebDeployment.targets(667,5): error :
Failed to create virtual directory 'abc'.
Done executing task "CreateVirtualDirectory" -- FAILED.
The TFSService user certainly is in the Administrators group on the TFS Build machine (which is running Windows Server 2008). I don’t know what else could be wrong. I’ve checked the event log an there’s no clues there. I am able to manually create the virtual directory on that machine through the IIS console with no problem.
Any ideas what could be the problem or suggestions for how to diagnose this further?
it has got to be permissions...did you try putting the TFSService in the same Groups you are in?
Is the TFS account running under the same privileges as the account that you use to connect to IIS? Do as Craig suggested and move the TFS account into the groups that you participate in.
You're sure that the build is running under the TFSService id and not under another id set up just for builds, and which may not be in the administrator's group? I haven't done more than just play with automated builds since I do mostly solo development, but I recall setting up a separate build id when I was looking at this.
I have seen this occur when the IIS server wasn't running on the default port. I'd recommend checking IIS to see if it's running on port 80 as a step to diagnose your issue further.
I eventually managed to get deployment working by calling the _CopyWebApplication build target of the web application from my TFS build script (after manually creating the IIS virtual directory).
I had to add an additional target though to get linked files in the project to be copied also as the built in _CopyWebApplication target doesn't include those.

Resources