I am using VS2013 Premium to publish a site to Windows Server 2012.
All files publish ok except these:
SqlServerTypes\x64\msvcr100.dll
SqlServerTypes\x64\SqlServerSpatial110.dll
SqlServerTypes\x86\msvcr100.dll
SqlServerTypes\x86\SqlServerSpatial110.dll
I get this kind of errors for each of the above files I tried to publish:
Web deployment task failed. (The file 'msvcr100.dll' is in use. Learn more at: http://go.microsoft.com/fwlink/?LinkId=221672#ERROR_FILE_IN_USE.)
Interrestingly, these files were published the first time (when they were not on the server), then they are no longer overwritten. Tried with 2 different web servers.
I have followed the guide here:
http://blogs.msdn.com/b/webdev/archive/2013/10/30/web-publishing-updates-for-app-offline-and-usechecksum.aspx
...But it only managed to put the site offline (VS is placing the app_offline.htm) but publish still fails with the same error.
All other files publish perfectly.
Any ideas?
You can take you app offline during publishing which hopefully should free up the lock on the file and allow you to update it.
I blogged about this a while back. The support outlined was shipped inside of the Azure SDK and Visual Studio Update. I don't remember the exact releases but I can find out if needed. Any update dating around/after that blog post should be fine.
Prerequisites:
VS 2012 + VS update / VS 2013 + VS Update / VS2015
MSDeploy v3
Note: if you are publishing from a CI server the CI server will need the updates above as well
Edit the publish profile
In VS when create a Web Publish profile the settings from the dialog are stored in Properties\PublishProfiles\ as files that end with .pubxml. Note: there is also a .pubxml.user file, that file should not be modified
To take your app offline in the .pubxml file add the following property.
<EnableMSDeployAppOffline>true</EnableMSDeployAppOffline>
Notes
ASP.NET Required
The way that this has been implemented on the MSDeploy side is that an app_offline.htm file is dropped in the root of the website/app. From there the asp.net runtime will detect that and take your app offline. Because of this if your website/app doesn't have asp.net enabled this function will not work.
Cases where it may not work
The implementation of this makes it such that the app may not strictly be offline before publish starts. First the app_offline.htm file is dropped, then MSDeploy will start publishing the files. It doesn't wait for ASP.NET to detect the file and actually take it offline. Because of this you may run into cases where you still run into the file lock. By default VS enables retrys so usually the app will go offline during one of the retrys and all is good. In some cases it may take longer for ASP.NET to respond. That is a bit more tricky.
In the case that you add <EnableMSDeployAppOffline>true</EnableMSDeployAppOffline> and your app is not getting taken offline soon enough then I suggest that you take the app offline before the publish begins. There are several ways to do this remotely, but that depends on your setup. If you only have MSDeploy access you can try the following sequence:
Use msdeploy.exe to take your site offline by dropping app_offline.htm
Use msdeploy.exe to publish your app (_make sure the sync doesn't delete the app_offline.htm file_)
Wait some amount of time
Publish the site
Use msdeploy.exe to bring the app online by deleting app_offline.htm
I have blogged how you can do this at http://sedodream.com/2012/01/08/howtotakeyourwebappofflineduringpublishing.aspx. The only thing that is missing from that blog post is the delay to wait for the site to actually be taken offline. You can also create a script that just calls msdeploy.exe directly instead of integrating it into the project build/publish process.
I have found the reason why the solution at
http://blogs.msdn.com/b/webdev/archive/2013/10/30/web-publishing-updates-for-app-offline-and-usechecksum.aspx
did not work for the original poster, and I have a workaround.
The issue with the EnableMSDeployAppOffline approach is that it only recycles the app domain hosting the application. It does not recycle the app pool worker process (w3wp.exe) which the app domain lives in.
Tearing down and recreating the app domain will not affect the Sql Server Spatial dlls in question. Those dlls are unmanaged code which are manually loaded via interop LoadLibray calls. Therefore the dlls live outside the purview of the app domain.
In order to release the files locks, which the app pool process puts on them, you need to either recycle the app pool, or unload the dlls from memory manually.
The Microsoft.SqlServer.Types nuget package ships a class which is used to load the Spatial dlls called SqlServerTypes.Utilities. You can modify the LoadNativeAssemblies method to unload the unmanaged dlls when the app domain is unloaded. With this modification when msdeploy copys the app_offline.htm the app domain will unload and then unload the managed dlls as well.
[DllImport("kernel32.dll", SetLastError = true)]
internal extern static bool FreeLibrary(IntPtr hModule);
private static IntPtr _msvcrPtr = IntPtr.Zero;
private static IntPtr _spatialPtr = IntPtr.Zero;
public static void LoadNativeAssemblies(string rootApplicationPath)
{
if (_msvcrPtr != IntPtr.Zero || _spatialPtr != IntPtr.Zero)
throw new Exception("LoadNativeAssemblies already called.");
var nativeBinaryPath = IntPtr.Size > 4
? Path.Combine(rootApplicationPath, #"SqlServerTypes\x64\")
: Path.Combine(rootApplicationPath, #"SqlServerTypes\x86\");
_msvcrPtr = LoadNativeAssembly(nativeBinaryPath, "msvcr100.dll");
_spatialPtr = LoadNativeAssembly(nativeBinaryPath, "SqlServerSpatial110.dll");
AppDomain.CurrentDomain.DomainUnload += (sender, e) =>
{
if (_msvcrPtr != IntPtr.Zero)
{
FreeLibrary(_msvcrPtr);
_msvcrPtr = IntPtr.Zero;
}
if (_spatialPtr != IntPtr.Zero)
{
FreeLibrary(_spatialPtr);
_spatialPtr = IntPtr.Zero;
}
};
}
There is one caveat with this approach. It assumes your application is the only one running in the worker process that is using the Spatial dlls. Since app pools can host multiple applications the file locks will not be released if another application has also loaded them. This will prevent your deploy from working with the same file locked error.
There are known issues with IIS and file-locks (why they aren't solved yet i dont know).
The question i want to ask however is if you even need to re-deploy these files?
I recognize the file-names and recall them to be system-files which should either already be present on the server or simply not need to be re-deployed.
I am not very experienced when it comes to IIS but i have ran into this problem before and several of my more experienced co-workers have told me that this is as i said a known IIS-issue and i believe the answer to your question is:
Avoid deploying unnecessary files.
try again
Reset website
try again
iisreset
I think what would be the easiest thing to do is to make these dll's as CopyLocal as true. I am assuming these dll's are pulled out from program files folder. Try marking them as copylocal true and do a deployment.Try to stop any IIS local process running in your local machine.
Watch out you don't have one of those new-fangled cloud backup services running that is taking file locks - and also you don't have things open in explorer or a DLL inspection tool.
I think it's kind of ridiculous that MS doesn't make better provisions for this problem. I find that 9 times out of 10 my deployment works just fine, but then as our traffic increases that can become 1 in 10 times.
I am going to solve the problem with :
two applications MySite.A and MySite.B, where only one is running at a time.
I always then deploy to the dormant site.
If there's a problem during the deployment it will never cause the whole site to go down.
If there's a major problem after deployment you can revert back very easily.
Not quite sure how I'm implementing it, but I think this is what I need to do.
please I have an application that needs IIS to run, now IIS needs to be turned on manually in the windows control panel. But I want to avoid this process and automatically turn on IIS how can I do this. Thank you
The IIS is implemented as a Windows Service. So the only thing necessary is you to make sure the Startup type of the W3SVC is set to Automatic. It is set to automatic by default, so your question implies that someone has changed it to one of the other options.
So open the services and look for World Wide Web Publishing Service. Then double-click on it and you'll be present with its properties. From the Startup type drop-down choose Automatic. Restart your computer. The IIS will start automatically.
UPDATE
Based on the OP's comment I assume the re-configuration of the service is necessary.
Unfortunately, there is no managed class to change the service startup type. You can go through P/Invoke and call the native Windows API. Another option is to utilize the WMI. But the quickest way is to spawn a privileged cmd.exe from your application installer and run the following:
sc config w3svc start=auto
However, this is not a bulletproof solution as someone else might later change it again to demand or even disabled.
If you're looking for a mechanism to start the service at the execution of the application installer, you might want the ServiceController class. It can start the service but it cannot change its startup type. Here is the official documentation.
So you could do something like this in your code:
using (var w3cvs = new ServiceController("W3Svc"))
{
if (w3cvs.Status == ServiceControllerStatus.Stopped)
{
w3cvs.Start();
}
}
I'm wondering if there is an event that can be handled or a method that can be overridden that takes place before the Web.config file is parsed and monitored by the asp.net 3.5 application / AppDomain lifecycle.
The practical reason for this is that I'd like to be able to write the Web.config file from a copy in the database while the application is starting up depending on the deployment environment. The reason for this is because we have a manual application deployment process and a web farm. Web.config changes often fall through the cracks or fail to be propagated to all servers on the web farm because of the manual process. Unfortunately we are going to be staying with a manual deployment process for the foreseeable future. This being the case, it would be great if there was a way for an app to go grab its web config on first startup. If I could get that working, the next logical thing to do would be to create a SQL dependency/notification to cause an AppDomain unload whenever the config file is changed in the databases so new changes would be pulled and written.
So far the only way I've figured out how to manage this is to do something like the below psuedocode that has the unfortunate side effect of causing two application load cycles per attempted start. Additionally, I'm pretty sure the first request that comes in if the app is idle will go up in smoke due to the restart.
// PSEUDOCODE
// In global.asax.cx
protected void Application_Start(object sender, EventArgs e)
{
bool loadConfigFileFromDB = GetConfigLoadOptionFromLoadOptionsConfigFile();
string webConfigPath = GetWebConfigPath();
if (loadConfigFileFromDB) // Most likely false in development so debugging works
{ // with a local web.config
if (File.Exists(webConfigPath)) // We are not starting up for the first time
{ // since app was deployed
if (File.GetCreationTime(webConfigPath) < DateTime.Now.AddMinutes(-1))
{
// Web config is more than a minute old, so chances are we
// aren't in an app restart after writing the config.
WriteWebConfigFromDatabase(); // This will cause a restart.
}
// else, web.config was probably just written and we are in a
// restart after writing the config. In this case, let the application continue on
}
else // First time starting up, so it's safe to assume we can write
{ // the config and restart.
WriteWebConfigFromDatabase(); // This will cause a restart.
}
}
}
Obviously a build or deployment task would be the best way handle replacing the Web.config per environment, but unfortunately I am not in a situation where that can happen.
EDIT
The intent of this is not to have dynamic settings while the app is running, it is to help manage differing Web.config files per environment (Stage/QA/Production). Example, in a separate non-Web.config file we'd have an environment setting. After deployment when the app fired up, it would use the settings in this file (the environment and the connection string) to go pull and write the web config for that environment. The settings would not be dynamic after application startup.
You are doing weird thing.
UPDATE (also removed unrelated text):
Ok. So you need to automatically propagte new version of the application to all servers. I do not see a reason to do it from application itself. Instead it should be another utility/batch/installer that does this kind of stuff.
I believe ASP.NET application deploying itself will hit a lot of issues (what if you will need to deploy assemblies along with web.config)?
I think simple batch-xcopy approach will do the job for you:
Create a .bat file that accepts 1 parameter:Envoronment=[Stage/QA/Production].
Copy all the required files to a separate temporary directory (so you can modify things without touching the original code).
Modify web.config and other things you need (you can use some utility for that) as per Environment parameter.
XCOPY all files to all required servers as per Environment parameter.
There is no need to incorporate the deployment process into the application itself.
For Windows applications it is ok as you can use bootstrapper, but not for ASP.NET.
Application_End is the closest event - it fires just prior to the unloading of the AppDomain for the web application. You could just update the Web.config file there.
In principle it should work - the AppDomain is unloaded, so the config has to be reloaded when the AppDomain starts up again, by which time the latest config would already exist on disk.
Also, I'm assuming that ASP.NET stops monitoring the Web.config for further changes, as it has already decided to shut down the application - my only concern is that writing the file again would cause an infinite loop to occur.
It can't hurt to try. It is a weird thing to do though. It would be good to have some more background on why you are needing to do this in the first place.
I'm wondering if there is an event
that can be handled or a method that
can be overridden that takes place
before the Web.config file is parsed
and monitored by the asp.net 3.5
application / AppDomain lifecycle.
After doing a few days of research I'm going to say the answer to this question is: No, there is no such event that can be handled or method that can be overidden. If someone ever comes along and can show otherwise, I will deselect this as the answer to the question.
Is it possible to update the site dll for a precompiled site without stopping IIS.
Currently, if I try to just copy the new file to overwrite the current file, All users receive runtime errors while the file is being copied. Is there a way to avoid this?
even if you don't stop, any change to the web.config file, BIN folder, App_Data or App_Code will force the .NET compiler to perform ...
and you will loose any Session variables in memory.
What I do is to use Session State in SQL Mode and if your system is set up like this, user will remain in the site (after a longer exposition to a page reload)
.NET will still invoke the compiler in order to compile the new set of instructions but soon it is done, all sessions will be read from SQL Server and because they are still there (and not lost with a memory refresh) users will remain in the website with current credentials.
it is a little bit slower than In-Memory Session State, but much more reliable, specially with Shared hosting :) this is the way to increse/decrese the minutes in your session, as Shared hosting do not allow it to change even if you do
Session.Timeout = 5;
their machine configuration will override everything you do, with SQL Session State, you will be able to set your time as this is all made by SQL Server.
Fell free to read this article to know how everything is done.
Hope it helps.
To deploy a new version of our website we do the following:
Zip up the new code, and upload it to the server.
On the live server, delete all the live code from the IIS website directory.
Extract the new code zipfile into the now empty IIS directory
This process is all scripted, and happens quite quickly, but there can still be a 10-20 second downtime when the old files are being deleted, and the new files being deployed.
Any suggestions on a 0 second downtime method?
You need 2 servers and a load balancer. Here's in steps:
Turn all traffic on Server 2
Deploy on Server 1
Test Server 1
Turn all traffic on Server 1
Deploy on Server 2
Test Server 2
Turn traffic on both servers
Thing is, even in this case you will still have application restarts and loss of sessions if you are using "sticky sessions". If you have database sessions or a state server, then everything should be fine.
The Microsoft Web Deployment Tool supports this to some degree:
Enables Windows Transactional File
System (TxF) support. When TxF support
is enabled, file operations are
atomic; that is, they either succeed
or fail completely. This ensures data
integrity and prevents data or files
from existing in a "half-way" or
corrupted state. In MS Deploy, TxF is
disabled by default.
It seems the transaction is for the entire sync. Also, TxF is a feature of Windows Server 2008, so this transaction feature will not work with earlier versions.
I believe it's possible to modify your script for 0-downtime using folders as versions and the IIS metabase:
for an existing path/url:
path: \web\app\v2.0\
url: http://app
Copy new (or modified) website to server under
\web\app\v2.1\
Modify IIS metabase to change the website path
from \web\app\2.0\
to \web\app\v2.1\
This method offers the following benefits:
In the event new version has a problem, you can easily rollback to v2.0
To deploy to multiple physical or virtual servers, you could use your script for file deployment. Once all servers have the new version, you can simultaneously change all servers' metabases using the Microsoft Web Deployment Tool.
You can achieve zero downtime deployment on a single server by utilizing Application Request Routing in IIS as a software load balancer between two local IIS sites on different ports. This is known as a blue green deployment strategy where only one of the two sites is available in the load balancer at any given time. Deploy to the site that is "down", warm it up, and bring it into the load balancer (usually by passing a Application Request Routing health check), then take the original site that was up, out of the "pool" (again by making its health check fail).
A full tutorial can be found here.
I went through this recently and the solution I came up with was to have two sites set up in IIS and to switch between them.
For my configuration, I had a web directory for each A and B site like this:
c:\Intranet\Live A\Interface
c:\Intranet\Live B\Interface
In IIS, I have two identical sites (same ports, authentication etc) each with their own application pool. One of the sites is running (A) and the other is stopped (B). the live one also has the live host header.
When it comes to deploy to live, I simply publish to the STOPPED site's location. Because I can access the B site using its port, I can pre-warm the site so the first user doesn't cause an application start. Then using a batch file I copy the live host header to B, stop A and start B.
OK so since everyone is downvoting the answer I wrote way back in 2008*...
I will tell you how we do it now in 2014. We no longer use Web Sites because we are using ASP.NET MVC now.
We certainly do not need a load balancer and two servers to do it, that's fine if you have 3 servers for every website you maintain but it's total overkill for most websites.
Also, we don't rely on the latest wizard from Microsoft - too slow, and too much hidden magic, and too prone to changing its name.
Here's how we do it:
We have a post build step that copies generated DLLs into a 'bin-pub' folder.
We use Beyond Compare (which is excellent**) to verify and sync changed files (over FTP because that is widely supported) up to the production server
We have a secure URL on the website containing a button which copies everything in 'bin-pub' to 'bin' (taking a backup first to enable quick rollback). At this point the app restarts itself. Then our ORM checks if there are any tables or columns that need to be added and creates them.
That is only milliseconds downtime. The app restart can take a second or two but during the restart requests are buffered so there is effectively zero downtime.
The whole deployment process takes anywhere from 5 seconds to 30 minutes, depending how many files are changed and how many changes to review.
This way you do not have to copy an entire website to a different directory but just the bin folder. You also have complete control over the process and know exactly what is changing.
**We always do a quick eyeball of the changes we are deploying - as a last minute double check, so we know what to test and if anything breaks we ready. We use Beyond Compare because it lets you easily diff files over FTP. I would never do this without BC, you have no idea what you are overwriting.
*Scroll to the bottom to see it :( BTW I would no longer recommend Web Sites because they are slower to build and can crash badly with half compiled temp files. We used them in the past because they allowed more agile file-by-file deployment. Very quick to fix a minor issue and you can see exactly what you are deploying (if using Beyond Compare of course - otherwise forget it).
Using Microsoft.Web.Administration's ServerManager class you can develop your own deployment agent.
The trick is to change the PhysicalPath of the VirtualDirectory, which results in an online atomic switch between old and new web apps.
Be aware that this can result in old and new AppDomains executing in parallel!
The problem is how to synchronize changes to databases etc.
By polling for the existence of AppDomains with old or new PhysicalPaths it is possible to detect when the old AppDomain(s) have terminated, and if the new AppDomain(s) have started up.
To force an AppDomain to start you must make an HTTP request (IIS 7.5 supports Autostart feature)
Now you need a way to block requests for the new AppDomain.
I use a named mutex - which is created and owned by the deployment agent, waited on by the Application_Start of the new web app, and then released by the deployment agent once the database updates have been made.
(I use a marker file in the web app to enable the mutex wait behaviour)
Once the new web app is running I delete the marker file.
The only zero downtime methods I can think of involve hosting on at least 2 servers.
I would refine George's answer a bit, as follows, for a single server:
Use a Web Deployment Project to pre-compile the site into a single DLL
Zip up the new site, and upload it to the server
Unzip it to a new folder located in a folder with the right permissions for the site, so the unzipped files inherit the permissions correctly (perhaps e:\web, with subfolders v20090901, v20090916, etc)
Use IIS Manager to change the name of folder containing the site
Keep the old folder around for a while, so you can fallback to it in the event of problems
Step 4 will cause the IIS worker process to recycle.
This is only zero downtime if you're not using InProc sessions; use SQL mode instead if you can (even better, avoid session state entirely).
Of course, it's a little more involved when there are multiple servers and/or database changes....
To expand on sklivvz's answer, which relied on having some kind of load balancer (or just a standby copy on the same server)
Direct all traffic to Site/Server 2
Optionally wait a bit, to ensure that as few users as possible have pending workflows on the deployed version
Deploy to Site/Server 1 and warm it up as much as possible
Execute database migrations transactionally (strive to make this possible)
Immediately direct all traffic to Site/Server 1
Deploy to Site/Server 2
Direct traffic to both sites/servers
It is possible to introduce a bit of smoke testing, by creating a database snapshot/copy, but that's not always feasible.
If possible and needed use "routing differences", such as different tenant URL:s (customerX.myapp.net) or different users, to deploy to an unknowing group of guinea pigs first. If nothing fails, release to everyone.
Since database migrations are involved, rolling back to a previous version is often impossible.
There are ways to make applications play nicer in these scenarios, such as using event queues and playback mechanisms, but since we're talking about deploying changes to something that is in use, there's really no fool proof way.
This is how I do it:
Absolute minimum system requirements:
1 server with
1 load balancer/reverse proxy (e.g. nginx) running on port 80
2 ASP.NET-Core/mono reverse-proxy/fastcgi chroot-jails or docker-containers listening on 2 different TCP ports
(or even just two reverse-proxy applications on 2 different TCP ports without any sandbox)
Workflow:
start transaction myupdate
try
Web-Service: Tell all applications on all web-servers to go into primary read-only mode
Application switch to primary read-only mode, and responds
Web sockets begin notifying all clients
Wait for all applications to respond
wait (custom short interval)
Web-Service: Tell all applications on all web-servers to go into secondary read-only mode
Application switch to secondary read-only mode (data-entry fuse)
Updatedb - secondary read-only mode (switches database to read-only)
Web-Service: Create backup of database
Web-Service: Restore backup to new database
Web-Service: Update new database with new schema
Deploy new application to apt-repository
(for windows, you will have to write your own custom deployment web-service)
ssh into every machine in array_of_new_webapps
run apt-get update
then either
apt-get dist-upgrade
OR
apt-get install <packagename>
OR
apt-get install --only-upgrade <packagename>
depending on what you need
-- This deploys the new application to all new chroots (or servers/VMs)
Test: Test new application under test.domain.xxx
-- everything that fails should throw an exception here
commit myupdate;
Web-Service: Tell all applications to send web-socket request to reload the pages to all clients at time x (+/- random number)
#client: notify of reload and that this causes loss of unsafed data, with option to abort
# time x: Switch load balancer from array_of_old_webapps to array_of_new_webapps
Decomission/Recycle array_of_old_webapps, etc.
catch
rollback myupdate
switch to read-write mode
Web-Service: Tell all applications to send web-socket request to unblock read-only mode
end try
A workaround with no down time and I am regularly using is:
Rename running .NET core application dll to filename.dll.backup
Upload the new .dll (web application is available and serving the requests while file is being uploaded)
Once upload is complete recycle the Application Pool. Either Requires RDP Access to server or function to recycle application pool in your hosting control panel.
IIS overlaps the app pool when recycling so there usually isn’t any downtime during a recycle. So requests still come in without every knowing the app pool has been recycled and the requests are served seamlessly with no downtime.
I am still searching for more better method than this..!! :)
IIS/Windows
After trying every possible solution we use this very simple technique:
IIS application points to a folder /app that is a symlink (!) to /app_green
We deploy the app to /app_blue
We change the symlink to point to /app_blue (the app keeps working)
We recycle the application pool
Zero downtime, but the app does choke for 3-5 seconds (JIT compilation and other initialization tasks)
Someone called it a "poor man's blue-green deployment" without a load balancer.
Nginx/linux
On nginx/linux we use "proper" blue-green deployment:
nginx reverse proxy points to localhost:3000
we deploy to localhost:3001
warmup the localhost:3001
switch the reverse proxy
shot down localhost:3000
(or use docker)
Both windows and linux solutions can be easily automated with powershell/bash scripts and invoked via Github Actions or a similar CD/CI engine.
I would suggest keeping the old files there and simply overwriting them. That way the downtime is limited to single-file overwrite times and there is only ever one file missing at a time.
Not sure this helps in a "web application" though (i think you are saying that's what you're using), which is why we always use "web sites". Also with "web sites" deploying doesn't restart your site and drop all the user sessions.