I have a website with huge number of pages, i keep pre-compiled version (with fixed naming) on production server.
Every time i make any change on my code i have to Publish the whole website just for a small change.
It takes about an hour to get the website published before i can deploy my changes to production server.
Is there a way to publish only a batch of pages so that the Publish process is faster?
Is there any other option to save the publishing time?
NOTE: By publishing I mean pre-compiling
Any suggestions are welcome.
If you're modifying only the html tags (nto the server side tags) or css, you can deploy only the part you changed.
If it's compiled code you got no choice.
I think you might have to ask your self why it's taking an hour to publish your web site ? Is your compilation time that much long ? .
One method to reduce the compile time, and size of a web-site project is to split your website into several smaller and more maintainable sites.
You can still deploy these separate publishes together in production.
References to pages from other projects work perfectly. All your pages within the same application on IIS will share the same session. So to an end user, this will still appear to be one website.
Since you reduce the work to be done while publishing any given module publishes will be faster. Divide your modules as per what you see as a suitable batch.
You must be aware of this, but I will say it just for completion. When you publish a website you get the option,Use fixed naming and single page assemblies. Select this to have a different dll for each page in your bin directory. You only need to upload the pages and corresponding dlls where you made changes. If upload time is a concern, this will take care of it.
Microsoft doesn't really have an idea of "pre-compiling" if you notice your pages have 3 components to them, the *.designer, *.aspx *.cs. The *.cs all needs to be compiled into a *.dll to be deployed to your website. Traditionally there are two types of executables, exe's and dll's. Asp.Net websites are compiled into a dll for all the code behinds that run on the server. Microsoft does not have a way to "half" compile a dll and then merge it with the other half you haven't changed.
If your website is taking that long, to compile & deploy. I would suggest you have more of an architecture problem then a code problem. Where I work our main website is 3,000,000+ lines of code, to accomplish everything the user needs to do and does. We don't take an hour to deploy. however what we have done is broken our business logic up into a number of dll's over 100 dll's and our website project in and of it self is just the aspx and the bare bones code behind to drive the flow through to our business logic. This allows us to alter x number dll's with our changes to support a new feature, We don't have to deploy all 100 dll's every time just the ones that have changed, that's the nature of dll's. if our business logic was 100% contained in our website project, then our compile, deployment would be significantly longer.
You want to consider refactoring your code into dll's. Another option if you're not married to the ASPX/ASP.NET solution is to consider an ASP/MVC.NET solution. I would consider refactoring your site. If it takes that long there's some serious issues, even if you could break Data access into a separate dll, then you wouldn't have to constantly compile & deploy the dll which handles your data access, every time you changed the website, only when you changed the Data access layer as well.
As previous posts mention, you cannot do this in an automatic fashion, but you could manually deploy your files if you want to reduce your publishing time.
When publishing a website, all code files for your site are compiled into a single .dll file in the website bin folder and all .aspx files are be deployed to their relevant paths.
To update the site manually, simply build the website on your local machine to create an updated .dll and overwrite the .dll in the bin folder on the production server. If the source/HTML has been modified on any of your actual pages/.aspx files then you will also need to copy them over.
Steps:
Build website locally
Overwrite production server .dll with locally built .dll
Copy any .aspx pages to production server where HTML/Source modified
Very simple.
Have all the HTML content stored in separate files to the code. A database would be an excellent idea. All one would have to do to change some text or swap an image would be to go into the database or file for that content and change a few tags. I recommend MySQL.
:)
Related
Our development team has used aspnet_compiler as part of our build process for years to verify our website build. I have noticed that when the process encounters a precompile error, it immediately shuts down. Sometimes, if you fix the issue and restart, it will fail again with another unrelated error that actually existed the first time. I am just wondering if there is a way for it to not "die" and continue the precompile and then list ALL errors that it finds so we don't have to do the iterative process of fixing the errors one at a time.
I have looked at this documentation: https://learn.microsoft.com/en-us/visualstudio/msbuild/aspnetcompiler-task?view=vs-2019 but nothing there seems like it supplies what I am looking for.
do you mean that VS shutdowns, or the external command line build process shuts down? One real problem area is the .net folder called app_code. While you can do a build->compile?
You note that code in that folder does NOT get compiled when you do this. It is only compiled by the build process. Even during debugging, you note that compile errors are not seen (or caught) unless you run (try) the site. As a result? I don't use that folder anymore. The 2nd huge problem is royslen editor code. I have started using that ability's for strings to span multiple lines in - really nice for in-line sql.
However, I find then if you let the web site build, it does NOT support + see and allow such Rosylen formatted code. (so upon run of the site, I get errors). (this might well be due to the web site running IIS 2016 - too old for that new code formatting support (at least in vb.net it is).
So, all I did was create my OWN folder for code and NOT use the .net one ().
Note that you create the folder, but remember for EACH code module, or class you drop into that folder? You have to individual set each to be compiled. (it is the default), but be careful if you import code via add. So you see (and want) to use this option:
So, I of course don't want the source included - and I don't require that since this is a web applicaiton vs a web site.
Thus, I don't get/see/have any runtime compile of my code by the web site compiler process - it all done during the build and even debug process in VS.
And then their is the build for a web site publish. Thus during a deploy build (and publish), I don't get any surprises either.
In other words, the GREAT advantange of a web site applcation is you don't let the web site comile code for you.
You do have to mark the code module as per above.
As a result, this folder behaves like any web form with code behind. The standard debug and build process during development will thus compile all my code - and catach errors.
If you use the built in (and special) folder app_code, then such code only compiles WHEN you run the site, and worse it is the web site that does this compile - NOT VS.
(but of course you publish build process ALSO does this compile!!!).
I need (want) a regular build + compile during the development process to catch and compile all that code (else debugging, and worse compiling means you only can find out issues at web site run time - and that's way too late for my tastes).
So, I don't bother with app_code anymore at all. Now I am lucky, since I am creating (using) a web site application, as opposed to a asp.net web site. (and yes, there is a massive difference).
Asp.net web site = each page and code behind will compile on demand. This choice is perferred for two big reasons:
First, you can update one web page (and code)- deploy that one web page + code. The web server will figure this out - and re-compile that one page on demand. This makes updates and maintains of the site OH SO VERY much easier.
And it also means in most cases that the whole site can be published to a sub-site folder on the hosted web server. And in fact most really cheap low cost web site hosting MUST use this option (web site applications in most cases can't be published to those lower cost .net hosting sites). (thus use web sites, not web site applications).
However, if you lucky, and you have FULL use of IIS and a server dedicated to JUST running your web site? Well, then you are MUCH MUCH better off to go with a aps.net web site application. And this also means you can say setup custom logon providers, and also configure things like re-direction or things like a custom web handier. In effect, this choice means that you as a developer not only have full use of the base starting page, but can directly change/configure the web server to your liking. This includes the base web config.
This choice is often not avaible on lower cost hosting plans. it requires that you have full IIS services, and IIS services is running your whole site, and the base starting page is your site. (you can publish to root on web hosting, but you still using THEIR EXISTING copy of IIS services, and you can't control things like creating a custom authneticaion (logon) provider. And you also can't create re-directs (custom one).
However, in both cases/choices app_code folder behaves like a compile on demand folder. Thus bugs and issues in that folder will thus not be found until you run the web site, or do a full deploy that then does the full site build. and as noted, you can't use Rosylen eiditor souce code features, since the web compiler might be a few verisons old. You be running the SAME .net version, but the older compiler does not support source code formatting with the new Rosylen editor features.
Since a web site application will compile everything (whole site) in your application down to a single .dll (and of course referenced libraries), then then you don't have on-demand occurring by the web site compiler (after you deploy). Of course the publish wizard does have options to combine all those .dll's into one - not a big deal either way.
So, the one exception is app_code. So I just stopped using it, and I see little reason to use that special .net folder anyway.
So, it was never clear if VS is shutting down, or your build process is seeing errors not being caught during the regular development cycle with VS - if this is your issue, then the above approach should solve the problem for you.
I have faced a lot of issues with Publishing like when you need to make small changes on the code, sometimes the generated DLL file (the dll file for example of default.aspx.CS when published) cannot be recognized by IIS saying the codebehind is wrong or something. Sorry for not remembering the exact error message. I am hoping you know what I mean at this point.
Therefore, I usually do a simple Copy Paste operation instead of Publishing.
Could you tell me what am I missing by NOT using the Publish method? How is Publishing better? Or which one do you prefer, why?
Basically its a pros and cons situation.
Thankyou
Well, it depends on what you mean by "copy":
With Publishing you have options to pre-compile all or part of your application. You can publish to a local folder in your file system (instead of your target/host) and then copy the updated file(s) (only). If you are making "code behind" (c#/vb code) changes, this means you'll likely only need to "copy"/overwrite dlls. Goes without saying that if you've made "content" changes (html/razor/script/etc) changes, then you'd need to copy/overwrite those as well.
If you're new to deployment, you may find yourself simply copying/overwriting "everything" which is the safest way to go. Once you get more experience, you'll "recognize" which assets you only need to update (one or a few dlls and or content code, instead of "everything"). There's no magic to this, usually, its a matter of just looking at the timestamp of the dll/file after you've published (locally) or rebuild your web application.
I'd recommend doing a local publish so you can see what is actually needed on your server. The files published to your local file system/folder is what needs to be on your host/server. Doing so will visualize and remove whatever "mystery" there is to Publishing:
you'll see what is actually needed (on your server) vs. what's not
you'll see the file timesstamps which will help you recognize what files were actually changed vs those that weren't (and therefore don't need to be updated).
once you get the hang of it, you will not need to "copy"/ftp "everything" and just update files that were actually modified (only).
So "copy" can mean the above, or if you are saying you will simply copy all of your development code (raw (vb/cs)html/cs/vb) to your host, then that means your site will be dynamically compiled as each resource is needed/requested (nothing is pre-compiled). Also "easy" but you do lose pre-compilation which means there is a delay when each of your web pages are requested/needed (ASP.net needs to dynamically compile). Additionally, you are also exposing your source code on the server. It may not mean much depending on your situation, but it is one more thing to consider.
Here's more info on pre-compilation and options.
Assuming we consider an aspx page and its aspx.cs code behind file, there are three alternative ways of deploying your site:
You can copy both to iis. The aspx will be compiled to .cs upon the first request and then both .cses will be compiled to a temp .dll
You can "publish" to iis, this will compile the code behind class to .dll but will copy the aspx untouched. The aspx will be translated to .cs and then to .dll upon the first request
You can "publish" the site and then manually precompile it with the aspnet_compiler. Publishing will compile the code behind to .dll as previously but then precompilation will clear out your .aspx files by removing their content and moving the compiled code to yet another .dll.
All three models have their pros and cons.
First one is the easiest to update incrementally but in the same time is the most open to unwanted modifications.
Second is also easy, can be invoked from vs, it closes the possibility of some unwanted modifications at the server but .aspxses still need time to compile upon the first request
Third takes the time and some manual actions but prevents any changes and also speeds up the warm up of the site as the compilation of assets is not necessary. It is great for shared environments.
I have a solution with a fair few projects, 3 of them web-based (WCF in IIS / MVC site). When the solution builds, it dumps each of the components of this distributed system in a 'Build' folder. Running the 'configurator' part of the whole output will set up the system in the cloud automatically. It's very neat :) However, the Web Deploy Projects are a major pain. They "build" (i.e. deploy) every, single, time I build - even when no changes have been made to their respective projects.
Changed a single line of code? Look forward to waiting around a minute for the 3 web projects to redeploy.
[These projects are VERY straightforward at the moment - two have a single .svc and one .ashx file - the other is an MVC app with ~5 views]
I realise I can change solution configurations to not 'build' them, but I've been doing that and it's very easy to log on the next day and forget about it, and spend a couple of hours tracking down bugs in distributed systems due to something simply having not been built.
Why I use Web Deploy Projects? Well, because I need all pages + binaries from the web project. The build output for the project itself is the 'bin' folder, so no pages. The entire project folder? It has .cs, .csproj and other files I don't want included.
This will be building on build servers eventually, but it's local at the moment. But I want a quick way of getting the actual output files from the web project to my target folder. Any ideas?
Not sure if this will help in your situation, (plug for own project coming up), but I am working on a project to help ease IIS deployments:
https://github.com/twistedtwig/AutomatedDeployments
The idea being you can use config files for IIS (app Pool, applications and websites) to automate the creation and update of sites locally (dev machines) or remotely (test and production machines).
It is still a work in progress but is ready to be used in production systems.
using the package creation as a post build step might get you closer to what you want, (don't believe it includes all the extra files), but that would still build it each time, (although if code hasn't changed it should not rebuild unless you choose rebuild all projects).
In the end I created a utility/tool which, given a project file, XCOPYies the project folder for the web project to a target location, then looks in said project file and deletes anything that doesn't have Build Action set to Content. Very quick and effective.
I know it is still in RC but VS2012 does have a neat feature when doing publish that it detects the changes and publishes only those. Might be something a little deeper down in the build where it does an automatic publish too.
You can take a look to the Octopus project: http://octopusdeploy.com/
Deployment based on nuget packages.
For example how this site is organized?
What i do not understand is what they upload to the Microsoft server?
I have created, with Visual studio, a very small web-page and i have to upload the whole site, even after the smallest change...
The usual approach is to replace everything with xcopy or the publish function in visual-studio, and in some cases replacing everything is the only approach - for example if you're using the web-application project model everything gets packaged into a single assembly and there you go - even to apply a small change you'll have to re-deploy the whole thing.
An alternative to this could be the Website model in visual studio, using which you should be able to deploy single code files on your server and they should be picked-up if you re-start the website from the IIS management tool. This model - in fact - works in a different way compared to the web-application project model. It's just a bunch of code files that will be dynamically compiled by the ASP.NET runtime.
Even if possible though - I wouldn't suggest the approach of deploying single files - as this is easily error prone (you deploy the code-behind and could easily forget to deploy the aspx counterpart, or similar). Unless you're delpoying Gigs of stuff over slow-networks, redeploying the whole thing is always the safest bet.
Have a look at this and this interesting links to find out more about website and web-application project models in visual studio.
It really depends a lot on how you're building your app.
If you're in VS and you're doing an ASP.Net site, then you can either do it as a Website Project, or as a Web Application project.
in the former case, your files will remain as aspx and .aspx.cs files and you xcopy (or FTP) whichever files change. if you want logic that's outside the scope of a single page, you'll either create a separate class library project or else use the App_code directory.
In the latter case, you'll compile all the logic into one or more .dll files that get copied to your site's /bin directory, and any number of aspx files that can either stay as such or be embedded (recommend leaving them as aspx files). Again, if an aspx file changes, you just movethe one that changed, if anything in the dll changes, you replace a whole dll.
All that said, a huge chunk of what's on the site you posted is probably being pulled out of the database. Most sites now dont' have content on pages, they just have organizational (view) logic on paes, and have other classes which fetch the actual content out of a database to serve up. This allows greater reuse and means that the 4,000 pages (number chosen at random) on MSDN don't have to be each coded individually as an HTML page.
After Googling i think, the check-box Use fixed naming and single page assemblies in the publish Website form of the Visual studio, is the right choice.
Although it might slow things down...
I am developing an asp.net website that will need regular updates for source codes and HTML sides. After i have completed the necessary updates, i use the 'publish website tool' and publish the site to my local directory. Then i upload all files to Remote File. Is there any way to fix my site with just changed files. For example if i have updated just 2 files of 84 (HTML or source side), is it possible to update just this 2 files without any problem?
While Visual Studio does offer a few different compiling/build options, I think that you are probably doing everything just fine for a Visual Studio Website project. When I'm working with a website rather than a web application, I will only FTP the files that have changed. For example, if I were to change some HTML tags around within an .aspx page, then only that page would need to be uploaded to the web server. If I change the Page_Load function in the .aspx.cs page, then I will definitely need to get the updated DLL on the web server as well.
I use FileZilla for my FTP tool and there is an option to only upload the changed files. So after you use visual studio to publish your website, you can grab the entire contents of that directory and drag it over to the FTP server location and only update what has changed.
The ASP.NET website project offers an option in the Project properties -> "MSBuild Options" tabpage where you can set "Allow this precompiled site to be updatable".
Setting this option
"Specifies that the content of .aspx
pages are not compiled into an
assembly; instead, the markup is left
as-is, allowing you to change HTML and
client-side functionality after
precompiling the Web site. Selecting
this check box is equivalent to adding
the -u option to the
aspnet_compiler.exe command."
according to the docs.
I've been entirely underwhelmed by the available tools to VStudio. And am instead using Gulp.
Even in 2017 the problem remains.
With gulp you can "watch" directories for changed files, so I have it configured to watch the [bin] folder, along with assets separate from aspx/cshtml files.
That way whenever I change anything, it is instantly copied to my publish folder where I can later zip it up and deploy ONLY the changeset. The script even has a delay so I can delay the copy/upload in situations like bundles where they are generated dynamically and take some time to be fully modified.
The only thing I have to do is clean it up before I start my next milestone, so that the "publish" folder which gulp "auto deploys" to is empty and ready for the next run. Emptying a folder and running "gulp watch" in background seems like a small price to pay for such a needed feature.
I feel like this should be a good item to develop as a vstudio plugin.
Recently I have been forced to move to a windows/C#/.NET/MVC environment from linux/node/angular. Sigh. I found the following solution (next paragraph) elminates the pain caused when using visual studio to "publish" the code. The VS2017 publish process copies every file in the entire application to the web server, even if just one character is changed in one file. This can take over an hour for our moderately sized app.
So here's what I do. I first publish the solution locally (typically to bin/release/Publish/). That takes about 1 minute as opposed to 1 hour to publish to the server. Then, I compare the files between my local Publish directory and the server directory using FreeFileSync. FreeFileSync is amazing -- and free. I have access to the server directory via a windows file share. The compare takes about 15 minutes. I can then see exactly what files are different and need to be pushed. Note, the option I use compares the actual CONTENT of the files, not just the create time. I then use the FreeFileSync sync feature (mirror option) to copy the few files needed to the server. This takes maybe two minutes. So the total operation takes about 20% the time Visual Studio "Publish" takes. But best of all, the actual hit on the prod server is only the two minutes it takes to copy the diff files, rather than the 1 hour outage inflicted by Visual Studio Publish as it slogs along copying each and every file.
It depends on the type of project.
If is a project created with File > New Web Site then is ok to just copy the changed .aspx files (make sure that the corresponding code - .cs - files are also copied).
If the project was created with File > New Project > ASP.NET Web Application then you will need to copy the .aspx files and the compiled project dll (by default the dll has the same name as the project like TestProject.dll) from the bin folder inside the project.
i think you need something http://winmerge.org/
where it will compare files that are changed and upload them.
hope this helps