As part of improvements to our build process, we are currently debating whether we should have separate project/solution files on our CI production environment from our local development environments.
The reason this has come about is because of reference problems we experienced in our previous project. On a frequent basis people would mistakenly add a reference to an assembly in the wrong location, which would mean it would work okay on their local environment, but might break on someone else's or on the build machine.
Also, the reference paths are in the csproj.user files which means these must be committed to source control, so everyone has to share these same settings.
So we are thinking about having separate projects and solutions on our CI server, so that when we do a build it uses these projects rather than local development ones.
It has obvious drawbacks such as an overhead to maintaining these separate files and the associated process that would need to be defined and followed, but it has benefits in that we would be in more control over EXACTLY what happens in the production environment.
What I haven't been able to find is anything on this subject - can't believe we are the only people to think about this - so all thoughts are welcome.
I know it's anachronistic. But the single best way I've found to handle the references issue is to have a folder mapped to a drive letter such as R: and then all projects build into or copy output into that folder also. Then all references are R:\SomeFile.dll etc. This gets you around the problem that sometimes references are added by absolute path and sometimes they are added relatively. (there's something to do with "HintPath" which I can't really remember)
The nice thing then, is that you can still use the same solution files on your build server. Which to be honest is an absolute must as you lose the certainty that what is being built on the dev machine is the same as on the build server otherwise.
In our largest project (a system comprising of many applications) we have the following structure
/3rdPartyAssemblies /App1 /App2 /App3 /.....
All external assemblies are added to 3rdPartyAssemblies/Vendor/Version/...
We have a CoreBuild.sln file which acts as an MSBuild script for all of the assemblies that are shared to ensure building in dependancy order (ie, make sure App1.Interfaces is built before App2 as App2 has a reference to App1.Interfaces).
All inter-application references target the /bin folder (we don't use bin/debug and bin/release, just bin, this way the references remain the same and we just change the release configuration depending on the build target).
Cruise Control builds the core solution for any dependencies before building any other app, and because the 3rdPartAssemblies folder is present on the server we ensure developer machines and build server have the same development layout.
Usually, you would be creating Build projects/scripts in some form or another for your Production, and so putting together another Solution file doesn't come in the picture.
It would be easier to train everyone to use project references, and create a directory under the project file structure for external assembly references. This way everyone follows the same environment.
We have changed our project structure (making use of SVN Externals) where each project is now completely self-contained. That is, any references never go outwith the project directory (for example, if Project A references ASM X, then ASM X exists within a subfolder of ProjectA)
I suspect that this should go some way towards helping solve some of our problems, but I can still see some advantages of having more control over the build projects.
#David - believe it or not this is what we actually have just now, and yet it's still causing us problems!
We're making some changes though, which are forced upon us due to moving to TeamCity and multiple build agents - so we can't have references to directories outwith the current project, as I've mentioned in my previous answer.
Look at the Externals section of this link to see what I mean - http://www.dummzeuch.de/delphi/subversion/english.html
I would strongly recommend against this.
Reference paths aren't only stored in the .user file. A hint path is stored in the project file itself. You should never have to check a .user file into source control.
Let there be one set of (okay, possibly versioned) solution/project files which all developers use, and the Release configurations of which are what you're ultimately building in production. Having separate project files is going to cause confusion down the road, when some project setting is tweaked, not carried across, and slipped into production.
You might also check this out:
http://www.objectsharp.com/cs/blogs/barry/archive/2004/10/29/988.aspx
http://bytes.com/forum/thread268546.html
Related
I am not a CQ guy. I have to use CQ5 for one of my project. I have a CAT and a production environment. I have following doubts-
I want to use author instance of my CAT only. Once I publish the content in CAT it should publish in Production also. Is it possible ?
Once I update the build of AdobeCQ in my production say new build, code changes etc- will my content be lost ?
I read somewhere about Content package in cq5. Can I separate content changes and code changes in one CQ5 environment ?
Thanks in advance.
To answer question 1...
This is not a recommended setup, but a common misconception for someone unfamiliar with AEM/CQ5. The "author" and "publish" instances should be part of the same environment. For example you should have a production author, probably behind your firewall, and production publish to serve pages to the public.
Your CAT environment should have the same thing. You want your testing environment to match as closely as possible to your production environment, including web server and dispatcher setup, to ensure quality.
Consider this. You can use one production publish instance, but it's a single point of failure. It's a general best practice to load balance across at least two. Two is sufficient for most websites. If you do this, you'd want to mimic the architecture in CAT.
To answer question 2...
If your code is written, built, and deployed correctly, it should not delete your content. Just make sure you are never deploying anything to /content (to avoid deleting content) and to /libs and most of /etc to avoid overriding platform functionality. AEM/CQ5 is a very open product, so you can do very bad things. But, if you know what not to do you are safe.
Code deployments should typically be done as part of a CRX Content Package, which brings me to...
To answer question 3...
The way we build and deploy code is to have Maven compile the Java, package everything up in a CRX Package, then deploy to the instance using the Package Manager REST API. Adobe provides a Maven Archetype that will facilitate this.
A CRX Package is a file system representation of your content repository, wrapped in what is effectively an annotated Zip file. Your compiled Java code is included in that file system representation, in a folder (to become node) named "config". That compiled Java is an OSGi bundle, which is an annotated JAR. When CRX Package Manager deploys all those nodes to the system, OSGi accepts the bundle, assuming it's valid. This is why you can do "hot" deployments of live, production AEM/CQ5 instances, with very little risk.
So...
This is a very high level answer to some very big topics. I encourage you to do a lot more research before you set this up. There are many good blog posts and documentation pages out there to help you get this set up according to best practice. Good luck!
After pre-compiled a ASP.NET web site, I got many files with the names like
App_Web_accountbalance.aspx.dfa151d5.dll
Do you know the rule for the random chars (in bold) above?
Can we fix the random chars?
The reason to fix it is that if we modify AccountBalance.aspx file later and re-compile the web site, can we just replace App_Web_accountbalance.aspx.dfa151d5.dll.
Thank you.
The characters are not random, but more in line with hashing. The purpose is to make the file name unique in the bin folder. Although not advisable, you can replace just certain files to update your website. If you modify AccountBalance.aspx and recompile, you need to replace App_Web_accountbalance.aspx.dfa151d5.dll, accountbalance.aspx.dfa151d5.compiled, and other assemblies and files that your aspx file depends.
I derived the answer from my experience. I was not looking for the file naming rule, but a way to deploy just the assembly of a changed page, same reason as the original post.
The setup:
A web application, deployed non-updateable (updatable=”false” in PrecompileApp.config), pre-compiled assemblies with no fixed names
What I did:
Make the change to the page (say, a.aspx) in development (Visual Studio 2010)
Publish the site with fixed naming to local drive (Build > Publish Web Site, check the box: Use fixed naming and single page assemblies)
Go to the bin folder of the local publish site and look for a.aspx.xxx.compiled
Open the file with Notepad and note any dependency (say, b.aspx, c.master)
Copy all the affected assemblies and the .compiled files to the bin folder in production server. In this example, they are:
a.aspx.xxx.compiled
b.aspx.xxx.compiled
c.master.xxx.compiled
App_Web_a.aspx.xxx.dll
App_Web_b.aspx.xxx.dll
App_Web_c.master.xxx.dll
If you want to know my story, the change was due to a change in a factor in a calculation. The customer knew of the change much earlier, but did not let us know until it became urgent. A proper deployment would involve other parties and much coordination, and would be too late. Plus, I only had the source code of two versions back, and requesting the latest would take time. So, a hot fix on just that calculation change was required as a temporary measure.
1) you can generate single assembly per web application if you want. So when you make a change in web application, you only need to deploy just one dll.
for this, you can check option "Use fixed naming and single page assemblies"
2) Reference from MSDN Article: "The assembly names are generated automatically by the compiler and it is therefore not obvious which assemblies map to which source files. The compiler also creates new names each time it runs, so that the names of assemblies might not be the same after each compilation. In addition, if source files have changed, the compiler might batch up source files differently, meaning that the resulting assemblies do not necessarily represent the same source files. If you are maintaining a deployed Web site and want to update only the assemblies for recent changes, the output from batch compilation can make that job more complicated.
To help you in this situation, aspnet_compiler.exe supports an option specifically designed for packaging and release management: the -fixednames option. This option enables you to create compiler output that has two benefits. The first is that the assemblies produced by the compiler have the same names each time you compile. The second is that the assemblies are based on the same input files each time."
http://msdn.microsoft.com/en-us/library/aa479044.aspx
I'm using Perforce, if that changes the tune of the answers at all.
I'd like to implement a build process that, when a solution is built in a "release" mode, tags the entire source tree with a label and pushes the output of the build (DLLs, webpages) to a /build/release directory in source control. This directory should always contain the latest complete build, nothing less and nothing more, so I can yank that directory to production servers in its entirety and it's ready to go.
Now say I had a DLL in a previous release that the new build is not supposed to include. Does this mean the best practice for updating that /build/release folder is to check the entire thing out, delete everything in it, add the new build files, and sync it? Sounds like an obvious answer, but I want to make sure I'm not missing some other voodoo that might be a better way to do it.
I think you are missing the simple voodoo:) You should consider just using a plain old file system for your build drops. Source control is designed to manage change, versioning, and collaboration and there really is no need for any of this related to builds. The whole point to an build system is to be able to reproduce the source code and create the application at a moments notice so I would focus on being able to do that more than relying on the permanent storage of the output files. Be sure to back up the build drop folder structure just as you would the source control database. Use a folder naming scheme that includes the build number in the filename. I would store all of the builds (back at least several) because there are times when QA wants to restore an old build to test in order to compare features or resurrect a bug. Using this system every build gets a new folder so you don';t have to worry about deleting out old files.
I'd say "Yes" - you should to start with a blank folder structure for your builds (regardless of source control system).
Is it possible to exclude a folder in a web project from being published? We've got some documentation and scripts that included in a particular project folder, and are added to the project, but when I do a VS publish, I don't want them to go up to the production server.
I know they shouldn't be in the project, but I thought I'd find a workaround before I try to convince the owner to modify the way he's doing things.
Old question, but I found if I mark the folder as hidden in Windows Explorer, it doesn't show/publish in your solution.
This is good for example to stop original photoshop images being included in uploads which aren't used and are big. Anything more complex though you'll probably want to write your own publish tool.
This doesn't answer your question, exactly, but my feeling is that unless you are a single developer publishing to a server, you would be better off doing builds on a dedicated workstation or server using MSBuild (or some other building and deploying solution) directly (and thereby would be able to very granularly control what goes up to production). MSBuild can not only build, but using some extensions (including open source types), it can also deploy. Microsoft has a product called MSDeploy in beta, and that might be an even better choice, but having no experience with it, I cannot say for certain.
In our situation, we have a virtual workstation as a build box, and all we have to do is double click on the batch file that starts up an MSBuild project. It labels all code using VSS, gets latest version, builds the solution, and then deploys it to both servers. We deploy exactly what we want to deploy and nothing more. We're quite happy with it.
The only downside, if it could be considered a downside, is that at least one of us had to learn how to use MSBuild. VS itself uses MSBuild.
For the files you don't want to go, loop at the properties and set the 'Copy to Output Directory' to 'Do not copy'
This option is not available for directories, however.
Can you not exclude them from the project through visual studio to stop them being published. They will the still exist in the filesystem
The only way that you can do this to my knowledge would be to exclude it from the project, do the publish, then re-include it in the project. That can be an issue.
There are probably much better ways to solve this problem but when we publish a build for our dev servers, we'll run a batch file when the build is complete to remove the un-needed folders and web.configs (so we don't override the ones that are already deployed).
According to http://www.mahingupta.com/mahingupta/blog/post/2009/12/04/AspNet-website-Exclude-folder-from-compilation.aspx you can just give the folder the "hidden" attribute in windows explorer and it won't publish. I tested this and it works for me.
Seems like a straightforward solution for quick and dirty purposes, but I don't think it will carry through our version control (mercurial).
Select all the files that should not be published.
Go to Properties
Set
Build Action -> None
Have to repeat the process for each sub-directory.
Here's our problem, we are a Flex shop that uses .NET for the server side logic. We use subversion for our source control and subeclipse in Flex Builder but are still quite new to using source control let alone subversion. Branching and merging seems to work very well on the .NET side but we are running into issues on the Flex side because of the final swf being built on our local machine.
The question is, what does a usual workflow look like for working with Flex and SVN? Particularly, how do you branch and where do you build?
Personally, I keep the Flash/Flex source code in a separate SVN repository that is away from what is deployed to any sort of web server. That way I can create branches and tags specifically for my Flash/Flex application. I also tend to publish any SWF's directly into my local copy of the deployment repository. It does not make sense to me to keep a published SWF under version control unless its part of the what is deployed to the server. I don't like to keep committing an SWF into my Flash source code repository because it takes up unnecessary space and all the source code should represent the latest version, not the resulting SWF.
You'd probably want to branch your project alongside your .Net project so your flex releases are consistent with your server logic.
We use a directory structure like this
+server-side-app
--trunk
--tags
--branches
+flex-client-app
--trunk
--tags
--branches
I would recommend something like that for yourself.
I agree with Matt W. At AKQA we have svn locations four our source and assets. We set up an svn ignore for the bin folders of a project. That way we aren't checking any swfs which means when we update we don't get someone elses swfs or output files.
A good bet is to look into continuous integration with something like cruise control. We build our output on the server which generates all of the files into one location on the server. There are loads of other benefits of continous integration and it's well worth having