I've had this setup, but it didn't seem quite right.
How would you improve Content Delivery (CD) development across multiple .NET (customer) development teams?
CMS Server -> Presentation Server Environments
CMS Production -> Live and Preview websites
CMS Combined Test + Acceptance (internally called "Staging") -> Live ("Staging")
CMS Development (DEV) -> Live (Dev website) and sometimes Developer local machines (laptops)
Expectations and restrictions:
Multiple teams and multiple websites
Single DEV CMS license (typical for customers, I believe?)
Enough CD licenses for each developer
Preferably developer could program and run changes locally--was this a reasonable expectation?
Worked
We developed ASP.NET pages using the Content Delivery API against the same broker database for local machines and CD DEV. Local machines had CD dlls, their own license files, and ran/debug fine with queries and component presentation calls.
Bad
We occasionally published to both the Dev presentation server and Developer machines which doesn't seem right now, but I think it was to get schema files on our local machines. But yes, we didn't trust the Dev broker database.
Problematic:
Local machines sometimes needed Tridion-published pages but we couldn't reliably publish to local machines:
Setting multiple publication destinations for a single "Local Machine" publication target wouldn't work--we'd often take these "servers" home.
VPN blocked access to laptops offsite (used "incoming" folder at the time).
Managing publication targets for each developer and setting up CD for each new laptop was good practice (as in exercise, not necessarily as a good idea) but just a little tedious.
Would these hindsight approaches apply?
Synchronize physical files from Dev to local machines on our own?
Don't run presentation sites locally (localhost) but rather build, upload dll, and test from Dev?
We were simply missing a fourth CMS environment? As much as we liked our Sales Guy, we weren't interested in purchasing another CM license.
How could you better setup .NET CD for several developers in an organization?
Edit: #DominicCronin pointed out this is only a subset of a proper DTAP setup. I updated my terms and created a separate question to clarify DTAP with Tridion.
The answer to this one is heavily depending on the publish model you choose.
When using a dynamic model with a framework like DD4T you will suffice with just a single dev environment. There is one CMS, and one CD server in that environment and everything is published to a broker database. The CD environment could be used as an auto build system, the developers purely work locally on a localhost website (which gets the data from the dev broker database), and their changes are checked in an VCS (based on which the auto build could be done).
This solution can do with only a single CMS because there is hardly any code developed on the CMS side (templates are standardized and all work is done on the CD side).
It gets more complex if you are using a static or broker publishing model. Then I think the solution is to split Dev up in Unit-Dev and Dev indeed as indicated by Nuno and Chris.
This solution requires coding on both the CMS and CD side, so every developer has a huge benefit in having its own local CMS and CD env.
Talk to your Tridion account manager and agree a license package that suits the development model you want to have. Of course, they want to maximise their income, but the various things that get counted are all really meant to ensure that big customers pay accordingly, and smaller customers get something they can afford at a price that reflects the benefits they get. In fact, setting up a well-thought-out development street with a focus on quality is the very thing that will ensure good customer satisfaction and a long-running engagement.
OK - so the account managers still have internal rules to follow, but they also have a fair amount of autonomy in coming to a sensible deal with a customer. I'm not saying this will always work, but its way better than blindly assuming that they are going to insist on counting every server the same way.
On the technical side - sure, try to have local developer setups and a common master dev server a-la Chris's 5th. These days, your common dev environment should probably be seen as a build/integration server: the first place where the team guarantees all the tests will run.
Requirements for CM and CD development aren't very different, although you may be able to publish to multiple developer targets from one CM if there's not much CM development going on. (This is somewhat true of MVC-ish approaches, but it's no silver bullet.)
Related
We own a small company and develop asp.net websites. Here is our work procedure:
We have a server at the company with Sql Server 2008 and IIS 7.5 installed on it. All our projects including the database and website pages are on the server. We connect to the server and edit the files using FTP, so any change to a web page can be seen at once. The programmers (less than 10 programmers) connect to the server using Visual Studio 2010.
Now we want to include source control system in our work. The problem is including a SCM in our work requires changing our way of working.
Does anyone have any advise on setting up the working environment?
Thanks in advance.
You first need to decide on what type of SCM you are going to use - centralized or distributed.
One centralized SCM is TFS - this is from MS and integrates very will with VS. I believe there is an express (basic) version that is free, but the other editions are quite expensive.
An easy and free centralized SCM to start with is subversion - you can install the SVN server on your server and setup a client for each developer.
A distributed SCM does not have a server - a popular one is GIT.
Do read up on all of these before deciding. You will also have to figure out a good workflow for your team. Start with a small project so you can gain understanding and minimize the cost of mistakes.
So many ways to do this :)
One way is to use something like http://beanstalkapp.com/ to store your source code under SVN. Each developer then has a local copy of the code to work on and a good history of changes is kept when developers commit their code (at least daily), and these changes can be emailed around to the team if you want them to be. One member of the team is then tasked with uploading the latest SVN code to the testing server once it's tested and approved locally (probably at the end of each day).
I'd recommend your developers install http://www.visualsvn.com/visualsvn/ Toolbar into Visual Studio if you use SVN.
As an alternative to hosting your SVN repository with someone like Beanstalk, you could use the free http://www.visualsvn.com/server/ which cuts out the need to upload the latest code to your testing server, as it'd be stored right there and updated on each SVN commit. But this adds an overhead in terms of backups etc.
Let us know what road you go down in the end.
This question is for ASP.NET and SQL Server developers. What are your best practices with respect to setting up your development and test environment? I'm interesting in the following issues:
How many tiers do you recommend and what goes on on each tier? Just dev, test, and production or perhaps dev, test, staging, and production?
Which types of applications and/or servers should run on actual physical hardware and which can get away with a VM?
What are your strategies for loosely coupling users from web sites, web developers from their web/app/DB servers, and DB developers from their DB servers?
How do developers stay "DRY?" (no deodorant jokes, please ;)
What are the pros and cons to putting web, app, and DB servers on their own machines? Does putting servers on separate machines in order to minimize contention for a machine's resources trump any NIC and network latencies that might be introduced by putting them on different machines?
How do you configure your web apps to minimize contention for resources (e.g. virtual directories, separate application pools, etc.)
How and how often do you refresh your databases on each tier? Do you just refresh the data or both the data and objects?
Thanks.
I can't comment on all of these but here's what i've found to work best in my experience.
1) Depends on your resources but ideally i like to have 4.
Dev is hyper flexible and owned by your dev team. It can get updated whenever they feel is best or as features are completed.
QA is updated on a scheduled or delivery basis depending on your process. If you do waterfall its updated when your in the testing phase, if you do iterative agile its updated each iteration. It should mimic prod as closely as possible but you may be able to get away with some compromise (see #2)
Staging should be identical in every way to prod. It should even use real production data if possible (potentially restored from a recent backup of the true production environment.) It should be used for acceptance testing prior to any release.
& Prod
2) Dev can be on a VM usually. QA can too most of the time. Staging and prod should match. I've seen folks run prod on VMs before, it depends on your resources and the demand for your app.
3) Our devs use a backup of prod on local SQL servers for development. This keeps everyone off of a central dev SQL server. Dev web and dev sql are separate boxes (just out of necessity they manage a bunch of projects.) same with QA, Staging and Prod.
4) A lot of testing and communication. If you have one small / medium team this isnt that hard. If you have lots of teams look at something like scrum, formal code reviews, something to keep communication going between teams. Don't treat DRY issues like suggested fixes, treat them like bugs, that need to be fixed. You'll spend way more time maintaining the code than writing it up front so treat maintenance as a 1st class citizen and make sure management is on board with that.
5 & 6) Not really qualified to comment
7) Dev whenever the teams need to, QA and up on a schedule depending on deployments. QA is every iteration / sprint, Staging and Prod is every release.
You know, the next "big" and "enterprisey" thing from Microsoft.
Is it just me, or is it really hardly for humans? Main highlights are (IMO):
Absolutely cryptic syntax (-skip:objectName=filePath,absolutePath=App_Offline.* just for skipping App_Offline.html)
Manifest as an after thought
Lack of thorough documentation
Not a word about extensibility (except for several blog posts out there). Moreover, all these extensions developed in great pains have to be registered in GAC and registry
Waaay too low-level (metadata/metakey; all this IIS jazz)
No integration with MSBuild
Granted, MSDeploy and MSDeployAgent are quite powerful, but do they really need to be that complex for relatively simple tasks?
I too share your frustrations over the lack of documentation and the apparent low-level nature of this tool.
However what MS has done is finally create a free tool with which you can actually script whole server deployments, including parameterising addresses, configurations etc. This is unfortunately a very complicated thing to do - given how many bits of configuration actually go into a web server - and this is probably the best way to do it all.
What we need now is a really good GUI that can help build up these packages, and scripts etc. The GUI that is embedded within IIS is good - but again, short on explanation - so hopefully soon that'll be addressed.
On the functional side, I'm using at the moment to deploy a site from dev -> staging -> live with parameters to change bound IP addresses etc. I was deeply frustrated that it took me a few days to get it all working - however now I have it, I can remove a lot of the possibly of human error at the IT Support side - who are responsible for our deployments. I now only have the configuration of my master staging server to worry about - and can be sure that all the servers in the web farm will be kept in sync whenever I deploy.
As Sayed mentions, as well, there are MSBuild tasks in 2010 (the Website Deployment feature is now implemented using msdeploy) to work with this - which also brings the possibility of a true Continuous Integration environment to VSTeamSystem - having a team build that can actually perform a full web deployment as its last step is very exciting (and scary, granted!).
Actually there are MSBuild tasks for MSDeploy. They will be shipped with .NET 4/Visual Studio 2010.
Although a bit rough around the edges, I've come to like MSDeploy quite a bit. Using it to sync web servers in a farm is very useful as it is efficient (only copies changes) and takes care of actual IIS settings in addition to content files. It seems like MSDeploy is a building block for various scenarios and uses. Also, as previously mentioned, there is a MSBuild task for MSDeploy in .NET 4. I've taken advantage of this MSBuild task to make deployment of my web applications from TeamCity trivially easy. I've blogged here it here:
Web Deploy (MS Deploy) from TeamCity - http://www.geekytidbits.com/web-deploy-ms-deploy-from-teamcity/
I have recently started implementing a deployment pipeline and I found below links quite useful:
MSBuild commands I used for Continuous Integration:
http://www.troyhunt.com/2010/11/you-deploying-it-wrong-teamcity_24.html
WebDeploy sync commands, I used for deployment packages to production server:
http://sedodream.com/2012/08/20/WebDeployMSDeployHowToSyncAFolder.aspx
Also I used these references:
Video about MSBuild on dnrtv.com
Microsoft Press book called "Inside the Microsoft® Build Engine: Using MSBuild and Team Foundation Build" which you can buy PDF version from Oreilly
Finally, "Continuous Delivery" book, gave me good ideas about deployment pipe line, although the book is not focusing on MSDeploy, but it is really worth reading.
The statement of documentation is typical of a MSFT 1.0 product, unfortunately MSDN no longer have dedicated Developer Technology Engineers to fill the gaps --- instead, there is a blind faith that the web will provide it.
I am actually considering dusting off my writing skills and write a short ebook on it since there is likely a market for it....
Msdeploy definitely has a touch of the PowerShell to it: power over simplicity rather than worse is better.
There is no Windows alternative to it, however you can hybridize some of its powers to make automated deployments. For example:
Compile your solution with Team City and msbuild
Use msdeploy to transform your site and web.configs on the build server
Manually FTP a ZIP file of your site (it doesn't support FTP)
Alternatively, use its remote deploy capabilities. This requires port 8172 open, lots of security changes and as far as I'm aware no concessions for load balancing
Use msdeploy on the live site to sync changes
As a tool it's clearly aimed at service providers as it's an enormous Swiss army knife. You can do all kinds of things to IIS with it, which for the most part are over kill for small businesses. I've no experience of large scale IIS setups so maybe that's where it shines.
Currently our team (web devs, one designer and one copywriter) all work on separate workstations but do our changes on the same dev environment (we all mount the same shared drive), it's a marketing site and not a web application, so no builds or deployments, we just push changes to the live site once they are done, but I think it's important for us to keep versions of files, especially serverside code, even if it only makes up a tiny percentage of our content (mostly static pages).
I'd like to use version control for our work setup, but I'm not sure if SVN or GIT will play along with more than one person checking in/out from the same dev environment. I've got existing experience with SVN, CVS, GIT, Perforce and PVCS but have always worked with individual dev environments.
I'd like a solution that doesn't require us to run separate dev environments as we lack the infrastructure.
Most, if not all, version control implementations are intended to be used by more than one person. Both Subversion and GIT will happily do everything you require but you might find Subversion easier to get up and running with quickly. If you intend to host it in a Windows environment, take a look at VisualSvn Server.
The major difference between GIT and SVN is that GIT is a distributed system whereas SVN relies on a central repository. In software development, there are good arguments for using a distributed system but the needs you describe would be easily served by the much simpler SVN implementation (I think).
Another good reason for using SVN (under Windows) is that the TortoiseSvn client is one of the best examples of a user interface to any version control system. It is extremely easy to learn how to use and well documented as well as supported by the OS community.
It may also be worth investigating the various providers of hosted version control systems if you don't want the overhead of maintaining your own source control servers.
You don't need lots of server hardware. Why can't each dev/designer/etc run the site on their own computer? You do have atleast one computer each? :)
GIT or SVN in that case is just a matter of taste.
the way we used to do this at an old workplace was to give each user an account, and give the "project" and account as well. We would each check out a local working copy. the project (in your case it would be the tools that people in your business use i guess) would check out a copy too. Once the devs were all satisfied with a certain build, we would issue an update to the project working copy.
In your setup you say that you all have the same dev environment. do you mean you each have your own PC on the network, or are you sharing a PC? either way you should be able to each have your own svn accounts and local working directoires, so this would not be a problem.
Depending upon the amount of data you are talking about, an option to consider would be Dropbox ("secure backup, sync, and file sharing made easy"). It supports versioning, and lets you share folders.
It also has the benefit of providing offsite backup of, and remote web access to, your data.
With svn, you can have each submit automatically trigger an update of the live site. Still having everyone work in the same directory is somewhat awkward (as it was all the time, anyway …), but old habits die hard and assuming people insist on that, svn does export its repository via WebDAV, so it should be possible to mount it as a network filesystem on the desktop machines.
"Currently our team (web devs, one designer and one copywriter) all work on separate workstations but do our changes on the same dev environment (we all mount the same shared drive)" Adding SVN would do away with the need for the shared drive. You would each work in a local directory that is checked in to SVN. Tortoise would be a perfect client for you.
"it's a marketing site and not a web application, so no builds or deployments, we just push changes to the live site once they are done" A simple batch job (ANT script or other) could be written and given to each of you. Once the files are ready for deployment simply execute the batch and have it check out the latest files from SVN and copy to your web server.
"doesn't require us to run separate dev environments as we lack the infrastructure"
Doesn't make a lot of sense.
Presumably, each of you has a separate workstation. And your workstations are separate from your web server. Just guessing, but that's typical.
You can -- trivially -- each have a private development copy on your workstations. You can then use SVN to synchronize your various changes.
You can tag a version as "good to go".
Someone can -- when you've got everything looking right -- do an update on the web server to get the official version into production.
This doesn't require any more infrastructure than you already have in place.
With SVN you can have the devs commiting to the server, and have one special user at the server checking out the latest version. So commiting you work involves commiting to SVN, then logging in to the server and do a checkout of the latest trunk.
You can use the same pattern with git and other decentralized version control systems. The advantage with these systems is that they don't enforce the central server pattern. Dev a could for instance push to b which then pushes it to the server.
It's one of those things I see a lot but never really think of. Do you think for the purpose of web application development (specifically ASP.NET WebForms/MVC). Do you think it's advantageous to do such a thing and if so, what kind of advantages come out of it?
By virtualization I mean using products like Hyper-V to separate the server context like your SQL and Web Server, etc.
First question is, virtualization of what? Do you mean server virtualization? Do you mean running VMWare on each dev's laptop with multiple OSes? Do you mean moving everything to the cloud?
Virtualization of servers, in web app context, is not really different from that in general IT - most of the servers on the Internet, including StackOverload's, are bought to handle peak loads and spend most of the time idling away the cycles, so virtualizing them makes sense when you have more than a certain amount.
VMWare on the desktop (or other parallels on other operating systems) is superb because a) your devs can run a full instance of your server environment, including multiple virtual servers connnected in a virtual network - this is about as close to the real thing that you can get, minus hardware costs and minus devs messing with each other's servers. For clients, you can use Linux and multiple Windows installs to test various browsers, font sizes, etc. quickly - also a big win.
Moving everything to the cloud makes sense in many cases, but is probably a topic for a separate full-sized question :)
One big advantage I see is, that every developer can have his/her own sandbox to work on. If someone messes up his/her sandbox he/she can take a clean image and all is OK again. So I guess that means that there is room to experiment without losing valuable time getting back to the normal setup, you can simply do a rollback.
I'm in doubt a bit on whether you should use virtualisation for production environments. Depending on the application of course.
The only time I would use a virtual for ASP.Net development was if the app required specific setup, such as relying on installed software, wierd settings or particular shares. Every developer has their own webserver and can run their own database so if it's a "basic" webapp I don't see much value in virtuals.. it's pretty hard to break anything with a basic web app deployment :)
With a virtual server, you can test your code in a production-like environment. It is also possible to quickly revert back to the original setup. For many applications, it is useful in that time period just after you write the code, but before it goes to production.
I'm a fan of virtualizaion and use it in testing and production (VMWare and Hyper-v) but over the last year I find it less important on a dev machine. TFS provides me with all the backup/rollback ability that I need, multiple versions of .net can now exist on the same machine and VS2008 can target all those versions.
In a development environment a virtual environment is useful to put several different servers on one box, you can have an instance for your web app, one for your services, one for database, etc. That way it mimics your production environment if you are using separate servers.
One of the benefits of using virtualization in production is that your application is not tied to a specific machine. If you wanted to move your web server instance to another box, it is trivial to do so. You don't need to install or configure things on the new server and hope that everything is set up properly.
One problem I have had though in testing virtual instances is that it can run slower for some applications, specifically engineering apps that like running the CPU at 100%. So test before you leap.