I'm working on transitioning my current project of about 20 developers to a modern development and build environment. We currently use an RCS based source control system and an associated issue tracking system, both with Motif UIs. There is no formal production build process, its just whatever works.
I'm interested in:
Development Tools
Version Control
Issue Tracking
Dependency Management
Configuration Management
Automated Building
Automated Testing
Continuous Integration
Artifact Management
Release Management
Deployment Management
Requirements Tracing
What else?
I'm interested in not just which tools you use, but how well the integrate with each other, how easy they are to setup and use, and how both developers and management likes them. Our project is a combination of Java, C++, and VHDL, but I'd still like to hear from people with other languages. I'm currently going down the path of eclipse, subversion, trac, maven, hudson, and nexus.
Also, is there a better term than "Build Lifecycle" that encompasses not just building, but the flow of code from when the developer creates it to when its built, tested, and in a production system? "Build Lifecycle" seems limited, but "Project Lifecycle" is already taken.
I hate Maven less than I hate Ant, and for Java, you need to choose one of those evils. If you're just starting out, choose Maven, especially since you've already recognized that your "build lifecycle" encompasses 12 different and complex disciplines! You're going to have to choose conventions for all of them. Save yourself the trouble and go with the conventions Maven has already established.
For continuous integration and general build automation, I like Hudson.
During the last two years we gradually switched from a "every-project-has-its-own-toolset" strategy to a Trac+SVN+SCons solution and are quite happy with that.
Switching to SCons was a bit of work but really paid off. We have a heterogeneous environment, mostly C/C++ for different embedded platforms, kernel modules, some desktop applications and various Python modules as glue code. SCons really shines when you want to add support for your own compilers and niche-tools and need to adapt the build system to your requirements. Formerly, we had to use a different GUI for almost every embedded platform - now that SCons directly invokes the compilers the work-cycle has slightly improved.
Our developers either used Emacs or Vim and no one wanted to switch to anything else, so we (fortunately) sticked with that. I'm not very familiar with deployment so I can't talk about that.
If you work with .NET, it's hard to beat Team Foundation Server for its integration with Visual Studio. It contains the development tools, version control, issue tracking, configuration management, automated testing, unit testing, automated building, artifact management and everything else you've described.
Of course, TFS is expensive, oftentimes non-intuitive and is missing some features compared to other tools I've used. If you have a MSDN license you can use TFS for Workgroups (up to 5 users IIRC) for free, though.
Development Tools JetBrains IntelliJ IDEA
Version Control Subversion
Issue Tracking Atlassian Jira
Dependency Management Maven
Configuration Management TeamCity
Automated Building TeamCity
Automated Testing JUnit(?)
Continuous Integration TeamCity
Artifact Management Maven
Release Management Homo Sapien
Deployment Management Maven/Homo Sapien
Requirements Tracing Wishful thinking
One-Off Automation Bash
Developer-to-Developer Documentation MediaWiki
We're an MS shop using VS2008. We use Subversion with Tortoise for SCC and versioning, and our repository is hosted online so our distibuted team can use it. For build we're using Hudson and CI, much better than Nant or MSBuild. Issue tracking is Bugzilla. Automated testing is NUnit
Tools to avoid include Team Foundation Server and Sharepoint, too clunky for real world usage.
BTW Does anyone know a good Scrum tool, which can produce burn down charts, ideally linking into Basecamp?
We also use a number of tools, but we're moving more and more to Zed Builds & Bugs. Our primary dev environment is Eclipse + Java, but we also do Visual Studio (all of 'em), and at least 5 different unix platform builds.
Here's the full list:
Development Tools - Eclipse
Version Control - Subversion
Issue Tracking - Zed Builds & Bugs
Dependency Management - Ant
Configuration Management - Zed Builds & Bugs
Automated Building - Zed Builds & Bugs
Automated Testing - JUnit
Continuous Integration - Zed Builds & Bugs
Artifact Management - Zed Builds & Bugs
Release Management - Zed Builds & Bugs
Deployment Management - Zed Builds & Bugs
Requirements Tracing - Zed Builds & Bugs
One-Off Automation - Zed Builds & Bugs
Developer-to-Developer - Zed Builds & Bugs
I use svn and tac on some oof my projects and svn and fogbugz on others. They integrate very well.
I am still using command line scripts for builds as they do everything I need - including grepping for errors and emailing results, but the days of that setup are numbered. I am looking into cross-platform build tools.
I use Inno for win32 releases. No shipping products yet for other platform - not sure how we'll deploy those.
We don't address a lot of the other items you mention other than on some ancillary documentation and in the code and in issue tracking.
Team Foundation Server and Visual Studio.
I remember when my ide was Sun's visual C debugger, and source control was copying all source files to a new named directory and putting it on a server that was supposed to be backed up.
Only it wasn't
Related
Is it possible to do Continuous integration.
We are using Power shell script to deploy BTDF packages. I found many resources to do continuous integration for BizTalk using TFS but is it possible to do continuous integration. Is there any resources available for the same.
Well, consider "continuous integration" is a pattern, not a thing. So sure, you absolutely can do CI with BizTalk apps using a SVN. Really, the code repository really isn't a factor.
So, if you are automatically, or at least very easily, doing a build-> deploy cycle with some tests, you're doing CI. The specific underlying products doesn't really matter.
However, yes, CI with BizTalk and TFS is easier because of the tooling.
Yes, it is perfectly feasible to use SVN for CI with BizTalk - I implemented such a solution back in 2008/9 using JetBrains TeamCity connected to SVN, utilising MSBuild scripts to perform the build, package and deployment to a test BizTalk environment. We then packaged the MSBuild scripts to perform the actual deploy to the various production environments.
If you're stuck with using SVN then I would seriously take a look at TeamCity (https://www.jetbrains.com/teamcity/) and use a Powershell Build Runner to build / deploy etc. your project - for a high-level overview on using PS in TeamCity, take a look at http://www.jokecamp.com/blog/tutorial-how-to-use-teamcity-powershell-runner-to-automatically-deploy-website/.
As for SVN, you seriously need to take a look at using a more modern, feature-rich and vulnerability-free VCS Take a look at using something a little more modern such as Visual Studio Online (i.e TFS in the cloud - https://www.visualstudio.com/en-us/products/visual-studio-team-services-vs.aspx) or Git (hosted through the myriad hosting providers, e.g. a private repo on https://github.com/)
Best of luck!
PS. I agree with #Johns-305 comments fully!
You know, the next "big" and "enterprisey" thing from Microsoft.
Is it just me, or is it really hardly for humans? Main highlights are (IMO):
Absolutely cryptic syntax (-skip:objectName=filePath,absolutePath=App_Offline.* just for skipping App_Offline.html)
Manifest as an after thought
Lack of thorough documentation
Not a word about extensibility (except for several blog posts out there). Moreover, all these extensions developed in great pains have to be registered in GAC and registry
Waaay too low-level (metadata/metakey; all this IIS jazz)
No integration with MSBuild
Granted, MSDeploy and MSDeployAgent are quite powerful, but do they really need to be that complex for relatively simple tasks?
I too share your frustrations over the lack of documentation and the apparent low-level nature of this tool.
However what MS has done is finally create a free tool with which you can actually script whole server deployments, including parameterising addresses, configurations etc. This is unfortunately a very complicated thing to do - given how many bits of configuration actually go into a web server - and this is probably the best way to do it all.
What we need now is a really good GUI that can help build up these packages, and scripts etc. The GUI that is embedded within IIS is good - but again, short on explanation - so hopefully soon that'll be addressed.
On the functional side, I'm using at the moment to deploy a site from dev -> staging -> live with parameters to change bound IP addresses etc. I was deeply frustrated that it took me a few days to get it all working - however now I have it, I can remove a lot of the possibly of human error at the IT Support side - who are responsible for our deployments. I now only have the configuration of my master staging server to worry about - and can be sure that all the servers in the web farm will be kept in sync whenever I deploy.
As Sayed mentions, as well, there are MSBuild tasks in 2010 (the Website Deployment feature is now implemented using msdeploy) to work with this - which also brings the possibility of a true Continuous Integration environment to VSTeamSystem - having a team build that can actually perform a full web deployment as its last step is very exciting (and scary, granted!).
Actually there are MSBuild tasks for MSDeploy. They will be shipped with .NET 4/Visual Studio 2010.
Although a bit rough around the edges, I've come to like MSDeploy quite a bit. Using it to sync web servers in a farm is very useful as it is efficient (only copies changes) and takes care of actual IIS settings in addition to content files. It seems like MSDeploy is a building block for various scenarios and uses. Also, as previously mentioned, there is a MSBuild task for MSDeploy in .NET 4. I've taken advantage of this MSBuild task to make deployment of my web applications from TeamCity trivially easy. I've blogged here it here:
Web Deploy (MS Deploy) from TeamCity - http://www.geekytidbits.com/web-deploy-ms-deploy-from-teamcity/
I have recently started implementing a deployment pipeline and I found below links quite useful:
MSBuild commands I used for Continuous Integration:
http://www.troyhunt.com/2010/11/you-deploying-it-wrong-teamcity_24.html
WebDeploy sync commands, I used for deployment packages to production server:
http://sedodream.com/2012/08/20/WebDeployMSDeployHowToSyncAFolder.aspx
Also I used these references:
Video about MSBuild on dnrtv.com
Microsoft Press book called "Inside the Microsoft® Build Engine: Using MSBuild and Team Foundation Build" which you can buy PDF version from Oreilly
Finally, "Continuous Delivery" book, gave me good ideas about deployment pipe line, although the book is not focusing on MSDeploy, but it is really worth reading.
The statement of documentation is typical of a MSFT 1.0 product, unfortunately MSDN no longer have dedicated Developer Technology Engineers to fill the gaps --- instead, there is a blind faith that the web will provide it.
I am actually considering dusting off my writing skills and write a short ebook on it since there is likely a market for it....
Msdeploy definitely has a touch of the PowerShell to it: power over simplicity rather than worse is better.
There is no Windows alternative to it, however you can hybridize some of its powers to make automated deployments. For example:
Compile your solution with Team City and msbuild
Use msdeploy to transform your site and web.configs on the build server
Manually FTP a ZIP file of your site (it doesn't support FTP)
Alternatively, use its remote deploy capabilities. This requires port 8172 open, lots of security changes and as far as I'm aware no concessions for load balancing
Use msdeploy on the live site to sync changes
As a tool it's clearly aimed at service providers as it's an enormous Swiss army knife. You can do all kinds of things to IIS with it, which for the most part are over kill for small businesses. I've no experience of large scale IIS setups so maybe that's where it shines.
At my company, we develop our ASP.NET applications as websites and often just work off of our network drive, which points directly to the files on our development web server. Our code is compiled at time of HTML request, so we don't build our web applications. I've read that automated builds are a best-practice, and aim to set that up as well at some point. Right now we're using VSS, which is awful, and I'd like to switch us to subversion.
I've read about NAnt for builds and deployment, and also just heard about CruiseControl.NET. Can I use these tools simply to push code from our SVN Repository to our development web server when a developer commits changes to it from their working copy?
You certainly can. I have gone through a very similar migration. We always compiled our web apps, but we migrated from VSS to SVN and then setup cruisecontrol and nant to automate our builds and deployments. We used to just drag and drop with windows explorer which was down right painful.
As it happens I have been blogging on this process. My last post specifically covers using cruisecontrol.net and NANT: http://www.mattwrock.com/post/2009/10/22/The-Perfect-Build-Part-3-Continuous-Integration-with-CruiseControlnet-and-NANT-for-Visual-Studio-Projects.aspx
I am CM/Developer at my company. We use Nant, CCNET and Subversion, for continous integration and automated deployments to the DEV servers. Works perfectly.
Things to note:
1. If your getting Nant, get Nant contrib as well
2. If your building and deploying installers, it will be easier to use devenv.exe to build the installers.
3. You can check out PSTools to install stuff on remote servers.
4. I would set up two different build categories in CCNEt, 1. for Continous and 2. For force builds....this should be your publish.
The set up can get pretty complex, I have used it with VSS as well, email me if you have any questions or need scripting help.
Yep.
At one company we built a nAnt script that did this. Very simple and effective, but extremely cryptic to change or update.
At another we used Cruise Control which worked great but again was a little cryptic (I think it uses nAnt on the backend), but was very nice to look at and see the steps and problems visually.
Honestly though, the latest Team Server from MS is very good at managing code and very nice at producing builds as well. By far the easiest and effective way I've ever used to deploy .net code.
I just started at a new company that has 99% of their code written in classic ASP (most of it poorly written) and part of the reason they hired me was because I had worked with both ASP and ASP.NET in the past. The ASP.NET experience was VB.NET but I've worked with C# in college but I prefer it just because I've worked a lot with PHP in the past and when I'm not focusing I just start typing C syntax and have had many occasions here in ASP where I end lines in a semi colon just out of habit and have to go back and delete it.
But I digress, basically I just don't have the knowledge I need to make the best decisions on things have made notes of some key processes that I'd like advice on:
Version Control - truth be told I've never really used it. On my own I just never had that whoops moment to push me to use it and shops I've worked in have always just thrown caution to the wind. Would like to know what you all think I should use as far as the server side and
Local Dev Environment - Probably something I'll just quickly Google but I want to setup a local dev environment so I can test stuff without having to FTP it somewhere first.
Pushing Changes Live - I've never really understood the step between checking something into version control and seeing it live, is there some sort of automated system that can go "hey I see you checked in, let me see what's different between this and the live version and push the affected files" – possibly just a lack of vc understanding all together: (
IDE - Downloading VS2010 Beta 10 now, hoping it's stable
MVC.NET - Easy to pickup? I always hated the whole concept of Web Forms, seemed like it didn't ultimately fit the Internet the rest of the world is developing.
Anything Else - Like I said I'm relatively new to this stack so I'd love any advice I can get early on to avoid any "shit, I wish I knew that 3 months ago" moments.
Version Control:
Definately have a look at SubVersion, its free, we use a paid-for hosted service called Assembla (www.assembla.com) but you can install the server on your network. Themost popular client is called TortoiseSVN (http://tortoisesvn.tigris.org/) its also free.
You can also have a look at VisualSVN (http://www.visualsvn.com/) which integrated into the VS IDE, their site also has a handy SVN server installation.
Local Dev Environment
Local IIS or Built-in VS web-server in VS are both fine, it really down to preference I think. Opening a website (File-system based) rather than IIS based seems to work quite well for us.
Pushing Changes Live
ASP is very page by page based development and doesnt require compiling, copying files up to a test or production site is quite easy. Depending on how you compile/publish your asp.net site, you will need to consider how you compile library code and publish your web application/website.
I would recommend starting off by using SVN to commit changes and to update test/production sites.
IDE:
I would recommend updating to the latest Beta version of VS2010 and use that.
Anything else:
Consider the rollout, are you replacing everything before going live?
Consider adopting an existing framework, use an ORM, or Business Objects framework, possibly even use Code-Generation. Have a look at CodeSmith, it has various ORM/Business Object Frameworks associated with it. (these will use various teqniques and technologies in the .net framework, which you will need to learn)
Look at the basic structure of the site:
Consider asp.net WebForms (Applications) or MVC (Websites) [yes, this is a very basic analysis of the differences]
Site Membership and Permissions: Have a look at the MembershipProvider, RoleProvider and build your own.
File System: Consider how and where you will upload files.
Separate UI into re-usable UserControls.
HTH, Good Luck you have a lot to do...
Wow. lots of questions here:
Version Control - Look into Subversion and Git. They represent two kinds of version control, you might like one more than the other. Git is free for open source projects.
Local Dev Environment - I would recommend using the built-in web server in Visual Studio. You can right-click a page in your solution explorer and say "view in browser". Then it will just compile it right there and open up an instance of the local web server.
Pushing Changes Live - I am not gonna offer advice here. Someone smarter than me will surely do so.
IDE - Yeah, just stick with the current version of Visual Studio. If you don't have any active (for pay) projects, I think the VS2010 is a good idea because it supports the newest upcoming features of asp.net and by the time you learn them, it will probably be released to the public anyways.
Anything Else - It's worthwhile (from getting work perspective) to know WebForms in and out. But I would highly suggest learning ASp.Net MVC (if, for no other reason than, "it's more fun").
After reading Mark Redman's answer, it triggered something for me. I would recommend (as he does) that you investigate some sort of ORM. I use SubSonic and really love it. Bu t there are lots of options out there.
Those are my 2 cents.
I'm going to focus on versioning and project tracking here. Development environments for ASP.Net are well documented.
Version Control - I use SVN (version control) with Tortoise SVN (windows client integration) and Trac (project tracking). SVN and Trac both run on Apache (a web server).
Pushing Changes Live - Commit your changes to SVN from your dev machine, do a SVN update on your test site to pull down the latest changes from the repository, run tests, then do a SVN update on your production site.
Anything Else - In this scheme Apache, Subversion, and Trac will be running on a server that is NOT your development machine and hopefully (but not necessarily) also not the same server that is running your production IIS.
Check out:
Apache HTTP server: http://httpd.apache.org/
Subversion: http://subversion.apache.org/
Tortoise SVN: http://tortoisesvn.tigris.org/
Trac: http://trac.edgewall.org/
One thing to take account of is basically going from Classical ASP to ASP.NET is about as similar as going from COBOL to ASP.NET. Everything will basically need to be a complete rewrite (especially as you said there were poor development practices in the original codebase)
Version Control: Definitely use it, there's also Team Foundation Server which if you can convince the company in the value of purchasing you a MSDN subscription TFS is included at no additional cost for 5 or less users.
Pushing changes live: I currently use Web Deployment projects (not sure if they're compatible with VS2010), TFS is also a build server I just haven't had time to setup the responsibility for that. If you take advantage of TFS fully I'd also recommend looking into a continuous integration (CI) tool something along the lines of Jetbrains's TeamCity or CruiseControl etc.
Local Dev Enviroment: Get Jetbrains's Resharper! This is the number one most important tool to developing software in .NET second only to Visual Studio itself!
Anything else: Learn generics and lambdas/expression trees both are integral to proper software development in .NET and both are moderately to extremely complex topics (IMO).
Version control: I see many recomendations of SVN here but... it's just outdated. Use GIT or Mercurial. They merge algorithms are more robust. The are faster and safere because they are distributed. They are more flexible - you can construct a workflow model that suits your company well while there is basically one way of using SVN.
Pushing changes live: After you have version control, the next thing to do is making a Continous Integration server thah repeatedly checks out repository and perform automated build. You can get immediate information who and where made an error. You can make this build do anytking you want: perform static code analysis, unit tests, deployment. You can make several builds - one, automatic, that builds a project and deploys it to a testing environment. And another one, triggered manually, that will deploy to live. I worked with Hudson, Jenkins, CruiseControl.NET, TeamCity and I've found TeamCity to be the most user friendly tool of that four.
Local dev env: Althoug IIS Express (that Visual Studio built-in) is OK in many cases, you have to know where it differs from the 'real' one. I would just recommend using the full IIS 7.
IDE: Visual Studio + Resharper + Notepad++ + LINQPad.
Anything else: Have some bug tracking/planning software. Even as simple as http://www.trello.com, but use it. My favourite is http://www.pivotaltracker.com
Introduce code review into the workflow. http://www.reviewboard.org might help.
I'm starting a new project using Flex3 for a reasonably intricate UI getting its data from a collection of RESTful web services. I wondering what experiences others have had in structuring a moderately complex Flex project. The issues I'm thinking about include:
I want to use our FlexBuilder licenses for visual editing of layouts and debugging code on Eclipse
automated build script so that we can use a CI server
testing strategies, particularly automatable unit tests
cross-platform Linux and Windows development environments (yes, I know the Linux version of FlexBuilder doesn't do visual editing, but everything else should ideally run cross-platform)
dependency management for at least the Java code (are there library dependency managers for Flex?)
I've just burned a day trying and failing to get a maven2/flex/jetty build setup working smoothly (based on Jeff Maury's tutorial). I'm not dead-against using Maven as we go forwards, but I'm sure there must be a less painful route. The main issue I had (other than maven documentation being patchy) is that maven's "single artifact per build" rule means that you have to jump through a lot of hoops with a complex build, and I'm not enough of a maven initiate not to trip over the hoops and fall on my face!
Would Ant and Ivy be a better option than maven2? What other choices are there?
I don't know Flex development but I know CI so I'll comment on that aspect.
I work on CruiseControl, so that's what I use (when not being paid to setup something else). And there are blog entries that describe setting up Flex projects under CC for both Windows and Linux. CC has been around for a long time so there is a large community of users and lots of 3rd party tools that work with it.
But other people are likely to recommend Hudson and it would be hard to argue against them. Hudson is very easy to get up and running and has a nice web based UI. This project is very active, has lots of fans and lots of buzz and has matured very rapidly. I don't have anything bad to say about it. This blog entry describes running acceptance tests under Hudson.
I don't know enough about Maven to offer advice there but I'm working with Ivy at a client right now and I really like it. We've got a combination of Java and C++ code and we're able to get Ivy to do what we need.
Well we are using maven. With some help of a lot of different web sties I have an enterprise application being built using maven.
This article via adobe has really help a lot.
http://www.adobe.com/devnet/flex/articles/fullstack_pt1.html
http://www.adobe.com/devnet/flex/articles/fullstack_pt2.html
http://www.adobe.com/devnet/flex/articles/fullstack_pt3.html
You really need a common configuration project that will be shared between both flex and java, and possibly your c++ code. This article explains how to set that all up.
Down side, if you are not knowledgeable with maven there is a slight learning curve.