How can I find out which tickets were closed between one build and the previous stable build? I'm trying to design a new build process, so I'm not set on particular tools yet. Which ones would let me see this sort of info in a dashboard, if any? Should I try to do this from a bug tracker, or from a build pipeline such as Jenkins or Bamboo, or somewhere else?
A possible set-up is to:
include the bug-tracker issue ID in your commit messages in your SCM ("[MYPROJECT-12923] add this new option in that nice feature")
launch your build with Jenkins that retrieves the source code from your SCM. Jenkins will show you a "Recent Changes" label linking to a page where you will find the commits that took place between last build and the current one. The commit messages will include the list of issues ID included in the build.
Note: that maybe does not answer your question perfectly because those commits could be intermediate one. Depends also on how granular are the commits.
In our DEV team, all the commits have the JIRA number in them (This is enforced by a plugin called TicketIt in Stash . There are various other plugins available for different repositories) . When we run a build , all commits that are part of the build are aggregrated by teamcity and displayed on a tab called issues . This solution that I am proposing works in teamcity and bamboo . I am sure thee would be some plugin with Jenkins for the same.
A hackier way would be to get the start time of the last build (x) and the current build(y) and get all JIRA tickets that were closed during this time via JIRA API. This might not be a foolproof method if your JIRA's are not always closed before a build
Related
Currently I am trying to improve automation in my test environment which uses Apache Karaf 2.4 and Jenkins.
My goal is, if Jenkins successfully builds the project (and updates a Maven repository), to deploy the new version in Karaf. Karaf and Jenkins run on different machines.
My current solution is to use a feature-descriptor, but I am facing a lot of trouble with it: I found no easy way to update feature-bundles in Karaf 2.4. As far as I know, there is no command that is able to update all bundles of an existing feature in one command.
My current approach is to grep the output of the list command after a special pattern and find out all BIDs and then run update for all IDs. This approach tends to create bugs (it may include bundles that are not part of the feature when they match the same naming-pattern). And I was wondering if there is a better way to automatically update all my feature bundles in one?
Also, there is another problem: When a new feature gets added or removed from the feature-file, I found no elegant way to update it. My current approach was to first uninstall all associated bundles (with grep again...), then remove the repository from Karaf and then replace the new version of the feature-file in the deployment folder. As this is very circumstantial, I was wondering, if there is a better way to do this in Karaf 2.4?
I think deployment got better in Karaf 3, but I am unable to use it because of this bug (link). As I am in a testing environment and my software tends to be extremely unstable, I often have to kill the Karaf process and restart it again. And this is a big problem in Karaf 3. Hopefully it will be fixed in Karaf 4 but I do not want to wait until it is released.
Do you have any suggestions, how my deployment-process could be improved? Maybe there are better ways than my solution (I really hope so!), but I haven't seen them yet.
This is a nice request, cause I've been working on something similar just recently. Basically I've got the same setup you got.
So you just need to install the latest Jolokia osgi servlet on top of karaf to make all JMX commands accessible via REST.
For a showcase I created a maven plugin (I'm going to publish those sources in a couple of weeks - might even get into the karaf maven plugin) that installs the current bundle via a rest request on top of Karaf, it's currently also able to check for existing bundles.
Basically you need to issue the following rest POST:
{
"type":"EXEC",
"mbean":"org.apache.karaf:type=bundle,name=root",
"operation":"install(java.lang.String,boolean)",
"arguments":["mvn:${project.groupId}/${project.artifactId}/${project.version}", true]
}
You'll need to use the latest Jolokia snapshot version to get around the role based authentication which is used in the latest Karaf versions.
For this you'll also need to create a configuration file in etc, called org.jolokia.osgi.cfg, wich contains the following entries:
org.jolokia.user=karaf
org.jolokia.realm=karaf
org.jolokia.authMode=jaas
For more details also check the issue for it.
I am working on a ASP.net MVC4 project where a same project needs to be deployed to many clients on daily basis, each client will have its own domain / sub domain and a separate app pool and db (MSSSQL).
Doing each deployment manually could take at least 1-2 hours if everything goes well. Is there anyway using which I can do this in some automated way?
Moreover, we also need to update all of the apps when a new version is released.. may be one by one or all of them at same time. However, doing this manually could take weeks and once we have more clients then it will not possible doing this update manually.
The update involves, suspending app for some time, taking a full backup of files and db, update application code/ files in app folder, upgrade db with a script and then start app, doing some diagnosis script to check if update was successful or not, if not we need to check what went wrong?
How can we automate this updates? Any idea would be great on how to approach this issue.
As a developer for BuildMaster, I can say that this scenario, known as the "Core Version" pattern, is a common one. If you're OK with a paid solution, you can setup your deployment plans within the tool that do exactly what you described.
As a more concrete example, we experience this exact situation in a slightly different way. BuildMaster has a set of 60+ extensions that rely on a specific SDK version. In our recent 4.0 release, we had to re-deploy every extension because of breaking API changes within the SDK. This is essentially equivalent to having a bunch of customers and deploying to them all at once. We have set up our deployment plans such that any time we create a new release of the SDK application, we have the option to set a variable that says to build every extension that relies on the SDK:
In BuildMaster, the idea is to promote a build (i.e. an immutable object that travels through various environments like Dev, Test, Staging, Prod) to its final environment (where it becomes the deployed build for the release). In your case, this would be pushing your MVC application to its final environment, and that would then trigger the deployments of all dependent applications (i.e. your customers' instances of your application). For our SDK, the plan looks like this:
For your scenario, you would only need the single action, "Promote Build". As I mentioned before, any dependents would then be promoted to their final environments, so all your customer deployments would kick off once that action is run during deployment. As an example, our Azure extension's deployment plan for its final environment looks like this (internal URLs redacted):
You may have noticed that these plans are marked "Shared", which means every extension we have has the exact same deployment plan, but utilizes different variables to handle the minor differences like names, paths, etc.
Since this is such an enormous topic I could go on for ages, but I think that should be sufficient for your use-case if you wanted to try it out.
There are others but you could setup Team Server Foundation to deploy automated builds.
http://msdn.microsoft.com/en-us/library/ff650529.aspx
I find the easiest way to do this from an MVC project is to create a publish profile.
This is done by right-clicking your project selecting publish and then configuring it to your needs.
Then from TFS you create a new build definition, this kicks of a wizard which takes you through it.
There are quite a few options which would be too long to go into for every scenario.
The main change I usually find the most important is to set an MSBuild Argument to deploy with the publish profile.
This can be found at Process > Advanced > MSBuild Arguments.
Once this is configured correctly it's a simple case of right-clicking and queue new build to build and deploy.
You wil need different PublishProfile/Build configuration per deployment environment.
For backups I use a powershell script which can be called manually or from TFS.
You also have a drop folder in TFS which keeps a backup of x many releases.
The datbases are automatically configured via Sql server to backup, TBH I didn't set that up it was a DB admin guy who is also involved with releases.
From a dev testing side I use jMeter (http://jmeter.apache.org/) to run some automated scripts that check that users can login and view certain screens, just to confirm nothing major has gone wrong. However there is usually a testing team to run more detailed tests, again not setup by me.
All of the above will probably take you sometime to setup but in the long run it will literally save you weeks of time over a year.
A free alternative to TFS is http://www.cruisecontrolnet.org/, I have used this in the past too and is pretty good.
You can automate your .Net deployments with Beanstalk, which will give you a way to trigger deployments with a single click, watch progress, manage permissions and see history of deployments. Check out this guide on the topic:
http://guides.beanstalkapp.com/deployments/deploy-dotnet.html
I hope you will find it useful.
P.S. - I work at Beanstalk.
I'm trying to embed git describe-generated version info into AssemblyInfo.cs plus some label within ASP.NET website.
I already tried using git-vs-versionino but this assumes Git executable on PATH. However default install of msysgit on Windows does not set this up; it uses git bash. This caused problems.
Now I am looking for a way to utilize libgit2sharp library (for zero external dependencies) to use as build number generator. However this library has no describe command...
Thanks!
git-describe is a UI feature that nobody has implemented in the library or bindings yet (or at least nobody's contributed it), but you can do it yourself fairly easily.
You get a list of the tags and what commits they point to, do a walk down the commits and count how many steps it took to get to a commit that you have in the list you built. This already gives you the information you need. If the steps were zero, then your description would be the tag name only; otherwise you append the number of steps and the current commit's id to it.
There's a work in progress libgit2 pull request that proposes an implementation of git-describe functionalities.
See #1066 for more information.
It's not finished yet. Make sure to subscribe to it in order to be notified of its future progress.
Once it's done, it should be quite easy to bind it and make it available through LibGit2Sharp.
In the project I'm working for we're having a continuous deployment setup. The goal is to always install the latest working build to production, unless someone manually overrides this functionality.
In order to make this working we
Run static code analysis
Run unit tests
Run integration tests
Run automatic UI tests, to the extent this is feasible
If any of the above steps fail, the build process is halted, and the build marked as failed. If the installation package is created it is then in steps installed to
CI --> staging --> production
At each step we run a integration and UI tests for the environment, to make sure we didn't introduce some new things which fail on on the subsequent environments. If none of the tests fail, and N minutes pass without anyone pressing the panic button, the build gets promoted to the next env. If the tests fail, we want to delete the package, and discard it completely. The installation packages are, however, delivered to other servers, so we need to run a bunch of remote (shell) scripts to make this step happen.
The problem is, that there are a big set of failure cases which we cannot reliably test in the normal automated cycle, e.g. page layout, or some integrations fail only production and so on.
So the actual question: How shall I demote/delete builds, once they've been promoted? Is it possible to either run a remote script when doing delete build or use any of the promotion plugins to achieve this functionality? Is there some think-outside-the-box solution for this that I might not have thought about?
Instead of deleting builds manually, you may write a Jenkins job that accepts the build number as a parameter, deletes it, and then does the rest of the housekeeping. You can configure Jenkins access privileges so that people do not delete builds manually by accident.
This might be a very particular case, but we decided against creating a separate job for removing the builds, for the very simple reason of keeping all the logging related to a specific build number in one single place. The setup was the following:
Promotion here means make the installation package (RPM) available to the given server, where auto-update handles the actual upgrade of the package.
We have one main build, that builds every time a new commit is available. We had some fine-tuning related to quiet times etc. but basically every new pushed set of commits resulted in a new build. The build contains all the relevant and available testing, which is far from being complete, and probably never will be.
Every hour a separate promotion step handles promotion from staging to production. This build kicks off a separate promotion which takes the latest accepted build from CI to staging. There is a 30min delay before builds were promoted CI-->staging, to prevent accidental promotions for last second commits. Delays were achieved with some bash find scripting. The order of promotions is this, to make sure a build is available in staging for (at least) one hour before going to promotion.
The actual answer:
The promotion steps were done as separate builds. In order to do a real promotion, rather than a separate build with a separate log, the build kicks off an actual promotion in the main build, using curl and calling the remote HTTP API. This leaves a relevan promotions star in the main build log. Using different colors, the promotions are visible with one look.
To demote builds I decided to create a separate "demote build" promotion step. This would then issue a purple star as a sign of the build being defective, and thus removed. The demotion is done by accessing the correct build in the UI, and pressing the "Remove build" button. No automation has been added to this step, but by creating a separate test step, it would be fairly easy to automate the demotion as well. We, however, have not gotten quite this far yet.
The benefits of this approach include
A build is deleted by accessing the failed build, not by providing parameters. Makes it much easier to document, and get right under pressure
Having a "panic button" like this available for anyone to press, builds trust and ownership for the process not only amongst the developers but also managers and DevOps.
It's dead simple to spot dead builds, as the log is available besides the other promotion logs
Having all the relevant promotion calls in the same build makes further scripting easier
Acute things we still have to improve include automating the testing on the later stages of the build pipeline, and also a suitable way of downgrading builds after demotion. E.g. in production a defective build and a demotion must always lead to installing the last good build, which has turned out to be fairly hard to achieve. Production data centers are rarely allowed to be accessible to this level from the development DC where the CI system sits. Also stopping and starting the build pipeline must be automated, as else there is the chance of slipping back to the manual state.
Naturally, in the spirit of continuous improvement, there are always things to improve. The whole setup is something of a bash/perl scripting mess, but since it's scripted and repeatable, there is always the option of improving one small piece at a time. The most important thing is the automation, as it allows for incremental steps, which any manual steps more or less prevent.
For anyone looking for an easy way to delete a build with custom steps:
Create a 'defective' promotion.
Make it manually triggered.
Force it run on the master.
Add a choice parameter DELETE with choices NO and YES.
Add action Execute Shell.
_
if [ "${DELETE}" == "YES" ]; then
# TODO: my custom steps
curl -X POST ${PROMOTED_URL}/doDelete"
fi
To delete a build now, just go to promotions, flip the choice to YES and click approve.
After reading the builtin help, it seems to me that both commads can be used for modifying the workspace to match a certain revision. But I don't understand the differences between update and checkout. Please include some trivial workflows in your answer which show when update/checkout are appropriate.
First major difference is that if you have a remote url set, update will pull first latest artifacts from the remote repository.
Another difference is that if you have uncomitted changes, checkout will not run (unless you force it), whereas update will retain your changes and reapply them. With update you can therefore integrate changes from other users before committing.
So:
Update is what you need when you collaborate on a project, in order to prevent forks.
Checkout lets you deploy a particular version.