I am currently developing a branch flow based on GitFlow, I am in the stage of defining how the interaction between the hotfix branch and the develop branch will be done after a correction in production, although GitFlow recommends that a direct merge be done, I particularly want consider an intermediate branch through which to create a Pull Request and in this way peer review can be done, in addition to avoiding committing directly to the "release" and "develop" branch. The simplest way to do this is to use a feature branch, however it is not appropriate to use it.
Please does anyone have any suggestions for the definition of the name of this branch that I need? I'm considering using a "develop-integration" branch.
First, let's reiterate what Git Flow recommends for hotfix branches:
Create a branch for the hotfix, perhaps hotfix-number or hotfix/number, based on master (or main if you call it that instead).
Merge the hotfix branch into master when you deploy to Production.
Also merge hotfix into release if it exists, or develop if release doesn't exist, and now delete the hotfix branch.
For #1, the branch name of the hotfix doesn't actually matter. In Git branch names are simply to help us know what they are for, but you could call your hotfix branch fix-some-bug if you want to. This also translates to your last question:
...does anyone have any suggestions for the definition of the name of this branch that I need?
In the context of an in-between branch, perhaps even moreso than a hotfix branch, it really doesn't matter. In one of my repos where I use Git Flow, we named this in-between branch temp/merge-hotfix-into-develop since it only exists for a few minutes.
Note for #3 above about merging, after using Git Flow for a while, my preference, for both release and hotfix branches, after merging into master, is to then merge master into develop rather than the release or hotfix branch itself. This is slightly cleaner in the long run, but the end result state is identical. Therefore, since I do it this way, my actual branch name in this case would be temp/merge-master-into-develop.
That being said, I'd like to challenge your reasoning for this in the first place:
... although GitFlow recommends that a direct merge be done, I particularly want consider an intermediate branch through which to create a Pull Request and in this way peer review can be done, in addition to avoiding committing directly to the "release" and "develop" branch.
In Git Flow, anytime you are merging one shared branch into another, we can divide the merge into 2 categories:
The merge does not have conflicts.
The merge does have conflicts.
In scenario #1 (which hopefully is the most common scenario), there is nothing to be reviewed, so the PR should be basically automatic. In this case simply create a PR from hotfix (or master) to develop and complete it. You can do a sanity check first if you want to, but it should be extremely rare that it would fail at this point. If the sanity check fails, then you could resort to treating this like #2.
In scenario #2, this is when you need to create the in between branch as you mentioned. I always test the merge first, and if there are conflicts I'll create a branch, as mentioned above, like: temp/merge-release-into-develop or temp/merge-master-into-develop, resolve the conflicts myself (except in rare cases when I can't in which case I'll pull in other devs), and push out the branch and create the PR into, in this case, develop.
I download code from Gitlab, then create a new branch, update some files and create a review. Then I add a file, execute arc diff --allow-untracked, it reports Usage Exception: There are several revisions which match the working copy:. I execute arc diff --allow-untracked --update D75 to update the existing review, but it creates a new review.
Linting...
No lint engine configured for this project.
Running unit tests...
No unit test engine is configured for this project.
SKIP STAGING No staging area is configured for this repository.
Updating commit message...
Created a new Differential revision:
Revision URI: http://codereview.domain.com/D76
I try many times but it always creates a new revision. How to use arc diff --update?
You shouldn't need --update but you do need to specify the base of your change, so to update the revision associated with the current HEAD, try the following:
arc diffHEAD^
If you've got two commits then you need to use HEAD^^.
To update everything on the current branch:
arc diffparent-branch
So assuming your current branch was branched from master:
arc diffmaster
The arcanist documentation covers this topic in detail, so for a more complete understanding of how arc diff decides which commits to submit, read Arcanist User Guide: Commit Ranges.
I have two users sharing R code on a TFS Server. One user is in RStudio (v1.0.143), the other in Visual Studio 2015.
When I look at the code history, it looks like the RStudio user is always doing two commits instead of one: the first is the legitimate commit, and the second looks like a merging of branches, with the comment "Merge branch 'master' of ..."
How do I stop this constant branch-and-merge? Is there a way RStudio can just do regular commits?
This is the default behavior of git when there's at least one commit locally that is not in the remote (and vice versa) when the push occurs. Git creates a merge commit to combine the branches that have diverged.
You might ask your RStudio user to perform a pull immediately before they make their commit, and to perform a push immediately after, so that they don't wind up with a divergent branch.
In git parlance, what you want is for the RStudio user to be able to do a fast-forward merge when pushing changes. Hope that helps!
I'm using git flow.
We have develop branched from master, and I have feature/INF-824 branched from develop.
One of my coworkers has added a change to develop that's relevant to INF-824.
How can I merge his changes into my copy of develop, and see them in feature/INF-824?
Thanks!
You can cherry pick the change from the develop branch.
git cherry-pick <commit-ish>
Commit-ish would be the SHA1-ID of the commit you want in your feature branch.
The other option would be to use rebase on your feature branch
git rebase feature branch
This might be better solution as you'll be merging your feature back in develop anyway on finishing the feature branch.
The process of creating a new build and releasing it to production is a critical step in the SDLC but it is often left as an afterthought and varies greatly from one company to the next.
I'm hoping people will share improvements they have made to this process in their organisation so we can all takes steps to 'reduce the pain'.
So the question is, specify one painful/time consuming part of your release process and what did you do to improve it?
My example: at a previous employer all developers made database changes on one common development database. Then when it came to release time, we used Redgate's SQL Compare to generate a huge script from the differences between the Dev and QA databases.
This works reasonably well but the problems with this approach are:-
ALL changes in the Dev database are included, some of which may still be 'works in progress'.
Sometimes developers made conflicting changes (that were not noticed until the release was in production)
It was a time consuming and manual process to create and validate the script (by validate I mean, try to weed out issues like problem 1 and 2).
When there were problems with the script (eg the order in which things were run such as creating a record which relies on a foreign key record which is in the script but not yet run) it took time to 'tweak' it so it ran smoothly.
It's not an ideal scenario for Continuous Integration.
So the solution was:-
Enforce a policy of all changes to the database must be scripted.
A naming convention was important for ensuring the correct running order of the scripts.
Create/Use a tool to run the scripts at release time.
Developers had their own copy of the database do develop against (so there was no more 'stepping on each others toes')
The next release after we started this process was much faster with fewer problems, indeed the only problems found were due to people 'breaking the rules', eg not creating a script.
Once the issues with releasing to QA were fixed, when it came time to release to production it was very smooth.
We applied a few other changes (like introducing CI) but this was the most significant, overall we reduced release time from around 3 hours down to a max of 10-15 minutes.
We've done a few things over the past year or so to improve our build process.
Fully automated and complete build. We've always had a nightly "build" but we found that there are different definitions for what constitutes a build. Some would consider it compiling, usually people include unit tests, and sometimes other things. We clarified internally that our automated build literally does everything required to go from source control to what we deliver to the customer. The more we automated various parts, the better the process is and less we have to do manually when it's time to release (and less worries about forgetting something). For example, our build version stamps everything with svn revision number, compiles the various application parts done in a few different languages, runs unit tests, copies the compile outputs to appropriate directories for creating our installer, creates the actual installer, copies the installer to our test network, runs the installer on the test machines, and verifies the new version was properly installed.
Delay between code complete and release. Over time we've gradually increased the amount of delay between when we finish coding for a particular release and when that release gets to customers. This provides more dedicated time for testers to test a product that isn't changing much and produces more stable production releases. Source control branch/merge is very important here so the dev team can work on the next version while testers are still working on the last release.
Branch owner. Once we've branched our code to create a release branch and then continued working on trunk for the following release, we assign a single rotating release branch owner that is responsible for verifying all fixes applied to the branch. Every single check-in, regardless of size, must be reviewed by two devs.
We were already using TeamCity (an excellent continuous integration tool) to do our builds, which included unit tests. There were three big improvements were mentioning:
1) Install kit and one-click UAT deployments
We packaged our app as an install kit using NSIS (not an MSI, which was so much more complicated and unnecessary for our needs). This install kit did everything necessary, like stop IIS, copy the files, put configuration files in the right places, restart IIS, etc. We then created a TeamCity build configuration which ran that install kit remotely on the test server using psexec.
This allowed our testers to do UAT deployments themselves, as long as they didn't contain database changes - but those were much rarer than code changes.
Production deployments were, of course, more involved and we couldn't automate them this much, but we still used the same install kit, which helped to ensure consistency between UAT and production. If anything was missing or not copied to the right place it was usually picked up in UAT.
2) Automating database deployments
Deploying database changes was a big problem as well. We were already scripting all DB changes, but there were still problems in knowing which scripts were already run and which still needed to be run and in what order. We looked at several tools for this, but ended up rolling our own.
DB scripts were organised in a directory structure by the release number. In addition to the scripts developers were required to add the filename of a script to a text file, one filename per line, which specified the correct order. We wrote a command-line tool which processed this file and executed the scripts against a given DB. It also recorded which scripts it had run (and when) in a special table in the DB and next time it did not run those again. This means that a developer could simply add a DB script, add its name to the text file and run the tool against the UAT DB without running around asking others what scripts they last ran. We used the same tool in production, but of course it was only run once per release.
The extra step that really made this work well is running the DB deployment as part of the build. Our unit tests ran against a real DB (a very small one, with minimal data). The build script would restore a backup of the DB from the previous release and then run all the scripts for the current release and take a new backup. (In practice it was a little more complicated, because we also had patch releases and the backup was only done for full releases, but the tool was smart enough to handle that.) This ensured that the DB scripts were tested together at every build and if developers made conflicting schema changes it would be picked up quickly.
The only manual steps were at release time: we incremented the release number on the build server and copied the "current DB" backup to make it the "last release" backup. Apart from that we no longer had to worry about the DB used by the build. The UAT database still occasionally had to be restored from backup (eg. since the system couldn't undo the changes for a deleted DB script), but that was fairly rare.
3) Branching for a release
It sounds basic and almost not worth mentioning, yet we weren't doing this to begin with. Merging back changes can certainly be a pain, but not as much of a pain as having a single codebase for today's release and next month's! We also got the person who made the most changes on the release branches to do the merge, which served to remind everyone to keep their release branch commits to an absolute minimum.
Automate your release process whereever possible.
As others have hinted, use different levels of build "depth". For instance a developer build could make all binaries for runnning your product on the dev machine, directly from the repository while an installer build could assemble everything for installation on a new machine.
This could include
binaries,
JAR/WAR archives,
default configuration files,
database scheme installation scripts,
database migration scripts,
OS configuration scripts,
man/hlp pages,
HTML documentation,
PDF documentation
and so on. The installer build can stuff all this into an installable package (InstallShield, ZIP, RPM or whatever) and even build the CD ISOs for physical distribution.
The output of the installer build is what is typically handed over to the test department. Whatever is not included in the installation package (patch on top of the installation...) is a bug. Challenge your devs to deliver a fault free installation procedure.
Automated single step build. The ant build script edits all the installer configuration files, program files that need changed ( versioning) and then builds. No intervention required.
There is still a script run to generate the installers when it's done, but we will eliminate that.
The CD artwork is versioned manually; that needs fixed too.
Agree with previous comments.
Here is what has evolved where I work. This current process has eliminated the 'gotchas' that you've described in your question.
We use ant to pull code from svn (by tag version) and pull in dependencies and build the project (and at times, also to deploy).
Same ant script (passing params) is used for each env (dev, integration, test, prod).
Project process
Capturing requirements as user 'stories' (helps avoid quibbling over an interpretation of a requirement, when phrased as a meaningful user interaction with the product)
following an Agile principles so that each iteration of the project (2 wks) results in demo of current functionality and a releasable, if limited, product
manage release stories throughout the project to understand what is in and out of scope (and prevent confusion abut last minute fixes)
(repeat of previous response) Code freeze, then only test (no added features)
Dev process
unit tests
code checkins
scheduled automated builds (cruise control, for example)
complete a build/deploy to an integration environment, and runs smoke test
tag the code and communicate to team (for testing and release planning)
Test process
functional testing (selenium, for example)
executing test plans and functional scenarios
One person manages the release process, and ensures everyone complies. Additionally all releases are reviewed a week before launch. Releases are only approved if there are:
Release Process
Approve release for a specific date/time
Review release/rollback plan
run ant with 'production deployment' parameter
execute DB tasks (if any) (also, these scripts can be version and tagged for production)
execute other system changes / configs
communicate changes
I don't know or practice SDLC, but for me, these tools have been indispensible in achieving smooth releases:
Maven for build, with Nexus local repository manager
Hudson for continuous integration, release builds, SCM tagging and build promotion
Sonar for quality metrics.
Tracking database changes to development db schema and managing updates to qa and release via DbMaintain and LiquiBase
On a project where I work we were using Doctrine's (PHP ORM) migrations to upgrade and downgrade the database. We had all manner of problems as the generated models no longer matched with the database schema causing the migrations to completely fail half way.
In the end we decided to write our own super basic version of the same thing - nothing fancy, just up's and down's that execute SQL. Anyway it worked out great (so far - touch wood). Although we were reinventing the wheel slightly by writing our own, the fact that the focus was on keeping it simple meant that we have far less problems. Now a release is a cinch.
I guess the moral of the story here is that it is sometimes OK to reinvent the wheel some times as long as you are doing so for a good reason.