Flyway is a very nice tool to automate database updates (also called migrations). However, as of version 1.7 it relies on a completely linear sequence of migrations. This assumption is immediately void if you have a production system for which you have to deliver fixes while you are already developing new stuff. The FAQ argues correctly that this is a non-issue for the production system itself, but if you have development and/or QA-systems that already on the development branch, you need to run migrations from the fixes for the production version out of band.
A solution that would allow this is pending with Issue 138, but is not done yet. Since this is pretty much a deadly problem: are there any clever workarounds if I want to use it right now?
The approach I recommend (and which becomes almost essential in a Continuous Delivery/deployment) environment is using Feature Toggles and release from HEAD, instead of using Feature or Release branches. This is then combined with backward compatible migrations to complete alleviate this problem.
If for some reason that isn't an option for you, you don't have to wait very much longer.
Flyway 1.8 (which will include the fix for 138) will be out soon.
The problem is obsolete since Flyway version 2.0: if you set the outOfOrder flag then flyway will also execute migrations with earlier version numbers that have not been applied yet. You need however to make sure that such out of band migrations can be applied in any order to the later migrations, or you will run into serious trouble.
With Flyway-1.7 you could make the following workaround. If you have a development and a production branch, you could have separate instances of flyway including separate metadata tables (say, SCHEMA_HISTORY and SCHEMA_HISTORY_DEV) for the production and the development branch. On the production server there is only the SCHEMA_HISTORY and you work as usual; for the development server you have both, and each time you run flyway you first run it on the production branch sqls with the SCHEMA_HISTORY and then on the development branch sqls with the SCHEMA_HISTORY_DEV.
When you switch branches you have to merge the SCHEMA_HISTORY_DEV into SCHEMA_HISTORY. (You need to exclude the initial revision and reset the CURRENT_VERSION on SCHEMA_HISTORY.) And when flyway-1.8 comes out, you do this merge and throw SCHEMA_HISTORY_DEV away.
Related
We have an application we inherited thats on version 500+, we are not sure why its such a high version, but so be it. We have been using Flyway for a few years on it now, and have multiple releases. Sample would be it started as 500.10.4, and we are now on 500.10.20, so 16 releases containing various flyway scripts on a lot of them, but not all.
Anyway, its been determined that for simplification we are to re-version the application to 6.0.0 in the next release. Is there an easy way to let flyway know of this change, so that if we stand up another instance when it runs through the scripts it would run the 500's first, then go back to the 6's?
Currently our flyway script files are named as such:
V500.10.20_2022.05.12.0000.1__xxxx.sql and so on. So in theory our next would be
V6.0.0_2022.05.13.0000.1__xxxx.sql
I know that flyway would see version 6 as lower than 500 and ignore it. We currently have flyway out of order set to false. Is there any other options to solve this other then to set out-of-order processing to true?
In our situation we do not have any flyway scripts pre version 500. So what we are going to attempt to do is have a manual script run that will update all the data in our xxx_db_version table to be version 5.00.xxxx instead of 500.xxxx. This way when we move to 6.0, all of the scripts would be seen as next in the sequence appropriately. While the versions in this table will then not match previous actual versions of the application this table is not used for the purposes of the actual displayed version of the system or anything, and once we move to version 6, the 500 vs 5 won't really matter. and the order/sequence will still be maintained.
If this does not work, I will post a follow up.
When you initialize Firebase hosting, it includes a comment in the header of the index.html file that is generated:
<!-- update the version number as needed -->
<script defer src="/__/firebase/7.5.2/firebase-app.js"></script>
My question has to do with "as needed;" I looked at the docs, and didn't see an explanation.
Probably this means it is supposed to be obvious -- but when you're a beginner, most things aren't!
So, to make my question more concrete:
When might updating the version make a Firebase web app break?
Relatedly, if an app is working, and one does not update for a long
time (many versions/years), does the app remain functioning? Or will it break if not kept current?
Does "as needed" imply "as needed [for access to new features]"?
Finally, is it implied that these changes should be implemented
manually -- by regularly looking up what the latest Firebase version
is, and typing a new version number in index.html -- or is there some
kind of automatic "stay current" workflow/tooling/convention that is
implied?
I realize that there are a number of sub-questions above, but they are all intended to be clarifications of "update as needed," so I think they belong in the same place.
I hope any answers will help other beginners understand the larger issue of when it is appropriate to update the services an app depends upon! Thanks.
Firebase follows what is known as semantic versioning (SemVer) rules.
From semver.org:
Given a version number MAJOR.MINOR.PATCH, increment the:
MAJOR version when you make incompatible API changes,
MINOR version when you add functionality in a backwards compatible manner, and
PATCH version when you make backwards compatible bug fixes.
That means that the API is guaranteed to stay compatible within minor version (7.x) in your case, but breaking changes may be made in major version (8.0). This means that minor versions (7.x) are used to fix problems, and sometimes add minor features that don't break existing behavior.
With that knowledge, let's see if we can answer your questions:
When might updating the version make a Firebase web app break?
Updating within the same major version (7.x, e.g. 7.5.2 -> 7.5.3, or 7.5.2 -> 7.6.0) should not break your app. There are some exceptions, such as when your code depends on buggy behavior that was fixed, or when there is a mistake in the release. The latter will typically be fixed by the Firebase team as soon as possible, while you'll typically want to roll back to the previous version and update your code in the former case.
Relatedly, if an app is working, and one does not update for a long time (many versions/years), does the app remain functioning? Or will it break if not kept current?
Once a version is published, it remains unmodified. So your app will stay working the way it did when you made it.
Does "as needed" imply "as needed [for access to new features]"?
Two main reasons for upgrading:
To get access to new features.
This is the most obvious reason to upgrade, as it allows you to add new functionality from Firebase to your app. Most often this
To get access to bug fixes.
Bugs may be discovered in the library version you use, and some of those bugs may be security holes. In that case, not updating to a more recent version means that you'll have a known security vulnerability in your app. Key to realize here is the known part: most hackers search for apps with known vulnerabilities, instead of trying to find new vulnerabilities.
Finally, is it implied that these changes should be implemented manually -- by regularly looking up what the latest Firebase version is, and typing a new version number in index.html -- or is there some kind of automatic "stay current" workflow/tooling/convention that is implied?
If you use a tool to build/pack your website, that typically has a way to automatically pull in new versions.
Many developers configure such a tool to automatically pull in new patches (7.5.x) upon every build, while some even pull in new minor release (7.x.x). But there's also a school of thought that prefers to hard-code the exact version number and only upgrade manually by regularly checking.
Either way, it's required to make a new build to upgrade, even in this case. That's a Good Thing™️, as the last thing you want is that your app breaks in production when Firebase accidentally releases a new version with a bug (a rare occurrence, but it has happened). By only including a new version in your build process, you reduce this risk, especially if you run automated tests of your app's functionality as part of the build.
There's no right or wrong answer here, as either can work just fine. It's really up to your own preference.
I just encountered the following situtation:
The test-server is currently running Flyway, with version 1 (V1). The test-server is automatically updated (including Flyway scripts) whenever anything is pushed on the develop branch.
A developer decides to start working on a new feature on branch feature/123. This developer creates a database script (Flyway compatible) called V2__cool_feature.sql. In the meantime, another developer also starts working on a feature branch called feature/456. This developer is also in need of an update script, and names it V3__another_cool_feature.sql, because the developer knows that V2 is already used on another branch. This feature/456 branch is finished and is merged, and so the current scripts on the develop branch are V1 & V3. This works well and the V3 script is executed, leaving Flyway its schema_version on version 3.
The other feature branch feature/123 is also merged, which means that the develop branch contains the scripts V1, V2 & V3.
Now this is were I'm having trouble with Flyway:
The build, including Flyway, is executed and it leaves the following message:
[INFO] Database: jdbc:mysql://example.com:3306/my_schema (MySQL 5.5)
[INFO] Successfully validated 2 migrations (execution time 00:00.019s)
[INFO] Current version of schema `my_schema`: 3
[INFO] Schema `my_schema` is up to date. No migration necessary.
What I want to happen is that the V2 script is executed, and I'm not sure how to do so.
I hope I explained my problem well, if not, please leave a comment.
Ugh, I'm not a smart guy. Putting a bit more effort into my Googling skills lands me upon the Flyway documentation, describing exactly my problem:
Option:
outOfOrder
Required:
NO
Default:
false
Description:
Allows migrations to be run "out of order".
If you already have versions 1 and 3 applied, and now a version 2 is found, it will be applied too instead of being ignored. (Even the same versioning as in my question is used >.< )
We use artifactory as a repo manager. We store packages for releases in repo libs-release-local and snapshot packages in libs-snapshot-local.
For example: If we send the war (web app) into test, it should be from libs-snapshot-local and if it is into stage & prod it will be from libs-release-local.
I will say a scenario, where I needed help below:
Once the war is certified good on test server we would send the same code to stage.
We saw a bug after deployed into stage, so we changed the code and again build it, it obviously cannot go to same versioned release(as in artifactory, we cannot override releases).
So,
What will happen if we recognized 10 bug fixes one at a time, after each deployment in stage ?
What if we have realized there are bug fixes after going to prod.
Artifactory will have bulk of folders with so many version names & folders. Is that good practice? Or else anything senses wrong in our procedure?
Thanks!
Suggest to read Binary repository management Refcard first.
You need define your strategy for your folders and wars (web application), it is already good to use different repo for different purpose (snapshot/release)
The process for maintenance is simple
fix the bug and increase the version, send to the libs-snapshot-local for testing
after testing, a.k.a QA passed, the packages are promoted to release/stage repo libs-release-local again for public use again.
In this case, bug fix is the same as normal development procedure.
or you can refine the questions to make your questions more clear.
Net solution for a website, consisting of 5 projects, and there are a few(less than 10) developers working on the solution. We deploy almost on a daily basis.
The question is, how to setup the SVN repo to support this scenario (the daily deploy), also mentioning that not every commited file should go to production, there is a QA check before deploying.
Try out TeamCity
(CI tool) as its free for smaller amounts of CI. this may be better for you than CruiseControl.Net as CCNET is very configuration heavy as its all done via XML. TeamCity uses wizards to create the scripts to manage releases
if you need any other help on CI then let me know as its something I am evangelistic about.
What you want to do is commonly referred to as Continuous integration (CI).
While you can do that using Subversion, it is probably not the right tool for the job.
There is special CI software, which will allow you to easily automate the necessary tasks (checkout from version control, compiling / building, running automatic tests, deployment etc). An example would be CruiseControl.NET.
As to "not every commited file should go to production", the common solution is to have a special "release" branch, which gets deployed. Only tested code is merged there (or have the trunk always be stable, otherwise same model). Of course, you can also (better: additionally) have tests before your automatic deployment, and only deploy if all tests pass.
Working with a release branch
In practice, this means that people check in their code as they produce it. Sometimes this code will work, sometimes not. When the release time draws nearer, a "release branch" is created in Subversion. This release branch is then effectively a frozen snapshot of the source as it was at the time of branching. Now this branch can be used to compile & deploy the application, which can then be tested.
No new code is checked into the branch (but checkins can continue elsewhere). Only if a bug is detected in the branch, will there be a checkin into the branch to fix it. This continues until the branch passes all tests. Then the branch can be released as a new version of the software; afterwards the branch will only be used if the released version needs to be patched.
Of course, any bugfixes checked into the branch need to also be put into the trunk (either by merging branch -> trunk, for which Subversion provides special support, or by reimplementing the fix in the trunk, as appropriate).