I am trying flyway for first time, evaluating how it will fit into our project.
Trying to understand how a Failed Scenario will work
Naturally, what i did next was, modified the sql script, and re tried running, but got checksum error
Have three Ques here
So I guess the only way out is ... need to make a 1.2 with correct format or manually modify 'schema_version' table. Right , or am i missing something?
Wondering how will such a scenario work in case if this script is called from continuous integration tools (Jenkins or Bamboo). Manual Intervention will be needed.
Not sure if some other tool like Liquibase will behave in a different (better) manner
In that situation I think you should use "flyway repair" rather the "flyway migrate"
https://flywaydb.org/documentation/command/repair
One thing from your post that is not clear is was the script you ran a single DDL statement or a number of statements, of which one or more failed?. The reason for asking is that flyway records the results of a migration, but does not itself clean up 'script errors'. Depending on the database you are using this could be done by running the DDL statements within a transaction.
Liquibased operates with a much tighter connection to the database as it directly interacts with the DDL that can be expressed in a range of different formats. As such it has a much tighter control over the management of DDL deployment.
Upstream insists on manual rolling back of failed migration and re-applying it again. There is no "skip" command.
But you can manually fix and complete failed migration and manually change "schema_version"."success" to 1.
Related
I use flyway 8.5.0 and I want that my beforeMigrate or my afterMigrate sql is reported in the history table. Is this feaseable? or is there any config to setup this?
Then an other question: my repetable only runs when they change (checksum) but for my understanding the repetible sql should run every time. not so?
The beforeMigrate and afterMigrate SQL wont appear in your history table. If you look at the tutorial example for callbacks you can see that beforeMigrate can be called before the schema history table is created which would cause issues if it was trying to add itself to it. Additionally, I'm assuming these will be mostly static executions and would not really be part of the version history.
https://flywaydb.org/documentation/tutorials/callbacks
For repeatable, no they are only applied when the checksum has changed.
Repeatable migrations are very useful for managing database objects whose definition can then simply be maintained in a single file in version control. Instead of being run just once, they are (re-)applied every time their checksum changes.
https://flywaydb.org/documentation/tutorials/repeatable
We have not used Flyway from the beginning of our project. We are at an advanced state of development. An expert review has suggested to use Flyway in our project.
The problem is that we have moved part of our services (microservices) into another testing environment as well.
What is the best way to properly implement Flyway? The requirements are:
In Development environment, no need to alter the schema which is already existing. But all new scripts should be done using Flyway.
In Testing environment, no need to alter the schema which is already existing. But what is not available in testing environment should be created automatically using Flyway when we do migrate project from Dev to test.
When we do migration to a totally new envrionment (UAT, Production etc) the entire schema should be created automatically using Flyway.
From the documentation, what I understood is:
Take a backup of the development schema (both DDL and DML) as SQL script files, give a file name like V1_0_1__initial.sql.
Clean the development database using "flyway clean".
Baseline the Development database "flyway baseline -baselineversion=1.0.0"
Now, execute "flyway migrate" which will apply the SQL script file V1_0_1__initial.sql.
Any new scripts should be written with higher version numbers (like V2_0_1__account_table.sql)
Is this the correct way or is there any better way to do this?
The problem is that I have a test database where we have different set of data (Data in Dev and test are different and I would like to keep the data as it is in both the environments). If so, is it good to separate the DDL and DML in different script files when we take it from the Dev environment and apply them separately in each environment? The DML can be added manually as required; but bit confused if I am doing the right thing.
Thanks in advance.
So, there are actually two questions here. Data management and Flyway management.
In terms of data management, yes, that should be a separate thing. Data grows and grows. Trying to manage data, beyond simple lookup tables, from source control quickly becomes very problematic. Not to mention that you want different data in different environments. This also makes automating deployments much more difficult (branching would be your friend if you insist on going this route, one branch for each data set, then deploy appropriately).
You can implement Flyway on an existing project, yes. The key is establishing the baseline. You don't have to do all the steps you outlined above. Let's say you have an existing database. You have to get the script that defines that database. That single script should include all appropriate DDL (and, if you want, DML). Name it following the Flyway standards. Something like V1.0__Baseline.sql.
With that in place, all you must do is run:
flyway baseline
That will establish your existing code base as the start point. From there, you just have to create scripts following the naming standard: V1.1xxx V2.0xxx V53000.1xxx. And run
flyway migrate
To deploy appropriate changes.
The only caveat to this is that, as the documentation states, you must ensure that all your databases match this V1.0 that you're creating and marking as the baseline. Any deviation will cause errors as you introduce new changes and migrate them into place. As long as you've got matching baseline points, you should be able to proceed with different data in different environments with no issues.
This is my how-to instruction on integration flyway with prod DB: https://delicious-snipe-938.notion.site/How-to-integrate-Flyway-with-existing-MySQL-DB-in-Prod-PostgreSQL-is-similar-1eabafa8a0e844e88205c2f32513bbbe.
I'm investigating how Flyaway works to see if it's suitable for our upgrade and migration needs.
Can somebody tell me if Flyaway automatically handles precedence so that for example, if a table has a trigger, it creates the table before attempting to create the trigger, or is the user responsible to manually creating and ordering the actual SQL Statements required in each schema update?
The latter - Flyway does not keep knowledge of the dependencies between database objects (as it's a very large and gnarly problem, and it's highly db-dependent). Which database do you use? A tool like Redgate Schema Compare will generate the script with all the correct dependency orderings for you.
I'm evaluating Flyway and want to know if it can check for the presence of any externally made changes? I.e. if someone makes a change directly to the database, outside of Flyway, can I catch that?
I tried validate and info but it doesn't seem to notice.
No, it can't. Flyway expects you to make all changes to what you want it to manage (structure, reference data, ...) via Flyway.
Here at Redgate we support the concept of a schema snapshot. If you use either SQL Server or Oracle (as these are the database we have best support for) then you can take a snapshot after deployment and later compare this artifact with target database the next time you deploy to make sure that it hasn't "drifted". Our tools can also output a candidate Flyway migration script that will ensure Flyway is consistent with the actual target. If you're interested in this approach I can send you sample scripts of how this is achieved. But remember, this only works with Oracle and SQL Server.
Background
I am running a flyway migration against DB2 using the command line interface. I have done a number of tests and all works fine. I added in some commands to cause a failure.
Question
Can someone confirm if alter table, drop table or create table SHOULD be rolled back if the migration fails and the database supports DDL? When i test it it looks like an alter table add column statement was not rolled back after a failure in the same flyway script.
Ok so i checked the flyway website and found a list of supported drivers and versions where DDL is supported. It seems like the version of DB2 we are using is below this threshold. Based on that i guess the execution of migrate script is not bound in a single transaction and that is why i am seeing some changes remaining that were applied before the script failure.