I use flyway 8.5.0 and I want that my beforeMigrate or my afterMigrate sql is reported in the history table. Is this feaseable? or is there any config to setup this?
Then an other question: my repetable only runs when they change (checksum) but for my understanding the repetible sql should run every time. not so?
The beforeMigrate and afterMigrate SQL wont appear in your history table. If you look at the tutorial example for callbacks you can see that beforeMigrate can be called before the schema history table is created which would cause issues if it was trying to add itself to it. Additionally, I'm assuming these will be mostly static executions and would not really be part of the version history.
https://flywaydb.org/documentation/tutorials/callbacks
For repeatable, no they are only applied when the checksum has changed.
Repeatable migrations are very useful for managing database objects whose definition can then simply be maintained in a single file in version control. Instead of being run just once, they are (re-)applied every time their checksum changes.
https://flywaydb.org/documentation/tutorials/repeatable
Related
I have installed Flyway Software and trying to deploy code. I have a Scenario
I created a file V1_01_CREATE_TABLE.SQL to create a table.
Created a file V1_01_CREATE_PACKAGE.SQL - This package will hold code to insert a row into one of the columns in table created in step 1.
Created a file V1_01_02_ALTER_TABLE.SQL - This SQL will rename the column which was being referred in step2.
This will invalidate the package if I run 1, 2 & 3. How does FLYWAY handle such a situation? Does it understand object dependency?
As explained in the manual, Flyway simply passes your SQL scripts to the database to be executed, and records the success or failure of their execution.
Flyway has no interest or understanding of the content of your scripts. Flyway never looks at the content of those scripts. In that sense, there is no “intelligence” in Flyway.
Flyway is like the postal worker delivering your letters without opening the envelope. You are the author, and you take full responsibility for the logic and correctness of the SQL scripts. You are responsible for following the naming conventions so your scripts run in the correct order.
After initially creating its metadata table, Flyway makes very limited use of JDBC and SQL. Flyway does little more than make a connection to the database server, determine which of the scripts have yet to be run, and say to the database “Here, run this script, and this script, and then run this script.” while recording the success or failure of each run.
You should name your scripts in order they needs to be executed
V1.0.0_CREATE_TABLE.SQL
V1.0.1_CREATE_PACKAGE.SQL
V1.0.2_ALTER_TABLE.SQL
This will execute scripts in migrated
I have delivered a Product to the customer. Now I have upgraded the Product, which includes changes to the database.
Customer wants to upgrade the Product. Now will Flyway help in the migration of Customer data from older version to newer version. Please let me know, if this is a valid use case. The flyway documentation talks about its use during development only.
Flyway allows you to change your database by running a set of scripts in a defined order. These scripts are called 'migrations' as they allow you to 'migrate' your database from one version to another.
The idea is you can start with an an empty database and each migration script will successively bring that database up from empty up to the current version. However, it's also possible to start with an existing database by creating a 'baseline' migration.
As SudhirR said, Flyway's primary use case is to define schema changes. However, it's perfectly possible to change data also. Since Flyway is just running plain SQL, in principle almost anything you can do in a SQL script you can also do in a Flyway migration.
In the case you described it should be possible to use Flyway to migrate the customer database. The steps you could take are:
Generate a sql script that includes the entire DDL (including indexes, triggers, procedures, ...) of the production database. To do this you will need to add insert statements for all the reference data present in the database.
Save this script in your Flyway project as something like 'V1__base_version.sql'
Run the flyway baseline command against your production database
This will set up your production database for use with Flyway
Add a new migration script to migrate your customer's data to the new version
e.g. create new table, copy data from old table to new table, delete old table
Run flyway migrate to upgrade production
These steps are adapted from the Flyway documentation page here.
Of course you should read the Flyway docs and manually test on a throwaway DB before you run anything against production. However I think in principle Flyway could be a good fit for your use case.
Flyway should be used for schema migrations and any reference data (basic data that is required by the system/application in order to function properly).
Putting client specific data migrations would not be a use case. However, if you can represent the data migration "generically" by not using IDs and instead use names or types than it could be a candidate. Meaning if you could write a migration in a way that could be applied to all clients, then that would be the use case to put it in as a flyway migration.
Otherwise data migrations would be applied in some other way outside of the process like requesting special access to the database or having some team that manages the database to apply the scripts.
If you are doing custom data modifications quite often then I'd say something is wrong in some other area of the SDLC and you may need to increase testing so that bugs don't mess up the data in the first place.
I am trying flyway for first time, evaluating how it will fit into our project.
Trying to understand how a Failed Scenario will work
Naturally, what i did next was, modified the sql script, and re tried running, but got checksum error
Have three Ques here
So I guess the only way out is ... need to make a 1.2 with correct format or manually modify 'schema_version' table. Right , or am i missing something?
Wondering how will such a scenario work in case if this script is called from continuous integration tools (Jenkins or Bamboo). Manual Intervention will be needed.
Not sure if some other tool like Liquibase will behave in a different (better) manner
In that situation I think you should use "flyway repair" rather the "flyway migrate"
https://flywaydb.org/documentation/command/repair
One thing from your post that is not clear is was the script you ran a single DDL statement or a number of statements, of which one or more failed?. The reason for asking is that flyway records the results of a migration, but does not itself clean up 'script errors'. Depending on the database you are using this could be done by running the DDL statements within a transaction.
Liquibased operates with a much tighter connection to the database as it directly interacts with the DDL that can be expressed in a range of different formats. As such it has a much tighter control over the management of DDL deployment.
Upstream insists on manual rolling back of failed migration and re-applying it again. There is no "skip" command.
But you can manually fix and complete failed migration and manually change "schema_version"."success" to 1.
I looked at the Flyway samples and documentation and tried to understand if it is useful in my environment.
The following conceptual detail is unclear to me: How does Flyway manage the changes between database versions? It obviously does NOT compare database life-instances (see answer here:Can Flyway find out and generate migration files from datamodel?)
In detail my setup looks like this:
I create SQL create and insert scripts when coding (automatically and manually). This means every version of my database is represented by a number of insert/create statements.
In my world I execute these scripts through a database tool (sqlplus from Oracle). Each run would setup the database _from_scratch_ (!).
Can I put these very same scripts 1 to 1 inside the "migration" path of Flyway? What happens if the target database is way older than the last "migration step" I did (or flyway did not yet exist when it was installed)?
Update:
I got some input from another Flyway user:
It seems like each "migration" (version of the database) has to be hand-written SQL/Java code and contains only "updates" from the previous "migration" of database.
If this is true, I wonder how this can be used with traditional coding technics: in my world SQL statements are generated automatically and contain all database init/create statements, not just "updates" to some previous version. If my SQL code generator could do that, then I wouldn't even need a tool like Flyway :-).
Your question about "how to handle a DB that has a longer history than there are migration scripts?" You need to create a V1_ migration/sql script that matches/recreates your latest DB schema. Something that can take a blank DB to what you have today. Create/generate that sql script using your existing DB tools and then put it in flyways migration directory. (And test V1 by using flyway against a clean DB and see if you get what you expect.) http://flywaydb.org/documentation/existing.html
After that point in time, all later versions must be added in as you work. When you decide you need a new table, in your dev environment, write a new V*_.sql that modifies your schema to the way you need it.
This blog goes over this situation for a Spring/SQL application. https://blog.synyx.de/2012/10/database-migration-using-flyway-and-spring-and-existing-data/
The scenario is this. I have a SQL Server database online that I am demoing an application. During development, I have added extra fields, modified field types, changed keys and added some new tables locally.
What's the best way for me to update the online database with the new structure and not lose the data? The database is a SQL Server 2005 one.
Download a trial of Red Gate SQL Compare, compare your two servers and you are done. If you do this often, it is well worth the $400, or get one of their bundles for a better bang for the buck.
And I do not work for Red Gate, just a happy customer!
Write update scripts to modify your live database structure to the new structure, as well as inserting any data which is required.
You may find it necessary to use temporary tables to do this.
It's probably best if you test this process on a test environment, before running the scripts on the live environment.
Depending on what exactly you've done you may be able to get away with alter statements, though from the sounds of it (removing keys and whatnot) you're doing some heavy lifting that may make that a less-than-ideal solution. You should probably look into creating a maintenance plan or, better yet, a SQL Server Integration Services project in Visual Studio. You should be able to migrate the data in the existing database to a new one using those tools.
This probably isn't of huge help retrospectively, but I always script all structural DB changes to my development database and then using a version number to determine the current version of the DB I can run the required scripts on the live DB, hence bringing it back in line at the same time as the new code is uploaded.
This also works for any content changes, for instance if the change in the underlying structure has an effect on the conent stored you can also write scripts to migrate the data accordingly.
Make a copy of the existing database to copy from.
Make another copy and alter it to your new schema. save DDL for reuse.
Write queries that copy data from #1 to #2. Save the queries for reuse.
Check the results.
Repeat until done.