I'm working on a project where we use flyway to upgrade our DB schema.
I'm using flyway through the gradle plugin.
I need to execute a specific function every time a migration ends: we don't want to add the call to every migration manually.
The script will be exactly the same every time, so repeatable migration is a no go (it will be executed again only when the hash of the script changes).
I've tried to use sql callbacks, but they seem not working properly with the gradle plugin: if I run flyway on command line the callback sql is executed correctly (all migrations and callback are in sql dir within flyway distro) while it's not executed in gradle (migrations and callbacks are in directory src/main/db/migration set as filesystem:src/main/db/migration in flyway gradle configuration).
Can anyone help with the correct gradle plugin configuration for flyway or maybe suggest an alternative solution?
Many thanks
Use the afterMigrate callback. See https://flywaydb.org/documentation/callbacks
Related
How to create script or Flyway can be configured to call it every time using a SQL callback ??
flyway has already a lot of callbacks depending when you want your script to run, for exemple if you want your script to run after each successfull migration you can create a file named:
afterMigrate__my_script_to_run_after_each_migration.sql
Flyway will automatically call that script after the migration completes successfully.
You can find the list of flyway callbacks here, note that some of them are limited to the Flyway teams edition.
https://flywaydb.org/documentation/concepts/callbacks
I'm using EF Core Migrations in my .NET Core projects, and deploy using DevOps pipelines.
In my build pipeline, I build a migration SQL script using a Command Line task to execute a dotnet ef migrations script command (using the --idempotent option), then execute it in the release pipeline using an Azure SQL Database deployment task. All fairly standard, to judge from a simple google search (eg. here), which is how I learned of the approach in the first place.
This is my problem: To achieve certain desired results, in some of my migrations I use Custom Migration Operations (migrationBuilder.Sql("...")) to execute some hand-crafted SQL as part of a migration.
However, as the project develops, and my DB schema changes over time, older such migrations inevitably contain SQL which no longer fits the schema. One would think this isn't a problem, as any migration is meant to be applied to a database with some specific schema version only. However, as it turns out, the dotnet ef migrations script command builds a SQL script with a bunch of conditional SQL block like this:
IF NOT EXISTS(SELECT * FROM [__EFMigrationsHistory] WHERE [MigrationId] = N'20190123160628_SomeMigrationId')
BEGIN
/* Generated or custom SQL here */
END;
Note: every migration is included in the script, and the IF NOT EXIST-clauses ensure that only the not-yet-applied ones are executed in the database.
However, the custom migration task SQL statements are included as-is in the script, and if they are outdated, they longer compile. Then the DB deployment task fails, as does the whole deployment.
Has anyone else had and solved this problem, or know another way to deploy migrations which doesn't have this issue? It seems to me this seriously affects the utility of the MigrationBuilder.Sql() command, which is the only way to arbitrarily "massage" data as part of a migration.
I'm using Flyway 6.5.7 and I have a afterMigrate.sql that truncates a table. I thought the script would run only if a migration occurred AND if the migration is successful. But it always runs (for example after restart of the application)
Schema "A_SCHEMA" is up to date. No migration necessary.
Parsing afterMigrate.sql ...
Executing SQL callback: afterMigrate -
Is it correct? If so, is there a way to achieve the described behavior?
Note: org.flywaydb.core.Flyway#migrate returns 0 after restart of the application with no migration
Thanks
The way it works is the afterMigrate callback runs after any migration that didn't have an error. A successful migration is defined that way. So, in your situation, you called the migrate command. It ran successfully. It didn't migrate anything. However, it ran without an error, so the afterMigrate callback ran.
There is also the afterEachMigrate event which you can attach a callback to. This runs after each successful execution of a migration script; it doesn't run if migrate finds nothing to do, but on the downside it will run more than once if migrate finds multiple scripts that it needs to run. Depending on how often you push out new scripts, this might be a better solution for you.
I changed a flyway script for error, and this brought my migrations to an error state (I have 5 script versions, but when I run my app it starts from the 4th, and get an error on 'reaction already exists').
I tried to use clean from flyway cli, but it didn't completely solve the problem... also, when I try pushing my branch to git, CI/CD pipeline will fail for the migration.
Since I'm in development environment, would it be a terrible idea to all delete migration scripts?
Would deleting all scripts allow me to 'start from scratch' in my development environment, or do I need to push the whole project again to avoid issues? (Project is not in production yet, I don't really need flyway migrations)
Depending on which flyway version you are using, one option is to go to the flyway_schema_history or schema_version table and just locate the row (corresponding to the failed script) and change the success column to true or 1. Then manually run the script (the correct one) against DB.
Sqitch has the ability to run Revert and Verification scripts. Does or are their plans for Flyway to support such abilities?
I'm unfamiliar with the capabilities of Sqitch, but I can briefly explain what's possible with Flyway.
Flyway does have support for revert scripts, via the Undo command. The process works by allowing you to specify an 'Undo migration' alongside a standard migration script. The Undo migration knows how to revert the changes specified in the standard migration script. Note that this is a Pro feature only.
Undo migrations do not work for Repeatable Migrations, which are used mainly for Views, procedures, and so on. These should just be amended and then migrated normally.
Flyway also allows you to verify your migrations to a certain extent with the Validate command. Validate checks that the migrations applied to the database match the migrations in your project.
From briefly looking through the Sqitch docs, it looks like their verification feature allows you to specify a script for each migration to see if it makes sense to run it. Flyway doesn't natively support such a process. However, since the migration scripts are just plain SQL, nothing stops you from manually including such verification logic in the migration scripts themselves.
As a potential alternative, you could use the Dry Run feature, which produces the script which will ultimately be used to migrate the database. You could execute the Dry Run script against a test environment to see if the deployment would succeed. Dry Run is also a Pro feature.
I hope that helps.
Thanks
Mikiel