I changed a flyway script for error, and this brought my migrations to an error state (I have 5 script versions, but when I run my app it starts from the 4th, and get an error on 'reaction already exists').
I tried to use clean from flyway cli, but it didn't completely solve the problem... also, when I try pushing my branch to git, CI/CD pipeline will fail for the migration.
Since I'm in development environment, would it be a terrible idea to all delete migration scripts?
Would deleting all scripts allow me to 'start from scratch' in my development environment, or do I need to push the whole project again to avoid issues? (Project is not in production yet, I don't really need flyway migrations)
Depending on which flyway version you are using, one option is to go to the flyway_schema_history or schema_version table and just locate the row (corresponding to the failed script) and change the success column to true or 1. Then manually run the script (the correct one) against DB.
Related
I'd like to be able to deploy to my database after a deployment failure, because in my case I am dealing with concurrent and independent deployments.
the flyway clean command helps to remove failed migrations in the migration table. Using that command helps as my future deployments will be executed but the downside is that I lose the history of failed deployments.
Is there a way to tell flyway to repair the migration table and keep the history of failed deployments - say put the success flag to '2' for example.
In my case all deployments are independents so we create unique migration numbers at deployment time - this allows us to redeploy anything without consideration of flyway's status (we re only using the automation mechanism provided with flyway and have disabled other 'migration' safeguards).
This isn't supported natively. I agree that removing the history of failed migrations would be handy to have as an option.
Use of the callbacks may give you a workaround. Specifically you could take a copy of failed migrations in the beforeRepair callback. If using SQL files make a beforeRepair.sql (or whatever you have for your migration suffix).
So while building a new database using our database migration scripts written in a springboot flyway project, we realized we made some mistakes.
Some old scripts need to be changed to ensure that we do not face these issues when we make a new database schema again. These issues are mostly related - an info table was not populated with entries in the project and there are scripts that refer to the data in the migration project -- this data does not exist because we never included a script to include data.
How can we correct this project - the only way I can think of is to correct scripts such that all inserts are replaced by - insert if not exists or replace create statements by create if not exists.
and then delete all entries in schema version and re-run this on all the database which are using this schema.
I cannot go back and correct my script because then the migration project will fail because of checksum issues.
You are rigth, if this project and the scripts are running in some existing projects you can not modify them because the checksum would fail.
Then the cleanest way I can think would be add a file called "DB-GENERAL-FIXES" or something like that, where you can add all SQL validations to restore the DB to a stable status. For the new implementations will be extra work first build it wrongly and then clean it, but if you are sharing the same code in production right now...is the best option
My current project has a few Flyway migrations in place that are used to import initial data into a database. This data is convenient especially for developers to be able to quickly setup the project. Production data is imported through some batch jobs and has a newer version.
Some of these migrations are quite big (~20MB) and so everytime the application starts, Flyway takes some time to calculate the checksum of the migrations. This also is a problem for integration tests as they also take longer because of this.
I consider this approach to be a misuse of Flyway, I think migration tools should be used mainly for structural data.
I want to remove those files from our application and rather use a configuration management tool (e.g. Vagrant, Puppet, Chef) to import test data on developer environments. However, I can't just delete the migration files from the application as Flyway will complain that a migration has been recorded in the database but is not present in the application migrations.
My first thinking was to create a new migration with a high-priority version number that will
Delete the test data
Delete the migration from the schema_version table
and then remove the migration scripts. This however does not work, Flyway still complains that the removed migration script is missing.
Is there a restriction that you cannot interact with the schema_version table in migrations?
What other options do I have? If at all possible I want to do this using Flyway and not manually.
Repair is your best bet. Empty those data migrations and run the repair command to have their checksums recalculated based on the empty files.
Currently I package my changes using the publish wizard in Visual Studio and take those files to the server and run the deploy batch file in command line.
What is the best strategy to use if I have issues in the deployment and I want to revert back my website/service to the last working state?
Since your deployment is a manual process, it would probably be best to create a backup zip or copy of the webroot - this could be added as a command in your batch file before the other commands.
Alternatively, you could create a label in your source control system each time you deploy which would give you the ability to re-deploy a previous version. I would only advise this approach though if you have some sort of CI process in place to guarantee that the label is done prior to each deployment/release, otherwise your process is dependent on the person deploying remembering to create that label - if they forget, then you can't restore that previous version.
OUR SYSTEM
We are trying to put migrations as .sql files under version control. Developers would write a VN__*.sql file, commit to version control and a job that runs every 5 minutes would automatically migrate to a Dev and Test database. Once the change proved to not cause problems, someone else would run a manual job to run the migration on Production.
MY PROBLEM:
I had a demo migration that created a few tables. I checked V4__DemoTables.sql into version control on my PC.
On our Linux box a job that runs every 5 minutes extracted the new file from version control, then ran the flyway.sh file. It detected the file and executed it.
But the .sql file had a typo. And we are using Neteeza which has problems with flyway automatically wrapping a migration in a BEGIN TRAN ... END TRAN. So the migration created 2 tables, then aborted before the third.
No problem I thought. I dropped the 2 tables that the .sql file did create. Checked V4__ out of version control, fixed the typo and re-submitted it.
Five minutes later the update was extracted but flyway complains that the checksum does not match. So it will NOT run the updated V4__DemoTables.sql file.
How do I get flyway to accept the updated file and update the checksum in the SCHEMA_VERSION file in case of a typo?
Reading the docs it seems like the developers suggest I should have created a new V4_1_DemoTables.sql file with the fix's. But this would have collided with the commands in the V4__ file so this seemed wrong.
So here is what the docs imply I need to do:
Leave V4__ as a 'successful' migration according to the
SCHEMA_VERSION table.
Create V4_1_ to delete the tables that were created before the typo
line in V4__.
Create V4_2_ which has the typo fix from the original file to do all
the real work.
Is this correct?
If the migration completes successfully, but some of the db objects are not quite right yet (typo in column name, ...), do as you said and push a follow-up script that fixes it (rename column, ...).
If the migration fails, and it didn't run on a DB with DDL transaction, the DB must be cleaned manually. This means:
Reverting the effects of the migration on the DB
Removing the version from the SCHEMA_VERSION table and marking the previous one as current
This second step will be automated in the future with the introduction of the flyway.repair() command.