Is it possible lock the database from any other connections when running the migrations through Database.Migrate()?
We have multiple service instances running the same code (on AWS Lambda), and do the migrations on startup. Now we have to manually make sure that only one instance is running when we want to apply some migrations, otherwise they can both try to do it and break things bad.
Is there a database level solution to this?
ef-core 2.1
Not really sure if this is what you are looking for, but if you are willing to add plain SQL to your migration you could set database to single user mode: Read more
Related
We have a production system with a large DB (several hundred tables) and would like to begin using Flyway to manage DDL changes that occur through the dev cycle. However, the organization is setup in such a way that there are production DB changes that sometimes occur, mostly just data changes but possibly DDL, that will happen outside of a data migration tool. While this is obviously an organizational challenge, does this fact alone cripple a tool like Flyway? Or is there a workflow where Flyway could rebuild its indices on demand such that any out-of-band DB change like this is pulled in?
We'd love to use Flyway, but would need to integrate it incrementally until all teams using the system are trained/bought in.
When introducing Flyway to a DB with existing data you will need to baseline Flyway to integrate with your existing data. See baseline.
For changes made after this, Flyway will only track and version changes made from its own migration scripts and not changes made externally to it. However, this does not mean you cannot use the two together, but you would need to be more aware of your database structure to avoid conflicts between your flyway migrations and external changes.
Transactional data changes made to production shouldn't impact Flyway as these won't be versioned.
If you're referring to static data (eg lookup data) that you'd like Flyway to manage, then this isn't detected by Flyway (at least not today). If you discover that you have drift you'll need to add the changes as a new migration script using idempotent syntax to ensure that next time this runs against production it doesn't try to make the same changes again.
For out-of-band schema changes, The enterprise edition of Flyway has a drift check, so at least you'd be made aware of them. However, as for the data changes described above, you'll need to manually add these schema changes as an idempotent migration script.
My current project has a few Flyway migrations in place that are used to import initial data into a database. This data is convenient especially for developers to be able to quickly setup the project. Production data is imported through some batch jobs and has a newer version.
Some of these migrations are quite big (~20MB) and so everytime the application starts, Flyway takes some time to calculate the checksum of the migrations. This also is a problem for integration tests as they also take longer because of this.
I consider this approach to be a misuse of Flyway, I think migration tools should be used mainly for structural data.
I want to remove those files from our application and rather use a configuration management tool (e.g. Vagrant, Puppet, Chef) to import test data on developer environments. However, I can't just delete the migration files from the application as Flyway will complain that a migration has been recorded in the database but is not present in the application migrations.
My first thinking was to create a new migration with a high-priority version number that will
Delete the test data
Delete the migration from the schema_version table
and then remove the migration scripts. This however does not work, Flyway still complains that the removed migration script is missing.
Is there a restriction that you cannot interact with the schema_version table in migrations?
What other options do I have? If at all possible I want to do this using Flyway and not manually.
Repair is your best bet. Empty those data migrations and run the repair command to have their checksums recalculated based on the empty files.
I have some flyway migrations that need DB superuser privilege to get executed. I want to avoid running the flyway migrations as a superuser and run the app say a different less privileged user (I can not run the app as a superuser).
Is there any good way to achieve this with flyway, i.e. have different migrations run as different users?
You could have two Flyway migrations runs (regular & superuser), however this could then set you up for trouble as you would have to be very careful with their interdependencies.
The best solution is the one you already mentioned: configure Flyway with a user that has the necessary privileges. Simple and easy.
I have a website solution that uses on SQLite database for each tenant account. Without going into much depth about why we chose this solution, we chose it due to SQLite support on distributed/offline systems.
All databases are manipulated using the same PHP file structure. I wish to update the database version iteratively for all accounts so that they are all at the same version number.
I have a script that loops over each, and can use either PHP(Yii) or the shell to execute queries.
I would like to wrap the manipulation to each database in a transaction. It appears as though DDL commands may already be auto-commit in nature.
Question: How to accomplish a SQLite DB upgrade which, if it fails, will report a failure? Using that report, I could prompt the system to re-attempt or report an error to an admin. Ideally, I would like to wrap the whole upgrade in a transaction to prevent inconsistencies, but I'm fairly certain that this is not possible.
One thought I had was to backup the database temporarily during upgrade, and delete it on success.
As CL stated in the comments, DDL is fully transactional in SQLite. So, for each database in our system, we wrap it in a transaction, and it executes atomically. We leverage the application to ensure that all databases upgrade successfully if any fail during upgrade.
IMHO using maven for migrations is fine only for development machines.
On servers you don't usually have maven available (and it might be impossible to have it installed there).
So: How do I init a database without maven?
Do I just call flyway.init()?
What if the db is already init-ed?
Can I execute sql statements to init the db?
My foreign keys and indexes are different/messed up in between different databases instances, so I already made a complete schema script and tested it with data export, schema drop, schema re-create and data restore. I am going to do that on all databases to ensure that they are exactly the same.
Yes, you can simply call flyway.init()
You can use flyway.status() to check if the DB has been inited.
This process will become easier with Flyway 1.8, where a new property called initOnMigrate has been introduced. The first time it runs, it will then init an existing non-empty database (PROD) when you run migrate or just migrate on an empty one (DEV).