I have some flyway migrations that need DB superuser privilege to get executed. I want to avoid running the flyway migrations as a superuser and run the app say a different less privileged user (I can not run the app as a superuser).
Is there any good way to achieve this with flyway, i.e. have different migrations run as different users?
You could have two Flyway migrations runs (regular & superuser), however this could then set you up for trouble as you would have to be very careful with their interdependencies.
The best solution is the one you already mentioned: configure Flyway with a user that has the necessary privileges. Simple and easy.
Related
Our company has about 30+ applications written in different languages (java, c#, visual basic, nodejs etc)
Our aim is to have development teams keep the database change sqls in their repositories, and do the migration from Jenkins with them starting pipelines with version number. Development teams don't have access to Jenkins configuration, they can only run jobs that we created and configured.
How should we go about this? Do we have too keep different flyway instances for each application? And how about pre-production and production stages?
Basically, how should we, as devops team, maintain flyway to do migration of different applications with different stages, without the development teams doing the migration part.
This should be possible with the Flyway CLI. You can tell Flywyay where to look for migrations and how to connect to the database. See the docs about configuring the CLI.
You can configure Flyway in various ways - environment variables, command line arguments, and config files.
What you could do is allow each development team to specify a migrations directory and connection details for the Jenkins task. The task can then call the Flyway CLI, overriding the relevant config items via command line arguments. For example, the command line call to migrate a database:
flyway url=jdbc:oracle:thin:#//<host>:<port>/<service> -locations=some-location migrate
Or you could allow your devs to specify environment variables, or provide a custom config file.
You can reuse a single Flyway instance since the commands are essentially stateless. The only bit of environmental state they need comes from the config file, which you have complete control over.
Hope that helps
Is it possible lock the database from any other connections when running the migrations through Database.Migrate()?
We have multiple service instances running the same code (on AWS Lambda), and do the migrations on startup. Now we have to manually make sure that only one instance is running when we want to apply some migrations, otherwise they can both try to do it and break things bad.
Is there a database level solution to this?
ef-core 2.1
Not really sure if this is what you are looking for, but if you are willing to add plain SQL to your migration you could set database to single user mode: Read more
I'd like to be able to deploy to my database after a deployment failure, because in my case I am dealing with concurrent and independent deployments.
the flyway clean command helps to remove failed migrations in the migration table. Using that command helps as my future deployments will be executed but the downside is that I lose the history of failed deployments.
Is there a way to tell flyway to repair the migration table and keep the history of failed deployments - say put the success flag to '2' for example.
In my case all deployments are independents so we create unique migration numbers at deployment time - this allows us to redeploy anything without consideration of flyway's status (we re only using the automation mechanism provided with flyway and have disabled other 'migration' safeguards).
This isn't supported natively. I agree that removing the history of failed migrations would be handy to have as an option.
Use of the callbacks may give you a workaround. Specifically you could take a copy of failed migrations in the beforeRepair callback. If using SQL files make a beforeRepair.sql (or whatever you have for your migration suffix).
Sqitch has the ability to run Revert and Verification scripts. Does or are their plans for Flyway to support such abilities?
I'm unfamiliar with the capabilities of Sqitch, but I can briefly explain what's possible with Flyway.
Flyway does have support for revert scripts, via the Undo command. The process works by allowing you to specify an 'Undo migration' alongside a standard migration script. The Undo migration knows how to revert the changes specified in the standard migration script. Note that this is a Pro feature only.
Undo migrations do not work for Repeatable Migrations, which are used mainly for Views, procedures, and so on. These should just be amended and then migrated normally.
Flyway also allows you to verify your migrations to a certain extent with the Validate command. Validate checks that the migrations applied to the database match the migrations in your project.
From briefly looking through the Sqitch docs, it looks like their verification feature allows you to specify a script for each migration to see if it makes sense to run it. Flyway doesn't natively support such a process. However, since the migration scripts are just plain SQL, nothing stops you from manually including such verification logic in the migration scripts themselves.
As a potential alternative, you could use the Dry Run feature, which produces the script which will ultimately be used to migrate the database. You could execute the Dry Run script against a test environment to see if the deployment would succeed. Dry Run is also a Pro feature.
I hope that helps.
Thanks
Mikiel
My current project has a few Flyway migrations in place that are used to import initial data into a database. This data is convenient especially for developers to be able to quickly setup the project. Production data is imported through some batch jobs and has a newer version.
Some of these migrations are quite big (~20MB) and so everytime the application starts, Flyway takes some time to calculate the checksum of the migrations. This also is a problem for integration tests as they also take longer because of this.
I consider this approach to be a misuse of Flyway, I think migration tools should be used mainly for structural data.
I want to remove those files from our application and rather use a configuration management tool (e.g. Vagrant, Puppet, Chef) to import test data on developer environments. However, I can't just delete the migration files from the application as Flyway will complain that a migration has been recorded in the database but is not present in the application migrations.
My first thinking was to create a new migration with a high-priority version number that will
Delete the test data
Delete the migration from the schema_version table
and then remove the migration scripts. This however does not work, Flyway still complains that the removed migration script is missing.
Is there a restriction that you cannot interact with the schema_version table in migrations?
What other options do I have? If at all possible I want to do this using Flyway and not manually.
Repair is your best bet. Empty those data migrations and run the repair command to have their checksums recalculated based on the empty files.