I am new to Flyway and I am using Flyway 2.1 code base for migration and execution of SQL statements on a previous schema version using Flyway. Once i execute my newer SQL statements the version entry in the metadata table is incremented.
But before I migrate my new version of the schema (i.e before I migrate/ execute the newer SQL statements on the database schema), I would like to capture all the new SQLs in another preview.sql file, so that the DBAs can see the SQLs before they proceed with the migrate.
I plan to do this by adding a flyway.preview() method to the Flyway.java file. Could you please let me know what other files would need to be changed to accomplish this?
Also, I only want to do this if the new version is > current schema version currently in the database. I checked the 2.1 code but the SchemaVersion class has been deprecated in 2.1 and I am not sure how to obtain the current version from the database.
I would appreciate your help or any suggestions that would correct my approach.
Thank you
The necessary abstractions are not in place yet to support this, but I plan on adding them over the next few weeks to support batch updates in 2.2. Once that is done, it should be much easier to implement this.
As for querying the state of the DB, you have the Flyway.info() to assist you.
Related
I'm investigating how Flyaway works to see if it's suitable for our upgrade and migration needs.
Can somebody tell me if Flyaway automatically handles precedence so that for example, if a table has a trigger, it creates the table before attempting to create the trigger, or is the user responsible to manually creating and ordering the actual SQL Statements required in each schema update?
The latter - Flyway does not keep knowledge of the dependencies between database objects (as it's a very large and gnarly problem, and it's highly db-dependent). Which database do you use? A tool like Redgate Schema Compare will generate the script with all the correct dependency orderings for you.
We would like to use the Edition Based Redifinition feature with Oracle.
Does anybody have experience creating updates with liquibase using this feature from Oracle?
I've checked in another forum https://forum.liquibase.org/topic/how-to-do-an-alter-session and the test they did, it was not solved.
Did somebody solve this and manage to use this feature from Oracle within liquibase?
Finally we managed to solve it with liquibase.
We're executing a sqlFile changeset that executes ALTER SESSION SET EDITION = and aftewards the changes we want to do.
It looks like the connection is properly changed and that we're able to do the changes in the proper version.
I have delivered a Product to the customer. Now I have upgraded the Product, which includes changes to the database.
Customer wants to upgrade the Product. Now will Flyway help in the migration of Customer data from older version to newer version. Please let me know, if this is a valid use case. The flyway documentation talks about its use during development only.
Flyway allows you to change your database by running a set of scripts in a defined order. These scripts are called 'migrations' as they allow you to 'migrate' your database from one version to another.
The idea is you can start with an an empty database and each migration script will successively bring that database up from empty up to the current version. However, it's also possible to start with an existing database by creating a 'baseline' migration.
As SudhirR said, Flyway's primary use case is to define schema changes. However, it's perfectly possible to change data also. Since Flyway is just running plain SQL, in principle almost anything you can do in a SQL script you can also do in a Flyway migration.
In the case you described it should be possible to use Flyway to migrate the customer database. The steps you could take are:
Generate a sql script that includes the entire DDL (including indexes, triggers, procedures, ...) of the production database. To do this you will need to add insert statements for all the reference data present in the database.
Save this script in your Flyway project as something like 'V1__base_version.sql'
Run the flyway baseline command against your production database
This will set up your production database for use with Flyway
Add a new migration script to migrate your customer's data to the new version
e.g. create new table, copy data from old table to new table, delete old table
Run flyway migrate to upgrade production
These steps are adapted from the Flyway documentation page here.
Of course you should read the Flyway docs and manually test on a throwaway DB before you run anything against production. However I think in principle Flyway could be a good fit for your use case.
Flyway should be used for schema migrations and any reference data (basic data that is required by the system/application in order to function properly).
Putting client specific data migrations would not be a use case. However, if you can represent the data migration "generically" by not using IDs and instead use names or types than it could be a candidate. Meaning if you could write a migration in a way that could be applied to all clients, then that would be the use case to put it in as a flyway migration.
Otherwise data migrations would be applied in some other way outside of the process like requesting special access to the database or having some team that manages the database to apply the scripts.
If you are doing custom data modifications quite often then I'd say something is wrong in some other area of the SDLC and you may need to increase testing so that bugs don't mess up the data in the first place.
We have a customer with a IT-department that insist on getting database updates as scripts prior to upgrades (they want to "read" the updates before implementing them).
Being an avid user of flyway it made me think about a way of generating a script from flyway based on updates and send this script to IT. This script would at the very least need to create version table if non-existant, check version and apply needed updates. It would of course be proprietary to database vendor (in my case Oracle).
This would allow us to run updates with flyway automatically in development environments and create manual scripts for test and production.
Are anyone aware of something like that having been contemplated or endeavored before? Would it be trivial or a momentous task?
We had this exact problem when I worked at a consultancy (Intelliware) so the devs there put together some code and pushed it up to GitHub.
We tried unsuccessfully to get it included into the Flyway core repo.
https://github.com/Intelliware/flyway-script-generator
I looked at the Flyway samples and documentation and tried to understand if it is useful in my environment.
The following conceptual detail is unclear to me: How does Flyway manage the changes between database versions? It obviously does NOT compare database life-instances (see answer here:Can Flyway find out and generate migration files from datamodel?)
In detail my setup looks like this:
I create SQL create and insert scripts when coding (automatically and manually). This means every version of my database is represented by a number of insert/create statements.
In my world I execute these scripts through a database tool (sqlplus from Oracle). Each run would setup the database _from_scratch_ (!).
Can I put these very same scripts 1 to 1 inside the "migration" path of Flyway? What happens if the target database is way older than the last "migration step" I did (or flyway did not yet exist when it was installed)?
Update:
I got some input from another Flyway user:
It seems like each "migration" (version of the database) has to be hand-written SQL/Java code and contains only "updates" from the previous "migration" of database.
If this is true, I wonder how this can be used with traditional coding technics: in my world SQL statements are generated automatically and contain all database init/create statements, not just "updates" to some previous version. If my SQL code generator could do that, then I wouldn't even need a tool like Flyway :-).
Your question about "how to handle a DB that has a longer history than there are migration scripts?" You need to create a V1_ migration/sql script that matches/recreates your latest DB schema. Something that can take a blank DB to what you have today. Create/generate that sql script using your existing DB tools and then put it in flyways migration directory. (And test V1 by using flyway against a clean DB and see if you get what you expect.) http://flywaydb.org/documentation/existing.html
After that point in time, all later versions must be added in as you work. When you decide you need a new table, in your dev environment, write a new V*_.sql that modifies your schema to the way you need it.
This blog goes over this situation for a Spring/SQL application. https://blog.synyx.de/2012/10/database-migration-using-flyway-and-spring-and-existing-data/