So just exploring the possibility of using flyway to maintain my DDL statements against Amazon Athena using Athena provided JDBC driver, Athena supports only CREATE statements (hive DDL) and no INSERTS.
So if database metadata table is the only one that flyway creates and updates, Is there anyway I can externalize the creating, storing into a totally different database ?
Currently this is not possible in flyway, as the schema history is read from/written to the database defined by the current jdbc connection. You can see this for yourself in the JDBCTableSchemaHistory file.
If you wish to add this support, you could create a pull request on the repo, or just add an issue detailing exactly what you want the behaviour to be.
Related
I have existing database postgres which not using Flyway and i need replicate it.
How to move existing database state to new empty database?
I don't have any migration sql.
So I am expecting command like generateChangelog in Liquibase but it seems in Flyway not existing command like that.
Flyway currently only manages the scripts that you create. It doesn't create scripts for you. So, in order to take an existing database and get it into Flyway processing, you'll need to generate the scripts for that database. You can use the methods outlined here to get the scripts for your database. Then, just rename them to the Flyway standard. You'll be off and running.
I have delivered a Product to the customer. Now I have upgraded the Product, which includes changes to the database.
Customer wants to upgrade the Product. Now will Flyway help in the migration of Customer data from older version to newer version. Please let me know, if this is a valid use case. The flyway documentation talks about its use during development only.
Flyway allows you to change your database by running a set of scripts in a defined order. These scripts are called 'migrations' as they allow you to 'migrate' your database from one version to another.
The idea is you can start with an an empty database and each migration script will successively bring that database up from empty up to the current version. However, it's also possible to start with an existing database by creating a 'baseline' migration.
As SudhirR said, Flyway's primary use case is to define schema changes. However, it's perfectly possible to change data also. Since Flyway is just running plain SQL, in principle almost anything you can do in a SQL script you can also do in a Flyway migration.
In the case you described it should be possible to use Flyway to migrate the customer database. The steps you could take are:
Generate a sql script that includes the entire DDL (including indexes, triggers, procedures, ...) of the production database. To do this you will need to add insert statements for all the reference data present in the database.
Save this script in your Flyway project as something like 'V1__base_version.sql'
Run the flyway baseline command against your production database
This will set up your production database for use with Flyway
Add a new migration script to migrate your customer's data to the new version
e.g. create new table, copy data from old table to new table, delete old table
Run flyway migrate to upgrade production
These steps are adapted from the Flyway documentation page here.
Of course you should read the Flyway docs and manually test on a throwaway DB before you run anything against production. However I think in principle Flyway could be a good fit for your use case.
Flyway should be used for schema migrations and any reference data (basic data that is required by the system/application in order to function properly).
Putting client specific data migrations would not be a use case. However, if you can represent the data migration "generically" by not using IDs and instead use names or types than it could be a candidate. Meaning if you could write a migration in a way that could be applied to all clients, then that would be the use case to put it in as a flyway migration.
Otherwise data migrations would be applied in some other way outside of the process like requesting special access to the database or having some team that manages the database to apply the scripts.
If you are doing custom data modifications quite often then I'd say something is wrong in some other area of the SDLC and you may need to increase testing so that bugs don't mess up the data in the first place.
In which way can I access another database (not OpenEdge) via ODBC from OpenEdge without using DataDirect?
The use case is data migration from one system to another, so performance cannot be neglected completely but it's a one time thing that is allowed to take a little longer.
Why without DataDirect? Extra cost. Our client doesn't have the license.
Why not dump and load (via CSV f.e.)? The client doesn't want to do the mapping between the systems this way but with database views.
As far as I know there is no way to directly access other database if you're not using DataDirect or something like DataServer for Oracle etc.
However, you could call a third party ODBC library as external functions, and write your handle your queries to the foreign database by accessing. This wouldn't allow you to use OpenEdge constructs like FOR EACH, buffers etc, but it would allow you to retrieve the data and process it using custom functions, and then insert into the OpenEdge tables etc.
See the following KB for accessing external library functions:
https://knowledgebase.progress.com/articles/Article/P183546
Another approach you could use, assuming your tables are in OpenEdge already, is to use the OpenEdge SQL92 ODBC driver from another language (C/VB/Java/whatever works for you), and read the data from the source database and insert into OpenEdge via SQL92 ODBC.
Looking at the website there are downloadable ODBC drivers for most platforms:
https://www.progress.com/odbc/openedge
We use a process to migrate our ETL changes, but not the associated SQL code that often need to be applied to both the source and target DBs. Is there a way for us to specify which SQL files get applied to which DB?
Use one Flyway instance per DB and set a different location to load the sql files from for each DB. You can also specify a common one for both.
I need to export the data of a SQLite database and import this data to another SQLite database. But the two databases have different schemas.
Does exist an open source tool that can help me doing that job?
The only opensource tool that i know is opendbcopy that i'm using for migrate from a database server to another and also for a similar kind of job that you want to do with SQLite but i've done it with PostgreSQL.
However opendbcopy is JDBC compliant and can connect to every database that have a JDBC driver, so you can try, also if the schema is not the same you can use the column mapping feature :
In addition i know also a good commercial alternative (that is easier to use) that is ESF Database Migration Toolkit .