Our common workflow when creating a new sql migration script is
to write and execute every single statement in the developers local datatase schema. When finished, it's checked into the source control system.
Problem is: at the database scheme of the creating developer, the script is already "executed". For scripts not beeing reentrant - it would be convenient to have s.th. like Dbmaintain's maven task "markDatabaseAsUpToDatemaven".
Does Flyway have s.th. equivalent?
P.S.: Our current workflow (as a workaround) is as follows:
"mvn flyway:migrate" this file as an empty file (without content - so it never fails).
put the sql statments in, save & "migrate" again.
"mvn flyway:repair"
Thanks
While the workflow you describe sounds like it can do the job, you can achieve the same in a simpler and fully automated way: set cleanOnValidationError to true (on for dev!) and everytime the script changes, the DB gets recreated.
More info: http://flywaydb.org/documentation/maven/migrate.html#cleanOnValidationError
Related
So while building a new database using our database migration scripts written in a springboot flyway project, we realized we made some mistakes.
Some old scripts need to be changed to ensure that we do not face these issues when we make a new database schema again. These issues are mostly related - an info table was not populated with entries in the project and there are scripts that refer to the data in the migration project -- this data does not exist because we never included a script to include data.
How can we correct this project - the only way I can think of is to correct scripts such that all inserts are replaced by - insert if not exists or replace create statements by create if not exists.
and then delete all entries in schema version and re-run this on all the database which are using this schema.
I cannot go back and correct my script because then the migration project will fail because of checksum issues.
You are rigth, if this project and the scripts are running in some existing projects you can not modify them because the checksum would fail.
Then the cleanest way I can think would be add a file called "DB-GENERAL-FIXES" or something like that, where you can add all SQL validations to restore the DB to a stable status. For the new implementations will be extra work first build it wrongly and then clean it, but if you are sharing the same code in production right now...is the best option
I looked at the Flyway samples and documentation and tried to understand if it is useful in my environment.
The following conceptual detail is unclear to me: How does Flyway manage the changes between database versions? It obviously does NOT compare database life-instances (see answer here:Can Flyway find out and generate migration files from datamodel?)
In detail my setup looks like this:
I create SQL create and insert scripts when coding (automatically and manually). This means every version of my database is represented by a number of insert/create statements.
In my world I execute these scripts through a database tool (sqlplus from Oracle). Each run would setup the database _from_scratch_ (!).
Can I put these very same scripts 1 to 1 inside the "migration" path of Flyway? What happens if the target database is way older than the last "migration step" I did (or flyway did not yet exist when it was installed)?
Update:
I got some input from another Flyway user:
It seems like each "migration" (version of the database) has to be hand-written SQL/Java code and contains only "updates" from the previous "migration" of database.
If this is true, I wonder how this can be used with traditional coding technics: in my world SQL statements are generated automatically and contain all database init/create statements, not just "updates" to some previous version. If my SQL code generator could do that, then I wouldn't even need a tool like Flyway :-).
Your question about "how to handle a DB that has a longer history than there are migration scripts?" You need to create a V1_ migration/sql script that matches/recreates your latest DB schema. Something that can take a blank DB to what you have today. Create/generate that sql script using your existing DB tools and then put it in flyways migration directory. (And test V1 by using flyway against a clean DB and see if you get what you expect.) http://flywaydb.org/documentation/existing.html
After that point in time, all later versions must be added in as you work. When you decide you need a new table, in your dev environment, write a new V*_.sql that modifies your schema to the way you need it.
This blog goes over this situation for a Spring/SQL application. https://blog.synyx.de/2012/10/database-migration-using-flyway-and-spring-and-existing-data/
I use Scrum methodology and deploy functionality in builds every sprint.
There is necessity to perform different changes in the stored data (I mean data in database and on filesystem). I'd like to implement it as a PHP scripts invoked from console. But they should be executed only once, during the deployment.
Is there any way to implement it through app/console without listing it in the list of registered Console commands? Or is there any other way to implement runonce scripts?
DoctrineMigrations covers some part of my requirements, but it's hard to implement complex changes in Model. And it does not cover changes in files on the filesystem.
I don't think symfony has a facility for that, and besides, hiding the command is not the same as securing the command.
Instead, I would make the script determine if it has been run already (could be as simple as adding a version number to a file and checking that number before running) and stop if it detects it has already run before.
I've got a state machine workflow implemented using WorkflowFoundation 4.1. I'm using the SQL Server persistence store and the WorkflowApplication class for loading and running workflows.
I'm making changes to the state machine workflow model, and am finding that existing instances break very easily. I've written code that can replay the workflow back into the correct state, which is basically a migration, which works fine, however I need to be able to clear out the old workflow instance as well.
The main issue is that if the workflow is invalid, I can't even load it, so I can't terminate or cancel it either.
Is there a way to use the workflow API to remove a workflow without loading it (ie, some command on the SqlPersistenceStore), or do I have to clean the database manually?
The SqlWorkflowInstanceStore doesn't allow you to do so directly. You will need to go into the database and delete the record there. If you are using AppFabric there is actually a command to delete a workflow instance without loading it first for just that purpose. There should be a PowerShell command to do that using code.
I have an ASP.NET web application that includes code for enforcing its own database schema ; this code runs on application start.
I've recently started using LINQ to SQL, and I've added a pre-build event to run SqlMetal on my database so that I get objects representing my db tables.
What would be really cool is if I could enforce the database schema in the pre-build event, and then run SqlMetal. As it is, if the schema changes (e.g. I add a field to a table), I have to (a) build and run the website once so that application start fires and the schema is enforced, and then (b) build it again so that SqlMetal runs.
So: What are my options for running code that lives in my web application, from the command line?
Here's what we do.
We have a local 1-click build that is required to be run before check in (an integration build also runs in a separate environment every check in...).
The NANT script will:
Rebuild the database from scratch using Tarantino (Database change management)
Clean & Compile
Copy DLLs to a separate directory
Run Unit Tests against the DLLs
We have a separate script for SQL Metal, but your question is going to have me look at inserting the call between steps 1 and 2. This way your database changes and linq generated files are always in sync.
You could either write a small program that use CodeDOM to interpret a file in your repository or directly call the compiler executable inside your pre-build event.
Using CodeDOM avoid any problems with having to know where the compiler executable is but if your code can't be contained in one file without any dependency it's unusable and calling the compiler, then executing the result is a better option.