We are in a situation similar to this blog post from 2013.
Because there is one project that will be used by many developers, we'd like to implement the OutOfOrder functionality from Flyway. The above mentioned post however makes a claim that the SQL will have to be written using idempotent commands, such as "IF NOT EXISTS" etc.
When attempting to reproduce the issue from the post, we noticed it did not matter as OutOfOrder would apply the earlier SQL file without attempting already applied migrations.
Was this fixed since the original post? Is there anything I'm missing?
OutOfOrder only applies unapplied versions that are made after the baseline of the database.
If the database wasn't baselined to start, then it applies all versions that have not been applied,
Flyway doesn't apply any that have already been applied.
I believe that the article was referring to manually reapplying the file.
Related
We have a database which stores, among others, identifiers from an external system. Now the identifiers changed (the system changed the scheme), and the database needs to be updated. It can be done - I have a mapping so I can generate just enough SQL to make it work, and in the end this will need to be done like this.
The question is - is this use case for a Flyway Java migration? I tend to think that it's not the case, but I can't really say why, it's a gut feeling. But, the external system's schema is not versioned, at least not by us, so I feel it doesn't fit into out Flyway migrations at all; I think it should be executed just once, outside of Flyway.
Can anybody with more experience maybe help, explain why or why not?
It's mostly opinion based, but it seems to me as it's just the same as to use a steam-hammer to crack nuts. Flyway is a very useful tool for periodical migrations and for cases, then there is a number of databases, you have to recreate from the scratch or update'em regularly, rather then for a single use.
What is the reason to include some relatively large framework in your project, spend some time to make it work, and use it only once? Furthermore, Flyway need some extra table to exist in your DB to store it's inner info about the the current version and applied migrations. Don't think, that is the thing you really want to have in your case.
As for me, I think that if you have to do this task just once and can do it without Flyway, then just do it this way.
I think one question we should be asking ourselves when we determine whether or not to write a Flyway script for our data migrations is "Is this necessary when creating this db from scratch?"
Flyway uses a versioning system so in your case, would it make sense to flip the values from the old version to the new version when standing up a new environment? What about multiple modifications? Does it make sense to store the old values and apply them sequentially if you are creating a new environment?
If you answer "NO", then flyway is probably not the way to go. Flyway is better utilized for schema changes where the structure of the database is changed and data is converted into the new structure. If you're just changing configuration values, I believe flyway is probably not your best bet, simply because it is not necessary to store all the changes to these configuration values.
I have been reading a blog post about Flyway, called Lessons Learned Using Flyway DB with Distributed Version Control. One of the author's suggestions is to create idempotent migrations.
Quoting from the article:
In a perfect world, each migration will only be run once against each
database.
In a perfect world, that is.
In actuality, there will be cases where you’ll need to re-run
migrations against the same database. Often this will be due to a
failed migration somewhere along the line, causing you to have to
retrace your steps of successful migrations before to get the database
back in a working state. When this happens, it’s incredibly helpful
for the migration to be written in an idempotent manner.
Assuming I am using a database that supports DDL transactions, should I be worried about idempotency while creating these migration sqls?
In general no, especially when you have a database that supports DDL transactions.
Versioned migrations are designed to run exactly once and can, but don't have to be idempotent (virtually no benefit).
Repeatable migrations on the other hand have to be idempotent as they'll be run over and over again.
Flyway let's you very easily recreate your database from scratch, and that's the approach you should favor when experimenting.
I have one application using a single database schema.
Nonetheless, the application has a core (having its DB objects) and can be extended with a plugin logic (each plugin having its DB objects).
Core DB objects and Plugins DB objects are distinct sets, since plugins are optional and may exist or may not.
Thus I need separate versionig and migration control for Core and each single plugin.
I'm wondering if there is some way, using Flyway, to manage this separate "migration paths".
The only thing I can think about is creating, under the same, single DB schema hosting the application, many different Flyway metadata tables (like schema_version_core, schema_version_plugin1, etc.) and managing migrations of each component independently.
Is this doable? Any smarter suggestion?
Many thanks
I highly recommend you split your DB into schemas, as this is really what they are for: managing sets of objects.
If that is not an option, the alternative you suggested works fine. Just be careful when invoking clean, as it will then clean the entire schema not just the part for one of your plugins.
I am currently struggling with the same problem: An application which is made of several "base" components which all could have their own database objects.
I tried the option to put all in the same schema and using different flyway meta tables, but this does not work: When flyway comes to process e.g. the second table for the second module and discovers that the schema is not empty (because the first module has migrated its db changes), it stops, as flyway has now no chance to determine the state of the db and its migrations.
Also using base line versions do not help, as in that case the changes of the base line would be skiped for the next modules... I think the only reasonable solution with flyway would be to use different schemas....
This is a repeat of a question in the (restricted) Tridion Forum about the inability to delete a structure group. However, since it didn't get a proper answer or solution by the person reporting the question I am re-asking it here.
I am stuck with a structure group, which I can't delete either. It is not localized, only blueprinted to one other Publication and does not have any pages in it. The contents have been migrated from a presentation environment, perhaps an old target stuck somewhere?
Deleting it directly in the database is not an option. Any other solutions?
It is possible you have multimedia components rendered using that Structure Group? This may cause some kind of lock. You might try changing the Set Publish States PowerTool for 2009 to set everything to UnPublished in that pub and see if it helps.
Brute force: Start a DB trace, try and delete the Structure Group via the GUI, look for the items it is finding when checking for dependencies.
Or
Open a support ticket, send them the DB, let them take a look at it.
We came across similar issues at a customer. Our initial analysis was to examine the stored procedures that do the delete, and to see what constraints were enforced. On examining the data, we could see records that would not show up in the user interface, but which would prevent the deletion.
We raised a ticket with SDL Tridion customer support, and were able to agree with them which records should be modified in the database.
So that's the take-away from this: you aren't allowed to modify the database, but SDL Tridion customer support can sanction it, but only once they have checked that the changes are correct and necessary. Obviously, if you were to attempt to do such things without the co-operation of support, you'd end up with an unsupported system.
We are a software development company and are using Wordpress for static portion of the web site. Naturally, all our workflow is built around version control: multiple developers -> continious integration -> staging -> deployment.
Our challenge with integrating Wordpress into our workflow is that its database is stuck like a bone in the throat: you cannot put it into the version control, easily roll back, promote from staging to production etc.
I am wondering what people do in similar situations? I would like to find a way to integrate WP into the development workflow and not the other way around :-)
Clarification we want to "develop" and test pages on the staging system and when ready then move them over to the production as part of the version upgrade process. We don't want to do full replication of the staging database to production.
That's a common question and one that I've worked on tackling. I've written some code to address these issues albeit the code's not ready for distribution. Basically the idea is to create scripts to import the content and then version control the scripts. (Actually my approach uses a custom import/export format designed to be easy to hand-modify, but the idea is similar.)
Anyway, there are some related questions over on StackOverflow's sister site WordPress Answers:
Questions tagged with the term [staging]
Questions tagged with the term [deploy]
UPDATE
Per the clarification, this would probably be helpful too:
Is there any way to draft a revision of a published page or post? What workarounds have you used?
Hope this helps.
-Mike
I've just hit the same problem. For now we are using MySQL dump files to export/import database content, but it gets ugly with several people working on the database changes.
Since the team that works on the project is all internal and consists of just a few people, I'm thinking into the direction of locking the database dump file in VCS. Subversion had this functionality built-in, but we are using git, which, I think, is conceptually opposite of any kind of locking.
Probably we'll have a workaround script with pre-commit hook to check for the existence of a lock file next to the dump. The person who committed the lock file will be the only one allowed to commit the dump. Once he finishes the work, he will need to commit the removal of the lock file.
It sounds ugly, I know. But I've thought about it for a while and don't see an elegant solution yet.
If you're only using WordPress for static content, then any tool/methodology for version controlling databases should work - for example, work the mysql command line tools into your CI and deployment routines.