I use maven for db migration with flyway. And I want to use names of sql scripts like: "test_1.1.sql". But flyway throws exception: "com.googlecode.flyway.core.api.FlywayException: Wrong migration name format". If I use name: "test_1.1__some_description.sql", all goes fine.
But such format isn't acceptable for me. How to solve this problem?
This is not currently possible. It used to be an option in pre-2.0 versions. Removing it made it possible to detect a bunch of possible mistakes, which couldn't be reliably identified before.
On a all but the most trivial projects, having a description makes a lot of sense.
If you feel we overlooked a valid real world usecase for not having a description, please file an enhancement request in the issue tracker.
There are some properties for modify flyway migration scripts naming convention.
Properties Name: sqlMigrationPrefix and sqlMigrationPrefix
In your Case test is prefix for migration script and suffix is .sql.
So, Your properties should be set like:
flyway.sqlMigrationPrefix = test
flyway.sqlMigrationPrefix = .sql
At Last, Your naming convention should be "test1_1__Some_Description" where
__ is use for separator for "Migration Script Description".
Related
I think I've a good understanding of Symfony and how bundle works.
However I've never found how to solve a simple problem: make a reusable bundle that provides data like tables/Doctrine entities pre-filled with (i.e.) all country names in the world, all provinces of Italy, tax rates history in England and so on.
Of course the purpose is to provide forms, services and controllers relying on this data source, without the need to copy and paste tables and entities across projects.
How would you do that?
Data fixtures IMHO are not an option because an obvious reason: you are going to purge your database while it's running.
A custom command reading from a static data-source (json, YAML) and performing inserts/updates?
First step is declaring a Doctrine entity in your Bundle. I think you should create DataFixtures to populate your datas into db.
You maybe should consider to use Seeds instead of Fixtures.
Fixtures are fake datas, used to test your application
Seeds are the minimal datas required for your application to work.
Technically, these are exactly the same thing, you declare it under the "DataFixtures/" folder and you import them with the "doctrine:fixtures:load" command.
You can create a folder "Fixtures/", and a folder "Seeds/" under the folder "DataFixtures", then load your seeds with the command
php app/console doctrine:fixtures:load --fixtures=/path/to/seeds/folder --append
It was suggested in the comments that it may be safer, especially in production environment, to create a custom Symfony2 command to force the "--append" mode. Without this mode, your database will be purged, and you could loose your production data.
This answer assumes you're using composer to install your bundles (and you really exclude fixtures as an option).
What you can do, is make an SQL export of the data you want, and make sure it uses INSERT IGNORE INTO, and get the correct unique constraints.
Then you save that file somewhere in your bundle, in a "data" or "fixtures" folder.
so your path to that file will be like:
"vendor/company/epicbundle/data/countries.sql"
What you then can do, is add post-insert and post-update commands in your composer.json, that looks like this:
"post-install-cmd": [
"php app/console doctrine:query:sql \"$(cat vendor/company/epicbundle/data/countries.sql)\""
]
If you only want it to run on install, you only add it there, if you sometimes update the sql file, you also add it to the post-update-cmd.
Please note that this solution only works if people don't temper with the table names, otherwise the queries will fail.
If you want a more save/stable solution, you can write your own post-install script in Symfony that uses the entity manager, and there you can use, for example, a csv file, and insert/update it row by row.
Basically, anything you could implement would surely rely on persistence mechanisms used in your ORM/ODM/whatever. So, you'll end up implementing a typical fixture loading mechanism, at least partially: you'd execute code that saves some provided data; if it's serialized you'd do XML/JSON/YAML parsing (but this is just a technicality) and persist the results into the database.
Thus, it's not bad to stick with Doctrine Fixtures. They are programmable and extensible (you can even fetch your data from the web upon loading).
As stated in #paul-andrieux's answer, if you are worried about data loss (e.g. your bundle's seeds are loaded when the end user's DB is already up), you should use doctrine:fixtures:load --append and let the constraints do their job (like, in a country names table you'd have a unique constraint on country name or even a 'slug') so that inserting duplicate rows silently fails inserting a single entity, in case if your bundle has updated the country list, and the end user had a previous version.
If you really worry about your end users' data you could write a wrapper for the doctrine:fixtures:load command that would have the --append flag always on and register it as a separate command. (You could run needed migrations there, too)
#lxg's hard-coded IDs problem is solvable, too. Try using natural keys where applicable (e.g. the countries table would have a slug primary key that would be great-britain for Grean Britain). This way your searches would be pretty easy: $em->find('\MyBundle\Country', 'great-britain');. If you cannot come up with a natural key, then maybe the entity is not really needed for the end user.
UPD. Here's an article that could be useful: http://www.craftitonline.com/2014/09/doctrine-migrations-with-schema-api-without-symfony-symfony-cmf-seobundle-sylius-example/
Generally speaking, the bundle embedded the entities that will be loaded via the ORM/ODM using their built-in commands (like doctrine:schema:update, doctrine:migration:diff, ...) and provides a custom command that load the required fixtures using the ODM/ORM
This command can read the fixtures in multiple way (parsing yaml, xml, raw sql, dql, ...), it is just a matter of taste. Tones of bundles, parser, ... exist for those tasks.
In your documentation, you just have to state in a clear way that the developer must run this command after your bundle installation and schema update.
I am wondering if there is any news for multi line comment directive support for MySql.
I believe the problem is related to:
Mysql dump comments directives and simple comments
Basically it seems SqlScript/MySQLSqlStatementBuilder fail in recognizing as directive a statement in the format:
/*!50001 <STATEMENT SPLITTED IN
MULTI
LINES> */;
I am using version 2.2.1
Additional notes:
the same goes for previous versions (2.1.1), and issues are multiple and not only related to comment directives; theu are also hard to debug because of missing exception trace.
Basically I think parser cannot currently handle pretty standard mysql scripts created with mysqldump; this IMO is a necessary feature for any usage in existing projects at least.
If anybody has suggestions to overcome these issues, it would be highly appreciated.
I am using Flyway 2.1.1 API in Java.
What I am doing is parsing a dump file (only the schema structure) and replace the version dependant comments.
This solution is pretty bad, but waiting for a better support of MySQL dump in Flyway, I didn't find anything else...
I need to be able to perform all of the available functions that the Package Manager Console performs for code first DB migrations. Does anyone know how I could accomplish these commands strictly through user defined code? I am trying to automate this whole migration process and my team has hit the dreaded issue of getting the migrations out of sync due to the number of developers on this project. I want to write a project that the developer can interact with that will create and if need be rescaffold their migrations for them automatically.
PM is invoking through PowerShell and PS cmdlets (like for active directory etc.)
http://docs.nuget.org/docs/reference/package-manager-console-powershell-reference
The Package Manager Console is a PowerShell console within Visual
Studio
...there is essentially very little info about this - I've tried that before on couple occasions and it gets down to doing some 'dirty work' if you really need it (not really sure, it might not be that difficult - providing you have some PS experience)
Here are similar questions / answers - working out the PS comdlets is pretty involving - in this case it has some additional steps involved. And PS tends to get very version dependent - so you need to check this for the specific EF/CF you're using.
Run entityframework cmdlets from my code
Possible to add migration using EF DbMigrator
And you may want to look at the source code for EF that does Add-Migration
(correction: this is the link to official repository - thanks to #Brice for that)
http://entityframework.codeplex.com/SourceControl/changeset/view/f986cb32d0a3#src/EntityFramework.PowerShell/Migrations/AddMigrationCommand.cs
http://entityframework.codeplex.com/SourceControl/BrowseLatest
(PM errors also suggest the origins of the code doing the Add-Migrations to be the 'System.Data.Entity.Migrations.Design.ToolingFacade')
If you need 'just' an Update - you could try using the DbMigrator.Update (this guy gave it a try http://joshmouch.wordpress.com/2012/04/22/entity-framework-code-first-migrations-executing-migrations-using-code-not-powershell-commands/) - but I'm not sure how relevant is that to you, I doubt it.
The scaffolding is the real problem (Add-Migration) which to my knowledge isn't accessible from C# directly via EF/CF framework.
Note: - based on the code in (http://entityframework.codeplex.com/SourceControl/changeset/view/f986cb32d0a3#src/EntityFramework.PowerShell/Migrations/AddMigrationCommand.cs) - and as the EF guru mentioned himself - that part of the code is calling into the System.Data.Entity.Migrations.Design library - which does most of the stuff. If it's possible to reference that one and actually repeat what AddMigrationCommand is doing - then there might not be a need for PowerShell at all. But I'm suspecting it's not that straight-forward, with possible 'internal' calls invisible to outside callers etc.
At least as of this post, you can directly access the System.Data.Entity.Migrations.Design.MigrationScaffolder class and directly call the Scaffold() methods on it, which will return you an object that contains the contents of the "regular" .cs file, the "Designer.cs" file and the .resx file.
What you do with those files is up to you!
Personally, I'm attempting to turn this into a tool that will be able to create EF6 migrations on a new ASPNET5/DNX project, which is not supported by the powershell commands.
Sorry, I'm a little unclear on the web2py manual explanation.
as an example, given app1 and app2
I want to have app2 share the database I have built in app1
So do I change the app2/models/db.py file to show: db = DAL('sqlite://storage.sqlite',migrate='false') ?
and include all other myModel.py files in app2/models directory as well?
if the database is in app1/databases/ how does app2 know how to find the correct database file?
This Thread begins to answer the question but I'm still unclear on how to define where the shared database lives.
Note, DAL(..., migrate=False) just sets the default value of migrate for each table -- it will not have any effect on the migration status of tables whose define_table() calls include their own explicit migrate argument. If you want to completely disable migrations for an entire db connection (regardless of the individual define_table() calls), instead use:
DAL(..., migrate_enabled=False)
Also, to share model definitions between applications, rather than simply copying the model files, you could put the definitions in functions or classes within modules and then import the modules. Another option is to use auto_import:
DAL(..., auto_import=True)
Note, auto_import will import the field names and types, but it will not include DAL-specific attributes, such as validators and defaults, so its usage is somewhat limited.
I can't test this right now but the answer should be:
you can override the folder in the DAL:
So both apps should point to the same file.
(see the docs and this thread).
.
db = DAL('sqlite://storage.sqlite',folder='path/to/app/databases')
yes, should need the model files in both apps too, otherwise the apps won't know how to access the db.
What method do you use to get a compile time error when the database schema changes occur in an ASP.NET project?
For example, if you have a GridView bound to a DataSource, I can only get runtime errors when a schema change occurs, not a compile time error. Intellisense works fine on the code behind using datasets, LINQ, etc, but I cant seem to get a compile time error on an ASP.NET page when I change the schema.
Any advice?
Create a unit test that verifies the correctness of you data access layer, and make sure it covers all your DB-related code. Not everything can be caught at compile time...
One way I can think of easily achieving this behavior would be to databind to a dynamic DAL. There are some tools that can help do this DAL generation, I'd recommend taking a look at SubSonic.
Once you have something like SubSonic in place you can bind to the resulting business objects. These business objects will automatically change in the case of a schema change in the database and this will break your binding code which will result in a compile time error.
Update
Assaf's recommendation to use Unit Tests is also a good idea. It doesn't solve your stated problem but it is definitely something that should be in place and is a great tool for flagging these type of problems.
We use a modest system (xml to c++) to create schemas from an independent description, this system also creates names for tables and columns that we use inside the code, when there is a change in the schema the names change, as the names we originally used are not there anymore and the compiler will flag an error.
You could probably configure a lot of the DAO generation tools to do something similar.
One solution would be to version your database and map an application build to a specific version (maybe in a properties file). In the entry point of your app, you can compare the expected version to the actual version and handle the error accordingly.
I'm not sure whats the equivalent in ASP.net of Migrations in Rails or dbdeploy in Java for versioning your database. But any DB versioning tool that makes schema changes incremental and versioned and tracks the version in a Version table will suit the purpose.
But if you want a compile time error while building your app, you might as well upgrade your schema to the latest version as part of your build process, avoiding the possibility of schema changes in the first place.