How does Flyway handle multiple schema dependencies?
Ie
V1_CREATE_TABLE.sql in SCHEMA_A
V2_CREATE_VIEW.sql in SCHEMA_B (based on table in schema A)
Is it possible to ensure that V1 is created before V2?
It is also possible to get into a cross dependency. For example say V3_CREATE_VIEW_2.sql in SCHEMA_A (based on view in schema B). How do we ensure that dependency?
Flyway will execute the scripts in order based on their version numbers. If you want a single instance of Flyway to manage objects in multiple schemas, you must prefix the object names in the migration scripts.
Related
We have a SQL 2019 database where all table names are fully qualified in views starting with the database name. We do NOT have the option of avoiding the fully qualified reference as the view definition is auto-generated (otherwise I would simply not fully qualify them). When views are defined by referencing tables within the same database as the view, the SSDT project complains that it has an unresolved reference.
Visual Studio does not allow adding a database reference to itself. The only way I can get it to compile is to create a DACPAC of the same database and then add that as a reference along with removing the database variable ($Name).
Is there any other method of providing fully qualified table names in views without having to create a DACPAC in SSDT project?
Only way I'm aware of would be to take the view code out of the project and handle in post-deploy scripts. This is done by design, because the database name may not be what was defined in the original code.
You can't use 3-4 part naming in SSDT normally. You can workaround this by using variables in the code. So let's say, that you have [localhost].[reports].[dbo].[your_table] you'll need to use [$(ReportServer)].[$(ReportDatabase)].[dbo].[your_table].
I have a DacPac project containing objects which use three part naming to refer to the containing database (hundreds of instances such as [thisDb].[dbo].[obj]* exist). I need compare and update this database, but the db project fails to build due to 200+ sql71561 errors.
I did not want to remove the unnecessary database name part or switch to using a database name variable. To successfully build (or compare, and then update) a database using three part naming or fully qualified naming to refer to itself, there is a way I found to pacify visual studio. It's not what I'd prefer, but it works.
Create a copy of the original db project.
In the copy db project, update all local database object references to use just two part names ([dbo].[obj]) instead of three part names (I used find & replace).
Make sure the copy db project targets the same SQL server version and builds successfully.
Reference the copy db project from the original db project (whether via database variable, database name only, or dacpac).
The original db project can now build because its references can be resolved. You'll end up with a dacpac for both the original and the copy, but at least the errors are gone and it compiles.
Upgraded our application from .Net Core 3.1 to 5.0.6, and everything was working using an existing database. If I drop the database and run the migrations I get this error:
There is no entity type mapped to the table 'UserClause' used in a data operation. Either add the corresponding entity type to the model or specify the column types in the data operation.
The UserClause table no longer exists it gets renamed in the migration that fails, but if this particular migration completed it would be the Agreement table, but it throws on this migration due to a missing entity type, which definitely doesn't exist either since this was originally run over a year ago.
Anyone know why the migration would be looking for an entity type, and how to resolve the issue? Doesn't seem like this should be checking for entities while it runs the migration.
Looks like in EF Core 5 that some of our migrations where there was a table rename had to have their update or insert statements moved to be after the rename of the table, otherwise you would get this error. Once they were after the table renaming and the table name for the update/insert were changed to the renamed table name everything worked.
In my case, i've to add the schema parameter to the InsertData function, like "dbo"
Is there a way to define the complete database schema in one go in Symfony 4?
I understand that individual entities/objects can be created using the make:entity and make:migration commands but I'm wondering if I could just define the entire schema in one sitting and then use it to build the associated entities and database.
I recall that in earlier versions of Symfony it was possible to define the entire schema in a YAML file and then just issue a build command.
Yes, you can create complete database schema mappings using any of supported mapping formats (e.g. YAML or XML) and declare mappings location in Doctrine configuration. After that you will be able to use any Doctrine console tools to generate and update schema. You can also use tools for reverse engineering mappings from already available database and to convert mappings between formats
Please notice that Doctrine commands names in Symfony application are different from ones that natively provided by Doctrine. You need to use:
doctrine:schema:validate for schema validation
doctrine:schema:create for initial schema generation with subsequent calls of doctrine:schema:update with either --dump-sql or --force depending on your needs
doctrine:mapping:convert to reverse engineer available database (with use of --from-database option) or convert between mapping types in a case if you want to.
I think I've a good understanding of Symfony and how bundle works.
However I've never found how to solve a simple problem: make a reusable bundle that provides data like tables/Doctrine entities pre-filled with (i.e.) all country names in the world, all provinces of Italy, tax rates history in England and so on.
Of course the purpose is to provide forms, services and controllers relying on this data source, without the need to copy and paste tables and entities across projects.
How would you do that?
Data fixtures IMHO are not an option because an obvious reason: you are going to purge your database while it's running.
A custom command reading from a static data-source (json, YAML) and performing inserts/updates?
First step is declaring a Doctrine entity in your Bundle. I think you should create DataFixtures to populate your datas into db.
You maybe should consider to use Seeds instead of Fixtures.
Fixtures are fake datas, used to test your application
Seeds are the minimal datas required for your application to work.
Technically, these are exactly the same thing, you declare it under the "DataFixtures/" folder and you import them with the "doctrine:fixtures:load" command.
You can create a folder "Fixtures/", and a folder "Seeds/" under the folder "DataFixtures", then load your seeds with the command
php app/console doctrine:fixtures:load --fixtures=/path/to/seeds/folder --append
It was suggested in the comments that it may be safer, especially in production environment, to create a custom Symfony2 command to force the "--append" mode. Without this mode, your database will be purged, and you could loose your production data.
This answer assumes you're using composer to install your bundles (and you really exclude fixtures as an option).
What you can do, is make an SQL export of the data you want, and make sure it uses INSERT IGNORE INTO, and get the correct unique constraints.
Then you save that file somewhere in your bundle, in a "data" or "fixtures" folder.
so your path to that file will be like:
"vendor/company/epicbundle/data/countries.sql"
What you then can do, is add post-insert and post-update commands in your composer.json, that looks like this:
"post-install-cmd": [
"php app/console doctrine:query:sql \"$(cat vendor/company/epicbundle/data/countries.sql)\""
]
If you only want it to run on install, you only add it there, if you sometimes update the sql file, you also add it to the post-update-cmd.
Please note that this solution only works if people don't temper with the table names, otherwise the queries will fail.
If you want a more save/stable solution, you can write your own post-install script in Symfony that uses the entity manager, and there you can use, for example, a csv file, and insert/update it row by row.
Basically, anything you could implement would surely rely on persistence mechanisms used in your ORM/ODM/whatever. So, you'll end up implementing a typical fixture loading mechanism, at least partially: you'd execute code that saves some provided data; if it's serialized you'd do XML/JSON/YAML parsing (but this is just a technicality) and persist the results into the database.
Thus, it's not bad to stick with Doctrine Fixtures. They are programmable and extensible (you can even fetch your data from the web upon loading).
As stated in #paul-andrieux's answer, if you are worried about data loss (e.g. your bundle's seeds are loaded when the end user's DB is already up), you should use doctrine:fixtures:load --append and let the constraints do their job (like, in a country names table you'd have a unique constraint on country name or even a 'slug') so that inserting duplicate rows silently fails inserting a single entity, in case if your bundle has updated the country list, and the end user had a previous version.
If you really worry about your end users' data you could write a wrapper for the doctrine:fixtures:load command that would have the --append flag always on and register it as a separate command. (You could run needed migrations there, too)
#lxg's hard-coded IDs problem is solvable, too. Try using natural keys where applicable (e.g. the countries table would have a slug primary key that would be great-britain for Grean Britain). This way your searches would be pretty easy: $em->find('\MyBundle\Country', 'great-britain');. If you cannot come up with a natural key, then maybe the entity is not really needed for the end user.
UPD. Here's an article that could be useful: http://www.craftitonline.com/2014/09/doctrine-migrations-with-schema-api-without-symfony-symfony-cmf-seobundle-sylius-example/
Generally speaking, the bundle embedded the entities that will be loaded via the ORM/ODM using their built-in commands (like doctrine:schema:update, doctrine:migration:diff, ...) and provides a custom command that load the required fixtures using the ODM/ORM
This command can read the fixtures in multiple way (parsing yaml, xml, raw sql, dql, ...), it is just a matter of taste. Tones of bundles, parser, ... exist for those tasks.
In your documentation, you just have to state in a clear way that the developer must run this command after your bundle installation and schema update.
I am working on a ASP.NET Web Forms project which is expected to experience lots of database changes even after deployment.
Our preference was to use Entity Framework 5, and the Database First design paradigm. However, since we have to make lots of changes to the database even after deployment, if I use database first approach, then every time I update my database, I will have to delete my entire model and regenerate it. Is there any best practices to make this process less painful?
You should use Code First, so that you can use Migrations.
More specifically I'd use manual Migrations.
With manual Migrations you can create Migrations at any point in time. A migration has the following information:
a snapshot of the current model
a set of "instructions" to upgrade from the previous migration
a set of "instructions" to downgrade to the previous migration
Apart from the necessary Migrations, you should add a new Migration when you deploy your app. For example, you can create a migration called "Version 1.0", when you deploy the version 1.0 of your app.
When you finish each new stable version, you simply add a new migration, for it, like "Version 1.1" or "Version 1.2".
The interesting part of migrations shows up when you have a deployed application version and you need to upgrade (or downgrade) to a new version.
There are commands that let you upgrade (or downgrade) a DB from one particular version
to another particular version. You can do it directly specifying a connection to the DB, or create a SQL script which will apply the changes to the DB. For example, if you deployed the version 1.0 in a customer server, and you need to upgrade the software to version 1.2, you can do this:
Update-Database -SourceMigration "Version 1.0" -TargetMigration "Version 1.2"
-Script
This will create a SQL script which can be run on the DB to upgrade from version 1.0 to 1.2.
If you need help about any of the Migrations commands, simply type:
get-help Update-Database -full
(Update-Database is a command name, you can specify any other like Add-Migration)
It's possible that you need to specify the project where the model is in, the connection string name, the name of the project with the .config file or some other things, depending on the specified parameters, and the structure of the projects in your solution.
To get more info on migrations, read MSDN EF Code First Migration.
NOTE ADDED IN EDITION: there is a new DB initializer that can automatically migrate to the latest version when the application runs. I've worked in a real application, and it works like a charm.
ALTERNATIVE SSDT
If you don't want to follow the advice, you can use SQL Server Data Tools (which can be installed inside VS, or work as an independente application, depending on the version you're using).
The idea of this tool is that you can compare projects (which are DB schema snapshots) to existing DBs, and generate the scripts to change the DB to match the schema in the project. (In fact you can compare any combination ofr project and DB)
If you keep versions of your project in a CVS you can even check the changes from one verison of the project to other version of the project.
NOTE ADDED IN EDITION: a SSDT project is a set of scripts that can build the whole DB, including all the objects in its schema. You can create it from an existing DB or viceversa. Then you can keep comparing any combination of DBs and SSDT projects, as soruce or target, and create an apply the scripts neccesary to change each particular object which has changes. (The scripts are created to change the target so that it looks like the source, but you can swap source and target easyly)
This is an alternative solution, but Migrations are much more powerful because you can customize them, for example to fill a "master table" when creating it, to set the initial value of a new column, and so on. If you use SSTD you'll have to do all of that by hand, and carefully keep track of it. So I highly recommend using Migrations.