edit removed. check Dealing with Database changes between version branches/rollbacks in ASP.NET
Just off the top of my head, using stored procedures would help with this, as your pages wouldn't need to know the schema. Additionally, you update your stored procedures once, and you don't have any additional changes to make in your code (unless it's an incredibly massive schema change).
Related
What are the practical advantages of Doctrine Migrations over just running a schema update?
Safety?
The orm:schema-tool:update command (doctrine:schema:update in Symfony) warns
This operation should not be executed in a production environment.
but why is this? Sure, it can delete data but so can a migration.
Flexibility?
I thought I could tailor my migrations to add stuff like column defaults but this often doesn't work as Doctrine will notice the discrepancy between the schema and the code on the next diff and stomp over your changes.
When you are using the schema-tool, no history of database modification is kept, and in a production/staging environment this is a big downside.
Let's assume you have a complicated database structure in a live project. And in the next changeset you have to alter the database somehow. For example, your users' contact phones need to be stored in a different format, not a VARCHAR, but three SMALLINT columns for country code, area code and the phone number.
Well, that's not so hard to figure out a query that would fetch the current data, separate it into three values and insert them back. That's when migrations come into play: you can create your new fields, then do the transforms and finally drop the field that was holding the data before.
And even more! You can even describe the backwards process (the down migration), when you need to undo the changes introduced in your migration. Let's assume that someone somewhere relied heavily on the format of the VARCHAR field, and now that you've changed the structure, his piece of code is not working as expected. So, you run migration:down, and everything gets reverted. In this specific case you'd just bring back the old VARCHAR column and concatenate the values back, and then drop the fields.
Doctrine's migration tool basically does most of the work for you. When you diff your schema, it generates all the necessary up's and down's, so only thing you'll have to do is handle the data that could be damaged when the migration is applied.
Also, migrations are something that gives other developers on your team knowledge on when it's time to update their schemas. With just the schema-tool, your teammates would have to run doctrine:schema:update each and every time they pull, `cause they wouldn't know if the schema has really changed.
When using migrations, you always see that there are some updates in the migrations folder, which means you need to update your schema.
I think that you indeed nailed it on Safety. With Migrations you can go back to another state of the table (just like you can do in Git version control). With the schema update command you can only UPDATE the tables. There is no log kept for going back in case of a failure with already saved data in those tables. I don't know exactly, but doesn't a migration also saves the data of the corresponding table that's being updated? That would be essential in my opinion, otherwise there is no big reason to use them.
So yes, I personally think that the main reason for using migrations in a production environment is safety and maybe a bit of flexibility. Safety would be the winner here I think :)
Hope this helps.
edit: Here is another answer with references to the Symfony docs: Is it safe to use doctrine2 migrations in production environment with symfony2 and php
You also cant perform large updates with plain doctrine migration. Like try to update index on 30 mln users database. As it will a lot of time while you app will not be accessible.
My project contains a lot of objects like views and stored procedures which are being changed quite frequently. Now I have to create new SQL script on every update which contains complete source code of changed objects despite I've actually changed only few rows. It leads to massive code duplication and I also found it difficult to review these changes.
I'd like to have only one actual version of SQL script for every object like view or procedure and recreate these objects every time I redeploy the database. As result I could change existing source file (like in Java or C programming) instead of creating a new update every time I need to alter view or procedure.
Is there a possibility to execute some scripts every time I migrate the database with Flyway?
I'm not sure why that got so many downvotes, it's a perfectly understandable and valid question. Perhaps it's because it closely resembles this open question:
Migrating Stored Procedures with Flyway
We are actually starting to push against this issue now. We've been using flyway for development and testing (and love it). But we've come to a point where we're starting to have to use procs/triggers/views (p/t/v's) and the fundamental disconnect between how we did it before, and how we must use flyway, is starting to be a strain.
Before, for a given database object (let's say it's a procedure), there'd be one source file. And if you needed to change the proc 'n' times, there would be 'n' versions of the same file in your VCS. Diff tools work great, IDE's all understand this, merges detect when two developers working in separate branches make changes to the proc, etc, etc. You know, old school.
But with flyway, any one proc with 'n' changes is now scattered across 'n' files. Instead of "one object in one file with 'n' versions", you have "one objecct in 'n' files with one change each". I now need to do a text search in my IDE for any instance of "proc_name" if I want to know the history of changes to the proc. The VCS knows nothing about it. Devs can each make a migration in their own branches that succeed when each is deployed, but leave the proc with a missing update.
I'm not saying any of this to complain about flyway, and I fully realize it's not a simple area. I'd almost say it's unsolveable (by flyway).
We're scheming how to handle this problem, and I'd be very interested to know how others have handled it.
Repeatable migrations are supported by Flyway 4.0, now.
Just add sql files starting with "R" without any version information to your migration folder:
R__Blue_cars.sql
You have to ensure, that the script could be repeatable migrated.
This is usually done by "CREATE OR REPLACE" clauses in your DDL statements.
https://flywaydb.org/documentation/migration/repeatable
We just created a new field in a database table, and so deleted, and re-inserted the table in the LINQ Class. The new database field appears in the LINQ Class in the diagram. However, when we're using the field, we get an error that says the table does not contain a definition for the field.
Any ideas on how we can solve this? Thanks!
UPDATE: What steps are required to update the LINQ to SQL Class? Maybe we're doing something wrong.
UPDATE 2: Picture of our problem - LINQ - http://img99.imageshack.us/img99/6033/usertable.png | Code - http://img43.imageshack.us/img43/5145/linqerror.png
Check the table def side of your mapping documents, either using properties in the designer, or by closing Studio and examining the XML. I recommend the designer.
Make sure the field name matches the field name in the database.
I've had problems with a few reserved keywords when using Linq2Entities, and I'd recommend you avoid reserved words in names (even though the [] handle them).
While this doesn't answer your question necessary, it may help to solve it - I've been a long time fan of the LINQ to SQL and Entities tools by Huagati. The re-sync aspect alone has saved me so much time, it's well worth the $50 (for the standard version) IMO.
http://www.huagati.com/dbmltools/
Hope it helps...
Edit:
In order to update the LINQ to SQL classes, you can either do it manually (bllurgh) or, you can remove them from the designer and drag-and-drop them from the Data Connections node in the Server Explorer.
I had to delete the entire LINQ Class, recreate it and re-add the tables for my problem to go away. Simply deleting a single table and re-adding it, or deleting all tables in the class and re-adding them did not work either.
I was having exactly the same issue, but I found that deleting the problematic tables in the Object Relational Designer and re-adding them (and re-adding the associations as well) solved the issue. I did not have to delete the entire DataContext, nor did I have to delete any of the tables that were still working properly. I would recommend trying this first before doing anything more drastic.
In our application,Many pages includes "update" and when we update a table,we update unnecessary columns,which dont change,too.
i want to know that is there a way to avoid unnecessary column updates?We use stored procedures in .net 2003.In Following link,i found a solution but it is not for stored procedures.
http://blogs.msdn.com/alexj/archive/2009/04/25/tip-15-how-to-avoid-loading-unnecessary-properties.aspx
Thanks
You can really only accomplish this with a good ORM tool that generates the update query for you. It will typically look at what changed and generate the query for only the columns that changed.
If you're using a stored procedure then all of the column values get sent over to the database anyway when you call the stored procedure so you can't save there. The SP will probably just execute a run-of-the-mill UPDATE statement. The RDMS then takes over. It won't physically change the data on disc if it's not different. It's smart enough for that.
So my answer in short: don't worry about it. It's not really a big deal and requires drastic changes to get what you want and you wont even see performance benefits.
When I was working at a financial software company, performance was vital. Some tables had hundreds of columns, and the update statements were costly. We created our own ORM layer (in java) which included an object cache. When we generated the update statement, we compared the current values of every field to the values as they were on load and only updated the changed fields.
Our db was SQLServer. I do not remember the performance improvement, but it was substantial and worth the investment. We also did bulk inserts and updates where possible.
I believe that Hibernate and the other big ORMs all do this sort of thing for you, if you do not want to write one yourself.
Is it a bad idea (and if why?) to add a a column to the auto generated asp.net (ASPNETDB.MDF, visual studio 2008, mvc framework) "user roles - database"?
(E.g I want to add the columns RealName and LastName to the aspnet_Users table in the database.)
The reason I want to add a column instead of creating an entire new table is to avoid the doule maintenance issue and unnecessary redundancy
There are two generation schemes that are used (from Pragmatic Programmer):
Those that are used once to generate code
Those that are used all the time to have some code synced
The ones that are used for syncing, the results should not be modified, since they could be overridden at a later date when the generation gets done again.
In the case of your generated asp.net database, there is no reason for you to rerun the generation, so it would be OK to edit it.
The only scenario under which you would rerun the generation of the db is if microsoft releases a new version of the users database and you want to use the new one (in this case you might have to edit some parts of your application, so you could readd those two fields), or if you want to regenerate the database with different options. Both of these happen if you are not happy with your current db.
In my opinion that autogenerated database should be replaced by a normal table in application database or at least there should be an official solution to this problem.
I heard that this is quite good solution: http://www.asp.net/downloads/sandbox/table-profile-provider-samples/
why dont you create a new table with a Foriegn Key restraint? It seems like a bad idea to add a column to the aspnetdb...it will be a nightmare if you ever need to recreate your db...
First, those tables aren't really anything specific to MVC: they're created by/for the default AspNetSqlMembershipProvider. (Also applies to other kinds of ASP.NET applications.)
You could probably add new columns safely, but the membership provider wouldn't "see" them. It does provide its profile mechanism to store extra information (which gets serialized, and stored in the aspnet_Profiles table).
If you need to store lots of additional information about the user, you might also check out this sample membership provider that stores profile information in first-class tables, rather that in profile blobs.