Does anyone have any best practices around deploying database changes in an EF 4.1 code-first solution? I know MS does not currently support database migrations for EF 4.1, but obviously people are going to need to do this from time to time.
Thanks
Once you deployed database to production you must do incremental changes. It means that before you deploy next version you must prepare two databases in your dev box:
Database with DB schema currently deployed in production - you should be able to get this from source control so always correctly label / tag your production releases
Database with new DB schema
Once you have two databases you can use some tool to make difference SQL script for you. I have experience with both:
Visual Studio 2010 Premium / Ultimate Database tools
Red Gate SQL Compare
These tools are for SQL server.
Once you have difference script you can test it on your dev box. Be aware that some more complicated changes cannot be created by difference script and require you to create custom migration script for example with storing data existing data in temporary tables while refactoring real table. Also if you use some new seed data in your new version you must add them manually into script or use Data Compare tools (also offered by both products).
After that you can plan outage of your production application, database backup and running upgrade script.
Related
I have a Xamarin Forms app that I have switched from using only restful API to using local SQLite DB, that will sync using this Dotmim.Sync - DB Sync'ing Framework (which is great!). I am using EF Core 3 in the Xamarin forms project to interact with SQLite.
My questions are around running migrations or just database updates on SQLite. I have read several blogs and forums about different approaches, but they all are several years old, from EF Core 1 to EF Core 2, and lots of work around.
https://forums.xamarin.com/discussion/101805/xamarin-android-entity-framework-core-2-and-migrations - this talks about running migrations by add console application
https://medium.com/#yostane/entity-framework-core-and-sqlite-database-migration-using-vs2017-macos-28812c64e7ef - this add's a Dot Net Cli Tool Reference. I tried this and got errors, but it also EF Core 1.1
https://www.algoworks.com/blog/xamarin-forms-and-entity-framework-core/ - gave me more questions then answers.
There are more links but I, these prove my point of all the different approaches taken.
What I am trying to achieve is:
When a user opens the app, it checks if there is DB version update, or the app updated and knows it has to update the database.
When there is a DB update, Dotmim.Sync Framework is in control of Provisioning and Deprovisioning the database on the client (Xamarin Forms). Which gives the opportunity to run EFCore Migrations or SQL Sricpts to update the SQLite store.
Option 1
I would like to go the EF Core Migration path, since the Master DB is updated with Migrations, The App would be able to leverage the same scripts. If running migrations on Xamarin forms at runtime is possible that would be great.
Q1. Can EF Core run mirgations in Xamarin Forms apps during runtime, same as it would in .netcore/netframework project?
Option 2
If running scripts needs to be the path thats fine, which i have explored and means I need to be able to SQLite schema compares Tool 1 and tool 2, I tried the tool one on SQLite db3 and it error'ed. Wasn't willing to pay for Tool 2 unless without verification it works.
Q2. Is there a SQLite compare tool to generate schema changes?
Q3. Or what is database update path for sqlite deployed on clients, that would be better, i.e drop the DB and just recreate it? then re sync the DB, to load back all user specific data. The chances of lost data would be minimal, the sync logic in the App will be frequent, not sure after every user data change event, but possible if we need to make sure the server never out of sync.
I hope I made this clear. Thanks
You'll need to create the dummy console app as mentioned. Here are instruction for installing the tools. After that, the normal Migrations workflow should work on Xamarin (Be sure to let us know if it doesn't).
Add-Migration MyMigration -P MyNetStandardClassLibrary -S DummyNetCoreApp
You can apply migrations at runtime using this method:
myDbContext.Database.Migrate();
I use Azure as the production environment for my ASP.NET web application.
I am trying to use a staging environment to test a deployment before it is in production. It seems like the staging environment should use a completely separate database to avoid messing up the production one (esp. to test a new data structure, etc.).
This led me to try setting up a one-way sync so that the production database periodically copies itself to a staging database. However, the Azure SQL sync does not support self-referencing tables (which are created by my implementation of EntityFramework), which makes it unusable.
Is there another method of doing a one-way sync on these databases? Is there another way of going about this problem that I am missing?
I think you are going about this the wrong way. If you make changes to your staging database in the future while developing, and you want to test it on staging before a go-live, the update wouldn't work as it would have a different structure to your live DB.
To begin with you should copy your live database to a staging version and then update that to test as you complete developments. Every now and then after you do a go-live, you can delete your staging DB and restore a new copy from the live site to keep your data clean and up to date for the next development.
I need some clarification on when to register a Database as a Data Tier Application (DAC). I've looked at all the guides but am stuck on a few points.
The database is NOT registered
Build Database Project to produce DacPac
Publish the Database Project
Check "Register as a Data Tier Application"
Check "Block publish when database has drifted from registered version"
First time round, this works. It registers the database and succeeds.
However, on subsequent publishes is fails as it says the DB has drifted noting two users which have not changed.
Am I following the correct process? i.e. setting the Publish script to re-register each time?
What is the best practice for making changes? By changing the relevant .sql files in the Database Project and then building? The guides talk a lot about being able to version the DB using the DacPac but its not clear how. Should I rename each DacPac and commit it to TFS?
My next step is to publish the Database as part of the overall ASP.Net Solution. When I try to do that (it works fine when the DB publish is not included), it comes up with the following error
Web deployment task failed. (The SQL provider cannot run with dacpac option because of a missing dependency. Please make sure that DacFx is installed. Learn more at: http://go.microsoft.com/fwlink/?LinkId=221672#ERROR_DACFX_NEEDED_FOR_SQL_PROVIDER.)
However, I have all the required elements installed on the publishing machine. Do they need to be on the SQL Server or IIS VMs?
Any guidance would be much appreciated.
If you want to deploy your changes to a database using a dacpac you would need to register the database as a DAC. This basically creates a snapshot of the database at that point in time. You do this before making a change to create the initial snapshot and then after a deployment.
The reason you do this is to detect drift. Lets say you do a deployment and someone makes a change directly in that database, for instance changing the logic of a stored procedure, you would want to know about that change before making a subsequent deployment. If you deploy your dacpac and ignore this change it will revert their change to whats in the dacpac model. This is where drift occurs. You can generate an xml report on what has drifted through the sdk.
You can enable a setting to disable deployment if drift occurs so that you can retrofit those changes in the database directly in your source code. You would then need to re-register the database as a DAC to create a new snapshot.
Am I following the correct process? i.e. setting the Publish script to re-register each time? Yes
What is the best practice for making changes? By changing the relevant .sql files in the Database Project and then building? Yes
The guides talk a lot about being able to version the DB using the DacPac but its not clear how. Should I rename each DacPac and commit it to TFS? You can set a version within the databse. Have a look at the properties of the database project. You shouldnt rename the dacpac.
About the ASP.Net publish, I would need a bit more detail around the project structure and environment setup.
I am building a blog-like publishing system on ASP.NET 4.0 (with EF 4.0) that I want to be very easily deployable/backupable in the first place. I am at a decision point of whether making the system to create a database in an SQL Server and use that (traditionally), or have a App_Data MDF file in the site and just attach to that one with SQL Express. I know the memory/size limitations of Express editions, and I won't be hitting the limits as this is not a performance-critical business application or any serious stuff. Just a simple CMS with blogs/writings/photos (actual photos are NOT saved in database, just their paths are saved in MDF) and that's it. I see no problem using MDF, but I'm not an expert on the topic too as I've never worked/created a website using MDF file. I always deployed on the SQL Server, but I don't want to deal with users/roles/permissions and the last thing that I want to have a user having installation problems due to database settings.
What should I go with? Any problems that I would face with MDF? Recommendations?
IF you use SQL Server Express - which is a server - I would always opt for a "real" database approach: attach your database to the server, access it by its database name, deploy SQL scripts to update it.
That "attach DB from file path" always seemed like a half-baked and rather messy kludge to me.....
If you don't need that kind of power - investigate SQL Server Compact Edition which is a one-file only, in-process database. It has its limitations - no stored procedures, doesn't support certain data types like VARCHAR(MAX) or XML - but for easier scenarios, it's perfect and easy to use - just deploy it along your app. It's single-user, e.g. you don't share the data between several clients, it's a local store for each user/app on its own
Summary of environment.
Asp.net web application (source stored in svn)
SQL Server database. (Database schema (tables/sprocs) stored in svn)
db version is synced with web application assembly version. (stored in table 'CurrentVersion')
CI hudson server that checks out web app from repo and runs custom msbuild file to publish/package app.
My msbuild script updates the assembly version of the web app (Major.Minor.Revision.Build) on each build. The 'Revision' is set to the currently checked out svn revision and the 'Build' to the hudson build number (incremented on each automated build).
This way i can match the app to a specific trunk revision also get other build stats from the hudson build number.
I'd like to automate the collecting of migration scripts (updated sprocs etc) to add to the zip package.
I guess by comparing the svn revision of the db that has yet to be deployed to, to the revision being deployed, i can find what db files have changed in the trunk since the last deployment to that database/environment.
This could easily be achieved by manually calling the svn diff -r REVNO:REVNO command to list changed .sql files. These files could then manually have to be added to the package.
It would be great if this could be automated.
Firstly i'd imagine I'll have to write a custom task to check the version of the db that has yet to be deployed to. After that I'm quite unsure.
Does anyone have any suggestion on how this would be achieved through an msbuild task either existing or custom?
Finally I'll have to autogen a script to add to the package that updates the database version table so as to be in sync with the application.
Integrating SQL changes into an automated build/deploy process is HARD. I know, because I've tried to to it a couple times with limited success. What you're trying to do is roughly on the right track, but I would argue that it's actually a bit too complicated. In your proposal, you suggest collecting the specific SQL scripts that need to be applied to your DB at build/package time. Instead, you should package all your delta scripts (for the entire history of your database) with your project, and calculate the deltas that actually need to be applied when you deploy -- that way, your deployable package can be deployed to environments with databases of differing versions. There are two implementation pieces you need to achieve this:
1) You need to package your deltas into your deployable package. Note that you should package deltas -- not static files that create the schema in its current state. These delta scripts should be in source control. It's okay to keep the static schema in source control as well, but you will have to keep it in sync with the deltas. You can actually use a tool like Red Gate's SQLCompare or the VS Database version to generate (most) deltas from the static schema. To get the deltas into your deployable package, and given that you're using svn -- you may want to look into svn:externals as a way to "soft link" the delta scripts into your web project. Your build script can then simply copy them into your deployable package.
2) You need a system that can read the list of delta files, compare them to an existing database, determine which deltas need to be applied to that database, and then apply the deltas (and update the bookkeeping information, like the database version). There is an open-source project (sponsored by ThoughtWorks) called dbdeploy that accomplishes this. I've had some success with that tool personally.
Good luck -- this is a tough nut to crack (correctly).
Have a look at SQL database projects. In VS 2010 they have been enhanced quite a bit and have built in deployment capabilities that can sync your DEV database to other environments.
Here are a few good links about DB projects in vs 2010:
http://msmvps.com/blogs/deborahk/archive/2010/05/02/vs-2010-database-project-building-and-deployment.aspx
http://weblogs.asp.net/gunnarpeipman/archive/2009/07/29/visual-studio-2010-database-projects.aspx
Try SQL Examiner:
http://www.sqlaccessories.com/Howto/Version_Control.aspx
You can automate script collecting with SQL Examiner command-line tool.
The solutions available today that target a .NET/SQL Server stack are:
DBUp (open source)
ReadyRoll (deeper Visual Studio integration,
auto-generation of scripts)
The latter product is one that we're actively developing here at Redgate.