We do rapid development of web applications and we're looking for ways to separate our development and production databases (we currently develop directly on production... it's bad news).
We use ASP.NET Webforms with LINQ2SQL and Dynamic Data for CRUD. How can we do database development locally and then deploy changes to production? I've seen Entity Framework Code-first migrations, but I don't know of any equivalent for LINQ2SQL. We don't want to switch to EF as our CMS is built around LINQ2SQL.
We would also need production data to be available locally (not up to the minute, but recent enough) so we can debug with real data if problems arise.
This is the only idea I've come up with so far but it's far from ideal:
Initial development is done locally then deployed to production
Subsequent maintenance is then done on a local replication of the production database. Then we use some kind of 'database diff' tool to determine the changes that were made, and migrate those changes to production.
Is this an acceptable way of doing things? Is there a better way we could use?
Thanks
Develop your data model and procedures in SSDT Database Projects. This keeps a perfect source controlled copy of what you want the database to look alike at any moment in time. Then let the tooling generate the publishing scripts for you.
Developers should always develop on their own local copy of a database. They can check out scripts form the database project and make changes which they publish locally They can get latest on the checked in project, merge their changes, deploy locally again, test it out, and then check in their changes. Only when everything is tested out, you then publish the changes to production.
You end up treating your database schema very much like code source files.
To get production data down to your development server I would take a .bacpac or .dacpac of the production DB and them import it into your local DB. This works well because you need the schema definition along with the data since it is likely that prod is an older version than what you would have in dev
Yeah I think you basically hit the nail on the head. Those are the two things I have done.
You develop locally and check in your SQL scripts to source control. Then you run the scripts for a deployment. What I've seen work well is dropping/re-creating all stored procedures (seems scary, but if you trust those scripts it's very helpful), and then having one-off scripts per deployment for schema changes and data migration.
Periodically you will copy down production data and restore it locally. Obviously this sync can only happen easily right after a deployment since that's when local and production will be the same. At my current job we actually duplex writes and send a copy to the lower environments and so I suppose that's an option. You could replicate data from production somewhere else and work off of that/write a tool to bring the data into local.
From what I've seen there are no easy answers.
Related
We have a ASP.NET web application and need to maintain the database creation and initialization script.
Are there any industry best practices that people know of for maintaining database creation and initialization scripts. I can think of two main approaches.
Maintain a tsql creation script directly by hand.
Maintain a master database and create the script that is then checked into source safe.
Also the script should be able to be tracked through source control, i.e. table order should be controllable.
If possible should also include the ability to track initialisation data either in the same or a seperate script.
Currently we generate the script from management studio but the order of the tables seems to be random.
And the more automated the solution the better.
The problem is not maintaining the script, nor maintaining a 'master' copy of the database. The real problem is upgrading existing database(s). You do your modification in the developer environment, which are then propagated to the test environment, and finally pushed into production environment. While at developer and test environment stage is possible to start from scratch, in production you always have to upgrade the existing deployment.
In my experience the best practice is to use upgrade scripts. This practice is useful even with a single deployed site, but it becomes invaluable with multiple locations that may be at different versions. But even with one single operational site is still useful to be able to test the upgrade repeatedly (starting from backups of current version), keep the changes in source control, have a well formalized and peer reviewed change procedure (the upgrade script). And upgrade scripts can be tailored to specific needs of the operational site, like handling a large table with special care, or deal with encrypted data, or whatever one of the myriad of the details diff based tools neglect or ignore. The main disadvantage is the the scripts have to be written, which require real T-SQL knowledge (forget all the 'designers' in you favorite management tool).
You might want to check out RedGate SQL Source Control.
Are you looking for Visual Studio Database Projects?
I use database projects to store all database objects (tables, views, functions, keys, triggers, indexes across schemas) and keep versioning in TFS. You can build the database to ensure that everything is valid. You can deploy to a fresh database, or do a schema comparison with an existing database.
I also keep all reference and setup data in post deployment scripts which are automatically run after deployment.
I'm interested to find out what would be the good way to make changes to production database and source code in web application (ASP.NET, SQL Server 2008).
A little bit more details, we develop on local machines, and then we need to transfer the code and database changes to production (pretty much standard story).
At the moment we do it in the evening, change the database directly from management studio on production server, and then just overwrite the existing asp.net code (copy/past).
You're talking about Release management. What you're asking about is a big subject with a LOT of different answers. The best solution for you is not something we can tell you. There are trade offs to consider.
For example, what you're describing is a very basic release management process that would be considered an "immature" process.... It does not take into account rollback plans, versioning, separation of concerns, proper testing, or any of a hundred other factors that a "mature" release management process involves.
A mature process is very good, but if you don't have the resources, it's not feasible.
To get to the point, I don't think you question can be answered fully here. I'd suggest starting to research "change management", "release management", "Application Lifecycle management", and "Applicaiton Development Lifecycle". I'll have a few good starter links for you in a minute.
Just a forewarning, though, you are asking a question that's going to open your eyes and your world in ways you probably haven't considered. There are things like automated builds to consider, tools to do it for you (high priced, free, and everything in between)
http://en.wikipedia.org/wiki/Release_management
http://en.wikipedia.org/wiki/Application_lifecycle_management
A few simple options for JUST what you're asking about can be found here:
http://msdn.microsoft.com/en-us/library/7hd4c0x3(VS.80).aspx
Also, since you talked about source code without mentioning which source control you're using, I need to say... if you're not already using source control, you need to. You'll wonder how you ever lived without it once you start using it.
Depends on whether it's the first deployment of a new app, or an update to the app.
For small updates, record all your database changes as sql scripts. You must strictly enforce that all changes to development are applied as sql scripts. Put the scripts in source control. Deploy the update by running the scripts on production.
For new apps you may have thousands of scripts. You can't run them individually. Consolidating them into a master script takes too much time. (although you still want to check EVERY script into source control). In this case you reach a milestone in development then FREEZE the development database, and declare it a baseline. Use the database tools to generate a master script(s). Deploy production by running this script(s). Manually create data scripts for your lookup tables to keep it separate from junk dev data.
Avoid a database copy. Avoid changing by hand through the GUI. Scripts are the way. How you go about collecting the scripts, consolidating to master scripts, generating the scripts, etc is another story.
All,
My dev team and I would like to setup a development environment for our ASP.NET projects. BY development environment i do not mean Visual Studio. I mean, that we have a Database Server, a Application Server and a Web Server in a 'Development Environment'.
We want to use this as our integration environment. Where the developers all work on there parts of ASP.NET Applications and then we can push our new changes up to test them as a whole.
My Question is , what is the best way to deploy our code together without stepping on our toes?
Thanks.
Team Foundation Server is a good candidate for this.
You need a source code control methodology and with it you'll get the benefits you're searching for. SVN and other solutions in this space offer "conflict resolution" to avoid inadvertent overwriting/toe squashing.
Setup a subversion repository, get all of the developers up to speed on svn and using it.
Once you have your source under control you can consider setting up a continuous integration server which can build your code and deploy to your target environment in batch. Organizing your project code properly into trunk, tags and branches per solution will make it very easy to control what is deployed or redeployed to your dev environment at any given time.
There are other options for source code control (git, tfs, and many others) but they all offer close to the same features... SVN is one of the nicer options because it's open source, free and stable.
Another thing to consider is keeping your database schema changes in sync with your code changes. Consider using migrator.net or similar solution to enable your team to keep everything in sync through revisions, including database state.
Ok, so here's the thing.
I'm developing an existing (it started being an ASP classic app, so you can imagine :P) web application under ASP.NET 4.0 and SQLServer 2005. We are 4 developers using local instances of SQL Server 2005 Express, having the source-code and the Visual Studio database project
This webapp has several "universes" (that's how we call it). Every universe has its own database (currently on the same server) but they all share the same schema (tables, sprocs, etc) and the same source/site code.
So manually deploying is really annoying, because I have to deploy the source code and then run the sql scripts manually on each database. I know that manual deploying can cause problems, so I'm looking for a way of automating it.
We've recently created a Visual Studio Database Project to manage the schema and generate the diff-schema scripts with different targets.
I don't have idea how to put the pieces together
I would like to:
Have a way to make a "sync" deploy to a target server (thanksfully I have full RDC access to the servers so I can install things if required). With "sync" deploy I mean that I don't want to fully deploy the whole application, because it has lots of files and I just want to deploy those new or changed.
Generate diff-sql update scripts for every database target and combine it to just 1 script. For this I should have some list of the databases names somewhere.
Copy the site files and executing the generated sql script in an easy and automated way.
I've read about MSBuild, MS WebDeploy, NAnt, etc. But I don't really know where to start and I really want to get rid of this manual deploy.
If there is a better and easier way of doing it than what I enumerated, I'll be pleased to read your option.
I know this is not a very specific question but I've googled a lot about it and it seems I cannot figure out how to do it. I've never used any automation tool to deploy.
Any help will be really appreciated,
Thank you all,
Regards
Have you heard of the term Multi-Tenancy? It might be worth look that up to see if that applied to your "Multiverse" especially if one universe is never accessed by another...
See:
http://en.wikipedia.org/wiki/Multitenancy
http://msdn.microsoft.com/en-us/library/aa479086.aspx
UPDATE:
If the application and database is the same for each client (or Tenant) I believe there are applications that may help in providing the same code/db as an SaaS application? ie another application/configuration layer on top that can handle the deployments etc?
I think these are called Platform as a Service (PaaS) applications:
see: http://en.wikipedia.org/wiki/Platform_as_a_service
Multi-Tenancy in your case may be possible, depending on client security requirements, with a bit of work (or a lot of work):
Option 1:
You could use the one instance of the application, ie deploy the site once and connect to a different database for each client. You would need to differentiate each client by URL to isolate content/data byt setting a connection string for each etc. (This would reduce your site deployments to one deployment)
Option 2:
You could create both a single instance of the application and use a single database. You would need to add a "TenantID" to each table and adjust all your code to accept a TenantID to ensure data security/isolation. Again you wold need to detect/differentiate the Tenant based on the URL to set the TenantID for the session used for every database call. (This would reduce your site and database deployment to one of each)
We have been a ColdFusion shop for 10 years, and are now switching over to ASP.net MVC. Our target framework is .net 4.0 BETA 2 using VS 2010 BETA 2. We set up two instances of Windows Server 2008 (staging and production), and will be using our existing database server (SQL Server 2008).
None of us really have much experience in ASP.net itself, though we are all very comfortable in C# and the MVC pattern. The coding itself isn't much of an issue; but the deployment process is. Our goal is to be able to have a CI setup that will automatically pull down, and test, our applications into staging on commit - then have the option to tag, then switch, the checkouts on our production sites when websites pass QA.
Some of the things I'm having issues with here is the concept of an ASP.net application and how it integrates into SVN. CF, like PHP or RoR, are all scripting languages and as such require no build process (checking out the source into production is very straightforward). But in this case, applications need to be compiled - which is where we start to have problems. Will we need to create another server (or use an existing one) that has some sort of application that pulls down code, compiles it, then somehow pushes it on the live servers? If so, what is considered the best way to accomplish this? I imagine if we end up using a build tool such as Nant, adding additional steps to migrate the database would be trivial, but what is the best way to accomplish this as well?
Another, slightly unrelated, problem is how our designers will work with our code. Most of them are on Macs, and using VS isn't much of an option. How will they be able to edit the aspx, css and image files easily? Our goal is to make this as transparent as possible to them.
We have done a lot of shopping around, and ASP.net MVC seems to be the best option as far as our familiarity with the language, and our current platform. We just need to figure out a good build process so everything is as transparent as possible. I understand there are a ton of resources available on this, but I wanted to get the opinions of the people here from first-hand experience.
Microsoft TFS has a wonderful build solution built-in. It's costly, but effective. In addition, you cannot lose by looking at CruiseControl, which is free. TeamCity from JetBrains is also a great option. All of these Continuous Build and Integration solutions would provide a good starting point for your research.
http://msdn.microsoft.com/en-us/teamsystem/dd408382.aspx
http://www.cruisecontrol.com/
http://www.jetbrains.com/teamcity/
Even Draco.net is a good consideration:
http://draconet.sourceforge.net/
We use http://www.cruisecontrol.com/ (CC) running on our SVN / Build server. You can configure CC via it's own config/script files to pull down the latest source from SVN and then spawn one or more Nant or MSBuild scripts which can perform your build and deployment.
We script all of our database changes into change scripts which also go into SVN. We then have a custom command line tool which will deploy the change scripts to SQL Server during the web site deployment. All of that is done in the Nant script.
So each project's Nant script handles the build, web site deployment and SQL change script deployment.
The tricky part is handling rollbacks if/when something goes horribly wrong. I would suggest posting another question for that specific problem.