Is it possible to build an ASP.NET website using EF where each customer logging in has separately stored data? We have customers demanding that their data won’t be stored in the same tables as other customers’ data.
I’ve read that EF can’t work with several databases but is it possible to switch database at runtime depending on input parameters? I have a feeling it won’t be possible since the migration features are tightly connected to the database being used, but I'm not sure.
One solution could be to have a separate website deployment and database for each customer. They’ll get separate domains to access but that’s not a problem. But this solution feels a bit clumsy if you’re having many customers, especially with deployment and future upgrades.
Am I missing some smart ways of solving this or is this a very tricky issue?
is structure (of the db) the same ?
if so you could switch connections - not w/o issues though, but should work. For details on how that should be done check the long discussion we've had here (and linked previous questions etc.)...
Code first custom connection string and migrations without using IDbContextFactory
Related
Team A has an enterprise app that uses ADO.NET for data access that executes stored procedures. The data access is encapsulated in it's own project (let's call it DAL.dll)
Team B is creating another unrelated app that's reusing the stored procedures in the enterprise app. This app is currently using the MS application block for data access. The issue we run into is that whenever Team A make any change to the input/output params in the stored procedures, there is a runtime error in Team B's app and this app needs to be updated to accommodate the additional params (or params that were removed). So, most of these go unnoticed until a user complains. At the very least, we would like to have the app throw a compilation error so that the build process warns us of the changes made.
One way to do this is to have Team B's project add a reference to the DAL.dll
I'd like to know if there are any other cleaner ways of solving the issue. We are ready to replace Team B's MS Data application block to use a different technology (Entity Framework?) if necessary.
Among the other answers, I'd strongly suggest getting those stored procedures into source control, in a Database Project. You then may be able to use the features of your source control system to do several things:
Lock some of the code so that it cannot be changed
Give you notifications if the code is changed
Warn you if the stored procedures change in a way that would prevent them from being called
Branch the stored procedures so that each team can have their own version of changed code, while keeping the unchanged stored procedures common. You of course will need to separate the different versions in the database.
I agree with the other posters on this thread that you should not share stored procedure's across different .NET DLL's, that is just a recipe for disaster. I would also shy away from ORM's like Entity Framework if you are doing anything at all complicated with your database schema because ORM's excel at getting a simple object model translated from your .NET application classes into SQL tables and SP's, but traditionally do poorly at optimizing them for performance on the database side. There will be people who claim otherwise, and they may have a valid point if you are an expert in wrangling an ORM to do waht you want like they are, but chances are you are not and it will cause you headaches in the long run.
A shared data access layer might work, but conceptually you are then just changing the implementation of the dependency from some code that a DBA wrote to some code that a .NET programmer wrote. Yes, you can use integration tests to achieve better verifiability, but the same case could be made for SQL with tools like Red Gate's SQL Test. I would shy away from this approach if the two applications are already experiencing some sort of pain from sharing SP's. That is an indication that the dependency just should be done away with.
If it were up to me, I'd just make a new schema for Team B's app. You can read more about schemas in SQL Server here: MSDN Schema description for 2008 R2. You can think of them as namespaces for SQL Server but with some additional bells and whistles like permission and access control. Separating out your different applications into separate schemas on the same shared database will probably make for the most flexible implementation in the long run.
unrelated app that's reusing the stored procedures in the enterprise app
If these two application are really unrelated why are those sharing procedures or even the same database. I know this is a long read, but I recommend you to read this: A Better Path to Enterprise Architectures
The partioning concept in there relates to the bounded context in Domain driven design:
Multiple models are in play on any large project. Yet when code based on distinct models is combined, software becomes buggy, unreliable, and difficult to understand. Communication among team members becomes confusing. It is often unclear in what context a model should not be applied.
Therefore: Explicitly define the context within which a model applies. Explicitly set boundaries in terms of team organization, usage within specific parts of the application, and physical manifestations such as code bases and database schemas. Keep the model strictly consistent within these bounds, but don’t be distracted or confused by issues outside.
It is expected you end with problems when you don't explicitely deal with this. You're lucky you're seeing early failures, as it can turn into problems much harder to find on the long run.
Analyze the problem again with the above in mind. Consider if you're missing some explicit context where this common functionality should live.
My question is: which team owns the store procedured and the database shared? Usually as a good architecture/design, you should not have two different apps sharing same database / procedures.
A better way to share data/functionality between two different applications is through a services or API, so the team who owns the functionality would be responsible to maintain it.
Also, have a good communication between both teams is highly recommend.
Depending on the owner of the DAL project, you could host web services and share the API. That way, you separate the Data Access Layer from the business logic, which allows anyone to use the same DAL without having to publish it to each different location.
From my point of view, it looks like both Team A and Team B should share the same core model and look at Multitier architecture as a possible solution.
It sounds like it would make sense to create a shared DAL that both applications can share.
I would add unit tests (or really integration tests) to make sure the DAL is compatible with the apps after changes. That way your tests would fail if incompatible changes have been made
"I'd like to know if there are any other cleaner ways of solving the issue."
The cleanest way is for Team B to sit down with Team A and encapsulate the relevant business logic into a shared API. It doesn't matter so much how you implement that API; what does matter is that the API's interface is documented and versioned so everyone knows what to expect.
One reasonable mechanism for this in a .NET environment is to use Microsoft's WebAPI.
In short, the question of "how do we share a stored procedure?" is most likely looking at the wrong level of abstraction.
I'd like to know the best architecture.
We have a web application running different domains. Each domain has its own MySQL database but the db structure is the same for all of them.
We have a web application for the visible part of the application.
We have a dataLogic project
We have a dataEntities project
We have a dataAccess that contains only the methods to connect to the data base.
Before we called stored procedures on a database. But we had to change it because the performance was bad. Also, the problem was that every change we made we had in a stored procedure we had to copy to every database.
We are thinking in using a WebService to retrieve the data. Every domain can call the web service with a connection string and connect its database to retrieve data. This way when we change a SQL query we only have to compile the webService and change it, we don't have to change versions on multiples domains.
Also, what do you think about the SQL queries? Since we don't want to keep using stored procedures, what is the best way to do it? Directly from code?
Thanks
T
If you have multiple Database servers you will have to make Structural changes from one DB to another one way or another. There are many tools to change Database structures. These tools will look for differences between Schema, and will either generate the SQL code for you, or do the changes by itself (it depends a lot in the tool, there are powerful ones and not so powerful ones). Please do take a look at Toad for MySql. Now, for the Data changes, you may want to replicate the data from one Database to another. This is done through Replication.
We are thinking in using a WebService to retrieve the data. Every
domain can call the web service with a connection string and connect
its database to retrieve data.
This sounds like a good idea and since you already have "dataAccess" and "dataLogic" projects, it should not be too hard to make the services.
Also, what do you think about the SQL queries? Since we don't want to
keep using stored procedures, what is the best way to do it? Directly
from code?
I don't think it is a good practice to have the SQL queries directly into your code, but it depends in a lot of things, so I would suggest Stored Procedure vs Hard-Coding the queries, or LinQ (Entity Framework 4.1).
Good luck with your project and I will take a look at this thread frequently to see what you end up doing.
Have fun!
Hanlet
I have three different applications, they all share the ASP.NET membership aspect of the database and almost definitely they won't share anything else.
Should I have a separate database for each of the applications, or would one suffice?
All the application tables are prefixed, so that wouldn't be a problem in integration. Although I was wondering if there would be any performance issues, or if having all three applications share the same database would be some kind of grave mistake.
The applications in question are three web applications, the "main site", a forum and a bug tracker. I'm wondering if this is viable because integration could be easier if I had a single database. For instance, the bug tracker registers asp.net membership tables in it's db connection, and it even creates an "admin" user, where the db that is actually supposed to be holding the membership tables would be the "main site" one.
Update: I added a bounty to this question since the answers seem to have pretty split opinions about whether I should or not use multiple databases for different applications that share only membership providers.
Separate apps = separate databases - unless you have to "squeeze" everything into a single DB (e.g. on a shared web hoster).
Separate databases can be backed up (and restored!) separately.
Separate databases can be distributed onto other servers when needed.
Separate databases can be tweaked individually.
I have always found it would be better to have more databases so that it is easier to:
Migrate to more servers if needed
Manage security / access easier
Easier (and Faster) restores and backups
I would actually go with four databases. A Membership database, and then one for each application (if the membership is truly shared). This will allow you to lock security across applications as well.
Looking at your question closer... You say that the data would "likely not be shared"... will a lot of your queries be joining tables with the membership? If so, might be easier if they are in the same database. However if you are going with a more entity based approach, I would think you would still be better with multiple databases. You might even want to look at something like an LDAP database or some other type of caching for your membership database to speed things up.
You should use the same database unless you have a current need to place them in separate databases - HOWEVER where possible you should architect your system so that you could move the data into a separate database should the need arise.
In practise this means that you should keep SQL procedures working the smallest amount of data possible - i.e. Don't have multi-step stored procs which do lots of separate actions. Have separate usps and call each from code.
Reasons to use separate databases:
1) Unrelated data - Group data that is interrelated - andonce databases get beyond a certain complexity, look to separate out blocks of related data into separate databases in order to simplify.
2) Data that is of either higher importance (e.g. Personal Details) should be separated to allow for greater security measures: e.g. screening this data from developers
3) or lower importance (e.g. Logging Info) - this probably does not need backing up - and if it's particularly volumous, you probably don't want it increasing the time taken to back up the main site database.
4) Used by applications living on different servers at different locations. Quite obviously you want to site data as close as possible to the consuming application.
Without really knowing the size and scale of your system, difficult to give full opinion, if it's just your own site, one db may work for now - if it's commercial then i'd have 4 dbs from the word go: Membership details, Forum, Bug Tracker and MainSite related stuff.
Thus in code you would have a Membership manager which only talks to the Membership db, A BugManager, A ForumManager and anything else will only talk to the MainSite db. I can't think of any reason you'd need any of these databases talking to each other.
Just my inclination: although the three apps might not share much (not yet, anyway: but what happens when a forum post wants to reference a bug report?), they all belong to the same "system," so to speak.
I would definitely put all of the tables in just one database.
In my opinion , it is better to split the database for increased flexibility, security, efficiency, and scalability.
In future if there is any addition of requirement (you never know) which is common to all the three applications , it might be a little difficult to maintain.
For example: User login /audit trace for your 3 applications.
It may sound like I'm wandering a bit, but have you taken into account another possibility, that is separating all the authentication/membership functionality into an application itself?
From your description it seems you may add another application in the future. It would start to look like a network of sites, much like 37signals web apps, Google web apps or MSN web apps.
And thus, you may go for a kind of Single-Sign-On / Connect service. This one single application may offer authentication methods via web-services or any other mechanisms, it will have its own DB for you to tweak, modify, backup and move without affecting the other apps. I myself have found this situation many times and thus I love how easy is to share your Google or Facebook login among applications.
Perhaps I'm seeing it from a little higher perspective than yours, sorry if it's the case. If this is not an option, you may keep 4 databases: 1 for each application and 1 for the membership provider, which has its own connectionstring most of the time.
Of course it depends on the size of your applications' footprint on DB-level. 10 tables per app is OK, 150 tables per app would make the DB a little ugly to us, that being a personal preference.
Good luck with whatever option you choose.
The membership framework allows for partitioning across multiple applications, so you probably should have the following configuration:
Membership Database
Application 1 Database
Application 2 Database
Application 3 Database
Then, in each of the application databases, create synonyms that point to the membership database's tables for when you need to write your own queries that access both application data and membership data. Synonyms are easy to maintain and allow you change where the database is without changing any dependencies on those tables as the synonym names don't change.
Your application configuration in Web.config will determine how the data is partitioned in the membership database as you specify an ApplicationName that should be different for each app.
We have an ASP.NET app with SQL Server & it is a photo & video sharing site.
Details of photos and videos are stored in tables & the files are in the file system.
Database has 75 tables and 225 stored procedures. The app will be ready for production deployment within next 6 months.
Due to longer time growth concerns, we decided to switch to NoSQL (MongoDB) database.
We have few questions regarding the best way to approach this:
Is it better to deploy the app with SQL Server backend and migrate to NoSQL later?
OR re-architecture now and rewrite/recreate database, tables, procedures and data layer
How difficult will it be re-architecture/recode with MongoDB? Any tools or BKMs?
EDIT:
Our app is Youtube+Flickr type site where user will share photos and videos with lots of comments, tags and ratings (photo\video & comments).
Is NoSQL a better database to move to? Reason for moving: cost + read query speed
Please help me with you valuable advise.
Thank you very much.
Change is always exponentially more expensive the later it is introduced to a project. This is a core principle of software engineering. You should do this now.
That said, I question your long-term vision. Relational databases, used properly, have a lot of performance in them.
This question raises more questions than answers.
Have you benchmarked your current implementation in terms of requests/responses?
Why MongoDB out of all possible NoSQL databases? (Don't get me wrong, I love Mongo, but love and hype should not weigh in technology choices)
Are you certain you will get the large userbase you're expecting? Why are you so certain?
Using stored procs seems to tip off that you aren't using an ORM? Why not?
Generally, I'm against these types of re-architectures. Firstly, you need to get your whole team acclimated to how Mongo affects development. Secondly, your ops team needs to get acclimated to how to deploy and maintain a Mongo installation. More likely than not, this will prevent you from launching in a timeline you want to launch.
I'd say that you should probably launch as is, fix the ORM part if you aren't using one, benchmark your app, benchmark a prototype of your app backed by Mongo and if the performance advantages are so big that it warrants the pain of re-architecture do it.
To your latter question, there aren't any tools right now, as far as I can tell, that'll automate or semi-automate the database import/export from SQL Server to Mongo. There are barely tools to do that for MySQL.
I've done such a migration a few month ago, during the early developement stage of a website in ASP.NET. It was a hard decision, but I could concentrate on that migration. The reason why I did this migration was the ORM that I couldn't trust anymore and some very slow queries that I had no idea how to optimize.
During coding phase, what I figured out was : I was spending a lot of time with the data model in SQL Server (using Entity) and all the plumbery code.
Now, no more store procedures (C# and Linq code instead), no more 2 layers to maintain (the code is the model).
My small experience says : The earlier the better but don't get me wrong, before migrating you really have to think in Document rather than in RDBMS. This means you may have to partially change the businness DataModel to correctly utilize MongoDB features, otherwise you could get bad performances and Mongo DB is useless for bad models.
Another point is the admin stuff. You'll have to quickly learn Mongo DB admin to be up to speed. And even if the tools are good, they completely differ from SQL Server tools.
In conclusion, If you're convinced MongoDB is your future data store and search database,
(and it was in my case), read documentation, take time to do some Proof Of Concept. Then you can think Document and load test you new model.
Your core question appears to be whether to make the switch to MongoDB now, or deploy on SQL and go to MongoDB in a future release.
You do not appear to be using an ORM (e.g. NHibernate, Entity Framework.) Setting other concerns aside, if you're convinced that you want to go to NoSQL, then I would do it now rather than later. Unless you integrate a Provider model for your data access, changing the underlying data access strategy after it is already established would be difficult.
I agree. Switching now is better, if only to avoid the data migration headache switching post-deployment will require.
I am in the early stages of planning a conversion of a large classic ASP database application to ASP.Net and I'm having trouble picking out which data access method to use. I have played around with Linq To SQL, Dynamic Data, strongly typed datasets, Enterprise Library (Data Access Application Blocks), and a tiny bit with Entity Framework, but none of them have jumped out to me as "the one". There are just too many choices - my head is swimming, help me choose!
Perhaps it would help to give some background on the application that I am converting along with the priorities...
The back end is Microsoft SQL Server (2005 or later) and we are committed to that, so I don't need to worry about ever supporting a different database platform.
The database is very mature and contains a great deal of the business logic. It is highly normalized and makes extensive use of stored procedures, triggers, and views. I would rather not reinvent two wheels at the same time, so I'd like to make as few changes to the database as possible. So, I need to choose a data access method that is flexible enough to let me work around any quirks in the database.
The application has many data entry forms and extensive searching and reporting capabilities (reports are another beast which I will tackle later).
The application needs to be flexible enough to deal with minor changes to the database structure. The application (and database) may be installed at different sites where minor custom modifications are made to the database. Ideally the application could identify the database extensions and react appropriately. In other words, if I need to store an O/R mapping in the application, I need to be able to swap that out (or refresh it easily) when installing the application and database at a new site.
Rapid application development is critical. Since the database is already done and the user interface is going to closely match the existing application, I'm hoping to find something where we can crank this out fairly quickly. I am willing to sacrifice not using the absolute latest and greatest technology if it will save time in development. In other words, if there is a steep learning curve to using something like Entity Framework, I'm fine with going something like strongly typed Datasets and a custom DAL if it will speed up the process.
I am a total newbie to ASP.Net but am intimately familiar with Classic ASP, T-SQL and the old ADO (e.g. disconnected recordsets). If any of the data access methods is better suited for someone coming from my background, I might lean in that direction.
Thanks for any advice that you can offer!
Look at all three articles in this series:
High Performance Data Access Layer Architecture Part 1
Great advice.
You may want to look at decoupling the database layer from the asp layer so that you can not only give more flexbility in making the decision, but when you have to make changes to a customer's database you can just swap in a new dll without changing anything else.
By using dependency injection you can use xml to tell the framework which concrete class to use for an interface.
The advantage to doing this is that you can then go with one database approach, and if you later decide to change to another, then you can just change the dll and go on without making any changes to other layers.
Since you are more familiar with it why not just go directly to the database at the moment by making your own connections? Then you can move the rest of your code and along the way you can decide which of the myriad of technologies to use.
For a new application I am working on I am starting with LINQ to SQL for it, mainly because development will be quicker, but, later, if I decide that won't meet my needs I will just swap it out.
nHibernate might be a good fit. You can store the mapping in external configuration files which would solve your needs. Another option might be using ActiveRecord, which is based upon nHibernate.
nHibernate has a neat feature which you might find helpful. It's called a Dynamic property which is basically a name value pair collection populated by pulling the column names from the mapping file. So when you add a column at your client site, you update the mapping file and you'd be able to access the data through a collection on the object.