I have a web site I developed for displaying the results of some data analysis work I did. It relied on ASP.NET for the front end and connected to a MySQL back end utilising Entity Framework and LINQ extensively.
I chose MySQL because I personally have used it in the past and like the database, but this resulted in some serious issues when I had to deploy it to a hosting provider (incompatible connectors, access rights, etc.)
I am now getting ready to redevelop and expand the site and I am looking for some advice to avoid the issues I had last time.
The new DB has to serve two roles. The first is to be a data provider for the charts that are the output of the analysis work. These tables are straightforward, almost flat files, with 10 tables. One table has roughly 200k rows of data the rest have aprox 1200 rows of data each. There are little references or queries between the DB tables, but there are a few. This data is updated periodically by a back end process and does not need to be added to or edited by the user.
The second role of the DB would be as a basic persistent store for a standard user management system. It would need to manage data for adding/ removing clients, user names, passwords, access rights. etc. No financial data or super secure data is involved.
What database approach would you recommend that would give me easy deployment and management at a web host and still allow me to use both Entity Framework and LINQ effectively.
Second, what tools/frameworks should I consider as I rewrite this system. It is very graphical and data focused. Presentation of charts and information is the key factor in this site. Are there any new technologies or frameworks that would add specific value to what I am doing?
A few notes. I am a one man shop and I maintain the entire system myself so I am less worried about enterprise level frameworks than other people. My focus is on the easy development and deployment of the site. Maintainability is also a key factor.
I am also an experienced C# developer, but new to ASP.NET and the web side of things. The first version of this site was a big learning experience. It was good, but I wasted an enormous amount of time on just understanding new technologies and approaches. I am very open to learning, but I can't afford the time to get my head around a complete paradigm shift.
I am looking forward to your thoughts, thanks.
Doug
The natural choice would be SQL Server. I'd guess by your description that you are way under the maximum space limit of the SQL Server Express edition. I of course supports Entity Framework and the drivers are part of the .NET Framework, so no problem with third party assemblies here.
This will also open up the possibility to host your app in the cloud (Azure) later on, because SQL Azure in fact is a Microsoft SQL Server, so there is no overhead in supporting that.
Regarding user management - ASP.NET has this all build in (Membership, Role and Profile provider) and also a SQL Provider for which default tables are available. So you don't have to design your tables by yourself and it runs very naturally on SQL Server.
Related
I am about to work on an LOB application for my customer. The asp.net web application has to be hosted on their on-premise Sharepoint server. In the past, I have created data entry forms using Infopath and designed workflows using Sharepoint designer tool but I have limited amount of experience on sharepoint development.
Project requirement:
1. An input screen for the users to enter data that will be stored in a data store such as SQL Server?
2. A dashboard to indicate the status of their submitted requests.
3. Application has to be hosted on Sharepoint server (on-premise at the moment). But I would like to know what if at later point, they decide to move it to Office 365 (cloud)? Will it be easier to migrate the application including any workflows that are created?
What I need advise on? Here are some of my questions:
Data store (SQL Server vs Sharepoint list) - Which one is better? For a typical LOB application such as this one, will it make sense to store the data in SQL Server or Sharepoint list?
Performance - Will it overtime degrade the performance of the application if the data is stored in Sharepoint list instead of SQL Server? What would be an ideal choice? Is Sharepoint list used as a data store for some very specific scenarios only and what are those scenarios? Amount of this application’s data is expected to grow as the time goes by. Probably, say there will be almost 30000 records added each month and may be more later?
Security - Is Sharepoint list a secure option for storing data? I know that a sharepoint admin can provide access to a sharepoint list if anyone desires that would enable people to modify the structure of the list or mess up with the data through sharepoint.
As i have limited experience on sharepoint, I need advise on whether to use list or SQL Server to store the data.
If your LOB Application has complex structure like tables with very relations and two or more linked table by relation , SharePoint will got you in trouble.
If you like to make summary report from all tables with complex join , it's better to use SQL Server.
Simple answer is SharePoint development with multiple linked tables is hard and software maintenance is hard too.
Performance: SharePoint list with very large of data (above 500000 record) will be slow depend on hardware on server performance , But you can use large list patterns if you like this url Link.
SharePoint list security architecture is very well, and there is know any concern for this subject and each type of security pattern can apply to data.
I have three different applications, they all share the ASP.NET membership aspect of the database and almost definitely they won't share anything else.
Should I have a separate database for each of the applications, or would one suffice?
All the application tables are prefixed, so that wouldn't be a problem in integration. Although I was wondering if there would be any performance issues, or if having all three applications share the same database would be some kind of grave mistake.
The applications in question are three web applications, the "main site", a forum and a bug tracker. I'm wondering if this is viable because integration could be easier if I had a single database. For instance, the bug tracker registers asp.net membership tables in it's db connection, and it even creates an "admin" user, where the db that is actually supposed to be holding the membership tables would be the "main site" one.
Update: I added a bounty to this question since the answers seem to have pretty split opinions about whether I should or not use multiple databases for different applications that share only membership providers.
Separate apps = separate databases - unless you have to "squeeze" everything into a single DB (e.g. on a shared web hoster).
Separate databases can be backed up (and restored!) separately.
Separate databases can be distributed onto other servers when needed.
Separate databases can be tweaked individually.
I have always found it would be better to have more databases so that it is easier to:
Migrate to more servers if needed
Manage security / access easier
Easier (and Faster) restores and backups
I would actually go with four databases. A Membership database, and then one for each application (if the membership is truly shared). This will allow you to lock security across applications as well.
Looking at your question closer... You say that the data would "likely not be shared"... will a lot of your queries be joining tables with the membership? If so, might be easier if they are in the same database. However if you are going with a more entity based approach, I would think you would still be better with multiple databases. You might even want to look at something like an LDAP database or some other type of caching for your membership database to speed things up.
You should use the same database unless you have a current need to place them in separate databases - HOWEVER where possible you should architect your system so that you could move the data into a separate database should the need arise.
In practise this means that you should keep SQL procedures working the smallest amount of data possible - i.e. Don't have multi-step stored procs which do lots of separate actions. Have separate usps and call each from code.
Reasons to use separate databases:
1) Unrelated data - Group data that is interrelated - andonce databases get beyond a certain complexity, look to separate out blocks of related data into separate databases in order to simplify.
2) Data that is of either higher importance (e.g. Personal Details) should be separated to allow for greater security measures: e.g. screening this data from developers
3) or lower importance (e.g. Logging Info) - this probably does not need backing up - and if it's particularly volumous, you probably don't want it increasing the time taken to back up the main site database.
4) Used by applications living on different servers at different locations. Quite obviously you want to site data as close as possible to the consuming application.
Without really knowing the size and scale of your system, difficult to give full opinion, if it's just your own site, one db may work for now - if it's commercial then i'd have 4 dbs from the word go: Membership details, Forum, Bug Tracker and MainSite related stuff.
Thus in code you would have a Membership manager which only talks to the Membership db, A BugManager, A ForumManager and anything else will only talk to the MainSite db. I can't think of any reason you'd need any of these databases talking to each other.
Just my inclination: although the three apps might not share much (not yet, anyway: but what happens when a forum post wants to reference a bug report?), they all belong to the same "system," so to speak.
I would definitely put all of the tables in just one database.
In my opinion , it is better to split the database for increased flexibility, security, efficiency, and scalability.
In future if there is any addition of requirement (you never know) which is common to all the three applications , it might be a little difficult to maintain.
For example: User login /audit trace for your 3 applications.
It may sound like I'm wandering a bit, but have you taken into account another possibility, that is separating all the authentication/membership functionality into an application itself?
From your description it seems you may add another application in the future. It would start to look like a network of sites, much like 37signals web apps, Google web apps or MSN web apps.
And thus, you may go for a kind of Single-Sign-On / Connect service. This one single application may offer authentication methods via web-services or any other mechanisms, it will have its own DB for you to tweak, modify, backup and move without affecting the other apps. I myself have found this situation many times and thus I love how easy is to share your Google or Facebook login among applications.
Perhaps I'm seeing it from a little higher perspective than yours, sorry if it's the case. If this is not an option, you may keep 4 databases: 1 for each application and 1 for the membership provider, which has its own connectionstring most of the time.
Of course it depends on the size of your applications' footprint on DB-level. 10 tables per app is OK, 150 tables per app would make the DB a little ugly to us, that being a personal preference.
Good luck with whatever option you choose.
The membership framework allows for partitioning across multiple applications, so you probably should have the following configuration:
Membership Database
Application 1 Database
Application 2 Database
Application 3 Database
Then, in each of the application databases, create synonyms that point to the membership database's tables for when you need to write your own queries that access both application data and membership data. Synonyms are easy to maintain and allow you change where the database is without changing any dependencies on those tables as the synonym names don't change.
Your application configuration in Web.config will determine how the data is partitioned in the membership database as you specify an ApplicationName that should be different for each app.
We have an ASP.NET app with SQL Server & it is a photo & video sharing site.
Details of photos and videos are stored in tables & the files are in the file system.
Database has 75 tables and 225 stored procedures. The app will be ready for production deployment within next 6 months.
Due to longer time growth concerns, we decided to switch to NoSQL (MongoDB) database.
We have few questions regarding the best way to approach this:
Is it better to deploy the app with SQL Server backend and migrate to NoSQL later?
OR re-architecture now and rewrite/recreate database, tables, procedures and data layer
How difficult will it be re-architecture/recode with MongoDB? Any tools or BKMs?
EDIT:
Our app is Youtube+Flickr type site where user will share photos and videos with lots of comments, tags and ratings (photo\video & comments).
Is NoSQL a better database to move to? Reason for moving: cost + read query speed
Please help me with you valuable advise.
Thank you very much.
Change is always exponentially more expensive the later it is introduced to a project. This is a core principle of software engineering. You should do this now.
That said, I question your long-term vision. Relational databases, used properly, have a lot of performance in them.
This question raises more questions than answers.
Have you benchmarked your current implementation in terms of requests/responses?
Why MongoDB out of all possible NoSQL databases? (Don't get me wrong, I love Mongo, but love and hype should not weigh in technology choices)
Are you certain you will get the large userbase you're expecting? Why are you so certain?
Using stored procs seems to tip off that you aren't using an ORM? Why not?
Generally, I'm against these types of re-architectures. Firstly, you need to get your whole team acclimated to how Mongo affects development. Secondly, your ops team needs to get acclimated to how to deploy and maintain a Mongo installation. More likely than not, this will prevent you from launching in a timeline you want to launch.
I'd say that you should probably launch as is, fix the ORM part if you aren't using one, benchmark your app, benchmark a prototype of your app backed by Mongo and if the performance advantages are so big that it warrants the pain of re-architecture do it.
To your latter question, there aren't any tools right now, as far as I can tell, that'll automate or semi-automate the database import/export from SQL Server to Mongo. There are barely tools to do that for MySQL.
I've done such a migration a few month ago, during the early developement stage of a website in ASP.NET. It was a hard decision, but I could concentrate on that migration. The reason why I did this migration was the ORM that I couldn't trust anymore and some very slow queries that I had no idea how to optimize.
During coding phase, what I figured out was : I was spending a lot of time with the data model in SQL Server (using Entity) and all the plumbery code.
Now, no more store procedures (C# and Linq code instead), no more 2 layers to maintain (the code is the model).
My small experience says : The earlier the better but don't get me wrong, before migrating you really have to think in Document rather than in RDBMS. This means you may have to partially change the businness DataModel to correctly utilize MongoDB features, otherwise you could get bad performances and Mongo DB is useless for bad models.
Another point is the admin stuff. You'll have to quickly learn Mongo DB admin to be up to speed. And even if the tools are good, they completely differ from SQL Server tools.
In conclusion, If you're convinced MongoDB is your future data store and search database,
(and it was in my case), read documentation, take time to do some Proof Of Concept. Then you can think Document and load test you new model.
Your core question appears to be whether to make the switch to MongoDB now, or deploy on SQL and go to MongoDB in a future release.
You do not appear to be using an ORM (e.g. NHibernate, Entity Framework.) Setting other concerns aside, if you're convinced that you want to go to NoSQL, then I would do it now rather than later. Unless you integrate a Provider model for your data access, changing the underlying data access strategy after it is already established would be difficult.
I agree. Switching now is better, if only to avoid the data migration headache switching post-deployment will require.
Is it better for a collection of ASP.NET web apps to share the same session database, or should each one have its own?
If there is no significant difference, having a single database would be preferable due to easier maintenance.
Background
My team has an assortment of ASP.NET web apps, all written in either Monorail 1.1 or ASP.NET MVC 1.0. Each app currently uses a dedicated session state database. I'm working on adding a new site to that list, and am debating whether I should create another new session database, or just share an existing one with another app.
The original question was "Share the same Session Database". I see this as different from the Application's Database. ASP Session for all applications would be the same. You would not be modifying the Schema for any of the Session Tables, SP's etc... We host a number of applications, each with their own private Database for the aplication data, and a single shared Database for session.
I would vote for separation here.
I don't think you'll necessarily find that it's "easier maintenance" in the long run to stuff everything into one database. If every app is using the same table and database instance, you can't separate one application from the pool without duplicating the entire database. What if one app goes viral and needs to be moved to it's own server cluster?
For the amount of work it takes to copy a database to a new instance, you'll be separating the concerns of the applications, making it easier to debug them individually, more scalable and much more portable.
The current application is a kind of CRM application built upon MS Access. The application is for internal use. My job is to migrate it to ASP.NET web-based application. Now boss requires to keep Access as database and develop ASP.NET code against it.
My question is, is there any disadvantages of using Access as database in ASP.NET application? (e.g. optimistic concurrency issue?) Should I persuade boss to upgrade Access to MS-SQL?
Many thanks!
We've used Access as a backend for web sites with good success. It's cheap, can be used effectively by moderately skilled programmers, and you can store the MDB on a document server so it gets backed up.
Most IT people dislike Access, but from a business perspective, Access can be very valuable.
MS Access is notoriously unstable in multiuser environments. A WEB app is by definition heavily multi-user.
So IMHO leaving MS Access as underlying DB is a call for trouble. At least use SQL Express (it is free)
The problem you are going to face in upgrading from Access to MS-SQL is that there is a major cost investment for the application. If your company already has the infrastructure in place(licensing, hardware...) then you won't have such a hard fight to pursuade your boss.
As for a technical answer:
I'd say you need to let you boss know that access databases aren't ideal for concurrent usage which a web application suggests is the intended goal of the application. My view is that Access is for database information that a SMALL set of users will be simply using for small data entry and querying. NEVER use Access to build an enterprise-level solution.
If you are planning to upgrade a Microsoft Access database to SQL Server 2008, use the SQL Server Migration Assistant (SSMA) rather than the upsizing wizard built into MS
10+ tips for upsizing an Access database to SQL ServerAccess.
Your boss probably likes to do ad-hoc stuff with access / excel. If you move the DB to SQL Server Express you can use Access and it's linked table feature to let your boss keep doing his ad-hoc needs through Access while keeping the data in SQL Server Express. If you keep the linked tables named the same as the old physical ones all his reports and queries will should keep working.
I'm an Access promoter, but not for use on websites because Jet/ACE is not threadsafe (though Michael Kaplan once said that is is threadsafe if you access it via ADO/OLEDB; I don't quite understand how a database abstraction layer can wash away a characteristic of the underlying database engine it's calling, but if MichKa said, it's 99% likely to be true).
Now, the exceptions would be if you're using it for prototyping something that will use a different database, or if it's read-only, or is read-write but will only ever have a very small number of users.
Michael Kaplan's website, trigeminal.com, used to use a Jet database as the back end (it may still -- I don't know that MichKa ever changed it), and when that was his main website he reported getting 100K hits a day. But it's a read-only site, so fits my restrictions.
There are so many different alternatives and they are mostly easy to use that I just don't see the point of trying to use Jet/ACE as back end for a website. I'd never do it myself (all the websites I'm responsible for use MySQL).
Simply put, go with MSSQL. Express edition is free, and will give you everything you need to migrate away from Access. These articles are talking about Access applications specifically, but the same issues will plague you.
http://resources.zdnet.co.uk/articles/features/0,1000002000,39285074,00.htm
https://web.archive.org/web/1/http://techrepublic%2ecom%2ecom/5208-6230-0.html?forumID=102&threadID=205509&messageID=2136367