We are a small team working on a very tight deadline to develop a large web application in .NET. We are using multiple databases (one per client) so our requirements are slightly different than most applications. The databases will only be used for this particular application so it doesn't matter if they are tightly coupled with the application. The main deciding factors are speed of development, long-term maintainability, and security. There are 3 options we are considering:
Option 1 - LINQ to SQL
None of us have any experience with LINQ, but we have been researching it and it seems like a good option and not too difficult to learn. Worth the risk of learning a new method on a tight deadline?
Option 2 - Stored Procedures
Seems like it could be a nightmare to maintain with the multiple database setup (or would it?) and it may slow down development to work in another environment as we don't have a dedicated database developer. Basic CRUD queries would be generated by code generator which is an advantage.
Option 3 - Inline Queries
This method would be the fastest to develop but I know people are generally against hard-coded queries nowadays and I fear we may suffer in the long term with maintainability issues. Basic CRUD queries would be generated by code generator.
Please let me know if there are any factors we are missing. What solution seems the most appropriate for this project?
If you have a tight deadline don't try something new. Ask the devs to study Entity Framework at home and in their spare time and try it in next project. Meanwhile do what you know best and have used in past successfully.
Inline queries are not bad if they are decoupled in a DAL assembly.
Since #Hasan Khan covered the primary answers regarding SQL. I'm going to throw out a somewhat different answer. Another option is to consider usage of RavenDB, a NoSQL db. It has the concept of Tenant databases inherently baked into it. Which from your requirements it sounds like this is the intended goal.
Related
We have a modular application, each app creates its own tables (typically one two) and manage the data.
We use Flyway in our main application but also need it for our modules. However, if we add the patches to our main application, ALTER TABLE queries won't work for some deployments if the corresponding module is not installed.
One way to solve this issue is to perform the schema evolution with multiple Flyway operations, each module gets its own Flyway and manage itself. However since Flyway creates tables for managing the state we ended up too much tables since we have ~20 modules right now.
What's the elegant way to solve this issue?
I would say having the migrations managed by the unit of software it is supporting is the cleanest and trumps "too many tables". In terms of neat organisation of those tables, you can silo those using a schema (if you RDBMS supports those) and Flyway lets you name the table that is used per migration managed application.
The key thing here is "modules". From your description, it sounds like not all applications are made up of the same modules. I would ask you, if we go to the effort of making our modules descrete to create decoupled / reusable software - why should database schemas of those modules be treated any different?
To counter your concern about "too many tables", lets try and debunk the costs of that.
Volume. Your RDBMS is made to handle thousands, there is no cost there.
Operational. Flyway does all the management here, they are effectively opaque to you.
Performance. They are a deployment concern, not a runtime liability.
Organisational. Hide them / name them with the methods mentioned above.
Our natural urge is to aggregate related things but doesn't always lead to the best outcome, so we must be pragmatic. In this situation, good/flexible design trumps aggregation.
Team A has an enterprise app that uses ADO.NET for data access that executes stored procedures. The data access is encapsulated in it's own project (let's call it DAL.dll)
Team B is creating another unrelated app that's reusing the stored procedures in the enterprise app. This app is currently using the MS application block for data access. The issue we run into is that whenever Team A make any change to the input/output params in the stored procedures, there is a runtime error in Team B's app and this app needs to be updated to accommodate the additional params (or params that were removed). So, most of these go unnoticed until a user complains. At the very least, we would like to have the app throw a compilation error so that the build process warns us of the changes made.
One way to do this is to have Team B's project add a reference to the DAL.dll
I'd like to know if there are any other cleaner ways of solving the issue. We are ready to replace Team B's MS Data application block to use a different technology (Entity Framework?) if necessary.
Among the other answers, I'd strongly suggest getting those stored procedures into source control, in a Database Project. You then may be able to use the features of your source control system to do several things:
Lock some of the code so that it cannot be changed
Give you notifications if the code is changed
Warn you if the stored procedures change in a way that would prevent them from being called
Branch the stored procedures so that each team can have their own version of changed code, while keeping the unchanged stored procedures common. You of course will need to separate the different versions in the database.
I agree with the other posters on this thread that you should not share stored procedure's across different .NET DLL's, that is just a recipe for disaster. I would also shy away from ORM's like Entity Framework if you are doing anything at all complicated with your database schema because ORM's excel at getting a simple object model translated from your .NET application classes into SQL tables and SP's, but traditionally do poorly at optimizing them for performance on the database side. There will be people who claim otherwise, and they may have a valid point if you are an expert in wrangling an ORM to do waht you want like they are, but chances are you are not and it will cause you headaches in the long run.
A shared data access layer might work, but conceptually you are then just changing the implementation of the dependency from some code that a DBA wrote to some code that a .NET programmer wrote. Yes, you can use integration tests to achieve better verifiability, but the same case could be made for SQL with tools like Red Gate's SQL Test. I would shy away from this approach if the two applications are already experiencing some sort of pain from sharing SP's. That is an indication that the dependency just should be done away with.
If it were up to me, I'd just make a new schema for Team B's app. You can read more about schemas in SQL Server here: MSDN Schema description for 2008 R2. You can think of them as namespaces for SQL Server but with some additional bells and whistles like permission and access control. Separating out your different applications into separate schemas on the same shared database will probably make for the most flexible implementation in the long run.
unrelated app that's reusing the stored procedures in the enterprise app
If these two application are really unrelated why are those sharing procedures or even the same database. I know this is a long read, but I recommend you to read this: A Better Path to Enterprise Architectures
The partioning concept in there relates to the bounded context in Domain driven design:
Multiple models are in play on any large project. Yet when code based on distinct models is combined, software becomes buggy, unreliable, and difficult to understand. Communication among team members becomes confusing. It is often unclear in what context a model should not be applied.
Therefore: Explicitly define the context within which a model applies. Explicitly set boundaries in terms of team organization, usage within specific parts of the application, and physical manifestations such as code bases and database schemas. Keep the model strictly consistent within these bounds, but don’t be distracted or confused by issues outside.
It is expected you end with problems when you don't explicitely deal with this. You're lucky you're seeing early failures, as it can turn into problems much harder to find on the long run.
Analyze the problem again with the above in mind. Consider if you're missing some explicit context where this common functionality should live.
My question is: which team owns the store procedured and the database shared? Usually as a good architecture/design, you should not have two different apps sharing same database / procedures.
A better way to share data/functionality between two different applications is through a services or API, so the team who owns the functionality would be responsible to maintain it.
Also, have a good communication between both teams is highly recommend.
Depending on the owner of the DAL project, you could host web services and share the API. That way, you separate the Data Access Layer from the business logic, which allows anyone to use the same DAL without having to publish it to each different location.
From my point of view, it looks like both Team A and Team B should share the same core model and look at Multitier architecture as a possible solution.
It sounds like it would make sense to create a shared DAL that both applications can share.
I would add unit tests (or really integration tests) to make sure the DAL is compatible with the apps after changes. That way your tests would fail if incompatible changes have been made
"I'd like to know if there are any other cleaner ways of solving the issue."
The cleanest way is for Team B to sit down with Team A and encapsulate the relevant business logic into a shared API. It doesn't matter so much how you implement that API; what does matter is that the API's interface is documented and versioned so everyone knows what to expect.
One reasonable mechanism for this in a .NET environment is to use Microsoft's WebAPI.
In short, the question of "how do we share a stored procedure?" is most likely looking at the wrong level of abstraction.
We have an ASP.NET app with SQL Server & it is a photo & video sharing site.
Details of photos and videos are stored in tables & the files are in the file system.
Database has 75 tables and 225 stored procedures. The app will be ready for production deployment within next 6 months.
Due to longer time growth concerns, we decided to switch to NoSQL (MongoDB) database.
We have few questions regarding the best way to approach this:
Is it better to deploy the app with SQL Server backend and migrate to NoSQL later?
OR re-architecture now and rewrite/recreate database, tables, procedures and data layer
How difficult will it be re-architecture/recode with MongoDB? Any tools or BKMs?
EDIT:
Our app is Youtube+Flickr type site where user will share photos and videos with lots of comments, tags and ratings (photo\video & comments).
Is NoSQL a better database to move to? Reason for moving: cost + read query speed
Please help me with you valuable advise.
Thank you very much.
Change is always exponentially more expensive the later it is introduced to a project. This is a core principle of software engineering. You should do this now.
That said, I question your long-term vision. Relational databases, used properly, have a lot of performance in them.
This question raises more questions than answers.
Have you benchmarked your current implementation in terms of requests/responses?
Why MongoDB out of all possible NoSQL databases? (Don't get me wrong, I love Mongo, but love and hype should not weigh in technology choices)
Are you certain you will get the large userbase you're expecting? Why are you so certain?
Using stored procs seems to tip off that you aren't using an ORM? Why not?
Generally, I'm against these types of re-architectures. Firstly, you need to get your whole team acclimated to how Mongo affects development. Secondly, your ops team needs to get acclimated to how to deploy and maintain a Mongo installation. More likely than not, this will prevent you from launching in a timeline you want to launch.
I'd say that you should probably launch as is, fix the ORM part if you aren't using one, benchmark your app, benchmark a prototype of your app backed by Mongo and if the performance advantages are so big that it warrants the pain of re-architecture do it.
To your latter question, there aren't any tools right now, as far as I can tell, that'll automate or semi-automate the database import/export from SQL Server to Mongo. There are barely tools to do that for MySQL.
I've done such a migration a few month ago, during the early developement stage of a website in ASP.NET. It was a hard decision, but I could concentrate on that migration. The reason why I did this migration was the ORM that I couldn't trust anymore and some very slow queries that I had no idea how to optimize.
During coding phase, what I figured out was : I was spending a lot of time with the data model in SQL Server (using Entity) and all the plumbery code.
Now, no more store procedures (C# and Linq code instead), no more 2 layers to maintain (the code is the model).
My small experience says : The earlier the better but don't get me wrong, before migrating you really have to think in Document rather than in RDBMS. This means you may have to partially change the businness DataModel to correctly utilize MongoDB features, otherwise you could get bad performances and Mongo DB is useless for bad models.
Another point is the admin stuff. You'll have to quickly learn Mongo DB admin to be up to speed. And even if the tools are good, they completely differ from SQL Server tools.
In conclusion, If you're convinced MongoDB is your future data store and search database,
(and it was in my case), read documentation, take time to do some Proof Of Concept. Then you can think Document and load test you new model.
Your core question appears to be whether to make the switch to MongoDB now, or deploy on SQL and go to MongoDB in a future release.
You do not appear to be using an ORM (e.g. NHibernate, Entity Framework.) Setting other concerns aside, if you're convinced that you want to go to NoSQL, then I would do it now rather than later. Unless you integrate a Provider model for your data access, changing the underlying data access strategy after it is already established would be difficult.
I agree. Switching now is better, if only to avoid the data migration headache switching post-deployment will require.
I want to use an ORM, and have been looking at EF 4. Is this platform scalable. I see a lot of stuff on the web, but everything looks very biased in one way or the other. Anyone know of benchmarks or non-subjective information.
On that point, does EF prevent SQL injection or XSS. I know that it used parametrized queries, but is that enough?
Any help is appreciated.
Okay so i see two questions here.
Is EF Scalable
Very difficult (and subjective) to answer, but IMO yes.
Here's a few reasons why:
Utilizes a common querying language (LINQ)
Allows for multiple providers (SqlServer, Oracle, etc)
Allows bi-directional mapping (code first, model first, database first)
Includes "classic ADO.NET" support (stored procedures, Entity-SQL)
The main real benefit in scalability is how the framework is built on LINQ-to-Entities. When you write queries, you are not writing against SQL Server or Oracle, you are writing against the Model. Depending on what Provider you have setup (in web.config), EF will translate these model queries into the appropriate T-SQL (or P-SQL).
Therefore (theoretically), you could write code against SQL Server, then change the web.config provider to Oracle, and your code should work. Obviously this isn't the case for Entity-SQL though (as you are writing T-SQL, not LINQ).
Does EF prevent SQL injection or XSS
No ORM tool can really "prevent" SQL Injection attacks - they can only provide the developer with the tools to prevent it.
As with classic ADO.NET where you use parameterized queries, Entity Framework has Entity-SQL, which allows to to execute pre-generated SQL, stored procedures, etc.
In this scenario, you need to use parameterized queries to prevent SQL injection. For most EF work, you will be writing queries with LINQ, which is a lot safer because it gets hydrated through a lot of stages before it becomes SQL.
XSS is exploited on the client-side via things like injected JavaScript, dodgy emails, etc. Has nothing to do with Entity Framework. Prevention of XSS is done on the client-side with things like HTML encoding.
No. ORMs are not a panacea for scalability. There is such a things called the impedance mismatch of objects and databases which has been around for many years. ORMS try to solve this by providing magic code generation/mapping solutions that give the appearance of just working with objects.
In a multi-tier environment with many client programs and a single/many server scenario, for every change that has to be committed to the database, checks need to be performed to make sure that your not over writing someone elses change on the data, or trying to update data that has been removed. This is not a new problem introduced by ORMs but one which appears many many times throughout the ages of updating databases in N-Tier environments. ORMS do not solve this problem. In some cases, if the ORM is the single entry to the Database, the ORM becomes a bottle neck. This means that to create a scalable architecture using an ORM becomes problematic as having DB checks performed on the ORM means that the update anomaly checks could be by passed if your using an N-Tier ORM solution where you have duplicate ORM tiers.
For the reasons above, this is why we use stored procedures. But if your using stored procedures, which naturally obfuscate the underlying data structures of the database then this increases the impedance mismatch of objects and database entities. One thing about using stored procedures and relying on table locking/row rocking, some of the update scenarios are solved, as we shift the bottle neck to the performance of the underlying database design.
So whats the answer. Don't use objects for databases. Object are great for analysis, bad for code design when interacting with RDBMS databases.
If your really thinking SQL and RDBMS data solutions are a problem, which in some scenarios they are, take a look at some of the NOSQL solutions out there. Still not a panacea for all problems, but in some cases they provide a better solution than a straight SQL solution.
Objects are not the answer to all problems. Step back from your code, take a look at what your trying to do, and think if an object is the right approach.
As for security, no ORMS do not aid security. Although they do help prevent some forms of injection attacks.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
I have around 4 years experience of ASP.NET & SQL Server (only web forms). I have a little knowledge of Ruby on Rails and have developed one very small 2-3 pages application in ASP.NET MVC in my job. Now I and one of my friend ( same experience like me) are willing to gain more experience of ASP.NET MVC. For that purpose we have thought of developing an open source application in ASP.NET MVC. It can be a a blog system or forum or anything else so that community can also take benefit of it. But we are not able to decide the back-end database for the application between MySQL and SQL Server. What will you suggest in terms of following -
Which will be more supported or required by community?
If we use MySQL, will it be a tough road ahead?
Is there any chance of supporting both by using ORM (I have no experience of working with any ORM)
Any other suggestion ?
Thanks in advance.
If you can't decide which to use as a backend, then one suggestion I have it define an interface for the data layer, use the IRepository pattern against this interface, and simply pick one and write an implementation of the data layer for that choice. In a (successful) open source project, it's unlikely that everyone who wants to use it will have the same ideas about what database will work for them. In that case, designing it so that it can use any database, given a suitable implementation of the interface is a wise move. Using something like nHibernate, which already supports many different databases may be the route you want to go if you don't mind having dependencies on other libraries.
Which will be more supported or required by community?
For most business users, and most dedicated ASP hosting providers, it will be Microsoft SQL Server IMHO. With a surprising spread across version numbers. I see quite many SQL Server 2008, a lot of 2005, and quite surprisingly many 2000's too.
For small "we're cheap, is it gratis?" businesses and individuals, I do see some MySQL on Windows too. I guess in my little circle of companies that I've worked with, it is 2/3 MSSQL and 1/3 MySQL of the two databases you mention.
If we use MySQL, will it be a tough road ahead?
In terms of more difficult development? Yes and no; LINQ to SQL is more or less only available for MSSQL, and many find it to be a simple and natural technology to work with. So "yes" if your plan was to use LINQ to SQL as your ORM of choice. Other good ORM's like nHibernate work equally well with both databases, as elder_george says.
Is there any chance of supporting both by using ORM (I have no experience of working with any ORM)
Sure, f.x. with nHibernate, and a bit of extra XML for nHibernate configuration, and some more extensive install docs, SQL scripts et cetera for the users.
Any other suggestion ?
Since this is your first application, play around with the design first using mocksups & prototyping, UML or something else of your choice. See if you can get some experienced MVC developers to critique your design, but don't take their critique as gospel, there is more than one way of doing things right.
Definitely consider tvanfosson's suggestion to use an IRepository to provide flexibility on the storage layer.
Consider using unit testing and dependency injection from the start too; both are IMHO very important and helpful in the longer run. On the downside, that would be a lot of new technology if you don't have prior experience with these.
MS SQL is easier to use with .NET. .NET includes LINQ to SQL ORM out-of-box, so nothing has to be installed.
MS SQL has a free Express edition but it's up to you to decide whether its capabilities (4Gb for DB, 1 CPU only and so on) will be enough for your application (IMHO they will be =))
There'are ORMs that allows to switch between DBMS's: NHibernate, SubSonic, DbLinq. NHibernate is the strongest beast out there, and it has an active record implementation built if you are looking for familiar concepts from RoR.
tvanfosson and elder_george had given good comments.
ASP.NET MVC aside, it's more of MySQL vs MSSQL. I've developed web projects using both and find them relatively on-par from a development point of view. ORM I've used is NHibernate.
From deployment/production perspective, you will be bounded by MSSQL Express edition's limitation, assuming you are not going to pay for it. You might want to spend some thoughts in this aspect.
Use ORM -> NHibernate
https://www.hibernate.org/361.html
I was in a similar situation a couple of months ago. Now my newest project on ASP.Net MVC is almost complete. Here's the set I chose
--ASP.Net MVC + MySQL. No ORM.--
I decided to go with MySQL instead of MS SQL because MySQL is free, and MS SQL is very expensive. If you're making an open source application, you gotta think about that seriously. This is going to be a major problem if you are targeting widespread adoption of your application.
At first I tried to use MySQL through an ORM -- Enterprise Framework (EF). It was the worst decision ever. Enterprise Framework totally killed the performance. It was the pits of the pits. I believe nHibernate is better, but I don't know how much better.
With my own SQL code things are fast enough.
The first iteration of my site will be in public beta this week and then there will be several iterations.
It went rather well so I car recommend the set I chose with ease.