I need to use an embedded database in my java application that will be run in a Linux device. The application uses Hibernate and derby database. This is not a Android application.
Due to slow performance of the database, we are looking for a better embedded database framework.
Looking at all the options, H2 seems to be better than SQLite as there is no cross-compilation involved and no JNI interface to build.
So, why isn't there a more usage of H2. Are there any drawbacks or issues that I am not aware of.
The SQLite library is implemented in C, so it indeed needs (cross-)compilation and a JNI interface.
However, SQLite is so widely used that it is likely that the SQLite interface already exists (as part of your language's runtime, or as a JDBC driver), and that using it is simpler than explicitly adding H2 to your project. (This might not actually be true in your specific environment.)
If you're looking to speed up your application, you have to measure yourself.
I recently switched from H2 to SQLite because of database corruptions in the H2 mv store.
If the application is not shutdown properly, or in case of unexpected reboots, the H2 database stored on a file using the MV store (the default) can get corrupt, and you can't restore data.
SQLite is much more robust to corruption.
Speed wise H2 was much faster in my case. With SQLite transactions are particularly costly, so you should prefer doing bulk operations within transactions or via batches.
As for cross compilation, you can use the jdbc driver from xerial which ships with all the native binaries precompile : https://github.com/xerial/sqlite-jdbc
Is there any program that can serve as a GUI front end for SQLite3 database?
The general idea is to connect to the database remotely, and administer it in FileMaker-like GUI interface, where the online scripts would have a job of just presenting stuff.
I tried FileMaker with ODBC drivers, but have not been successful. SQLite3 is not directly supported, and I couldn't find another driver or software online. (Please mind that I am not talking about database editor software, but something that would be used as a database "CMS" if you will.)
Thanks!
Are you using a mac? Try the ODBC from Actual Technologies that will work for the most databases out there.
There is a more direct approach, that I like better because ODBC is not fast. For small amounts of data it's ok. Besides that communicating direct with the external database without to much middleware is in my opinion always better.
MBS has a very nice plugin, and supports a lot of SQL connections. It works really well and fast. Perhaps you should take a look. I use it a lot, as I've used ODBC a lot in the past.
We have an application that can use both Postgres and SQLite as its backend, depending on what the customer requires: High load and concurrency, or easy setup.
But I fear that some customers will be too clever for their own good, and to get around the (slightly) complex Postgres installation, they use SQLite, but put the file on a network disk.
Then the inevitable happens and the database gets corrupted, because SQLite is not meant for that, and we have Postgres support for that very reason.
Is there an ideal way to prevent SQLite from working on a network drive? There are some questionable ideas like looking for a \\ at the beginning, or the colon in "C:\" (it's purely a windows app), or parsing for IPs, but none of these are fool-proof and can be circumvented by making a link to a network disk.
I use Informix DBMS in all my web applications. My question has two parts:
Does the DBMS have a big effect on the performance of my applications
and if the answer is yes what about Informix and `MS SQL Server in this
issue?
I want some GUI tools to facilitate my job when writing queries,
creating database, relationships, ERD, etc. The Informix client
is so bad. There are no facilities at all. I want some tools
like SQL Server Management Studio
As a GUI tool for Informix you can use Aqua Data Studio from Aquafold. It also supports MS SQL Server.
As of the performance: it depends. How well is your Database design. Do you use indexes, is your query well-written, etc. etc. Very hard to answer your question, we just don't know enough.
To design a solution that would perform the best, one needs to know the nature of the application you are building. For example, if you are building a system that needs to process and compute large volume of data and computations can be distributed, a "traditional" relational database is not a good option no matter what vendor you choose. You would be better off with an option that supports sharding, Hadoop and will likely be based on some kind of NoSQL solution.
If you are sticking with RDMS and building something that has a lot of reads and not a lot of writes, go for a database that supports Snapshot Isolation which will allow your readers to not be blocked by writers.
Cost also plays into this - some RDMS systems are free, some are not. Your question is way to general to be answered specifically.
Aqua Data Studio is good but quite expensive. An open source tool SQL Workbench/J is also an effective tool for informix.
Informix have its own charm but i guess it should not be said that MS-Sql Server is slower or not good in performance. You may decide DBMS according to your nature of application. There are many techniques to optimize Database performance like, Applying Indexes/ Not too many Joins/ Queries can be optimize too/ Stored Procedure can also be used/ Multi-DBs level etc.
Once i need to develop Social Media site, i used MySQL in this project but only for POSTs i installed MongoDB.
Regards,
Salik
I have read this article from High Scalability about Stack Overflow and other large websites. Many large high traffic .NET sites such as plentyoffish.com, MySpace and Stack Overflow all use .NET technologies and use SQL Server for their database. In the article it says a source in Stack Overflow said:
As you add more and more database
servers the SQL Server license costs
can be outrageous. So by starting
scale up and gradually going scale out
with non-open source software you can
be in a world of financial hurt.
Why don't these sites use MySQL instead of SQL Server?
Adding into what AJ said... Remember Facebook also pays C programmers to hack up MySQL code and also PHP code to get things to really work "well" for the amount of traffic they get.
Facebook already made statements in the past and this year about having wished they made a better choice.
As a matter of fact, for coding they're now compiling their PHP down to C++ code using HipHopPHP and about 90% of their servers are running the C++ binaries instead of the PHP scripts.
Their MySQL database might save them a dime or two, but the costs to maintain it, scale it, etc. is extremely intense.
A product like Oracle however would really allow you to scale seamlessly compared to MySQL.
I have a site right now that uses a lot of bandwidth on my database, large number of queries, and the truth is, scaling is a pain in the neck with MySQL and their Clustering product isn't that great and requires a license. Oracle right now has the best "grid" database setup but the costs are insane there...
Also, I code C# as well.. Let me tell you it's MUCH easier to integrate enterprise level sites with SQL Server compared to MySQL.
I would guess that it's probably because it's really really easy to get started making a site with ASP.NET hooked into SQL Server. And for the sites you mentioned, speed to market was probably more important than getting the architecture "right" (not to say that SQL Server is or isn't the right choice - just that speed to market is the priority). Remember that a developer's job is to release software.
So long as one avoids using too many database specific features, it will be relatively straightforward to switch to a different database with moderate effort. But why bother unless your site becomes super-popular?
Edit: And if you become super-popular, you may even want to venture into the land of NoSQL.
While this doesn't directly answer your questions I really have to refute your comment about outrageous licensing costs. ALL ENTERPRISE grade commercial software comes with a high price tag because it has the VALUE for it. If it doesn't have that value, it wouldn't be a successful product.
SQL Server's pricing is extremely competitive and has a very substantially lower TCO than Oracle. Another reason a decision to use MS SQL Server would be made is that most shops that develop on the Microsoft stack are Windows Server shops. MS SQL Server is built specifically for Windows Server so it can integrate as flawless as possible with the operating system. Many other products are not primarily and solely developed for Windows Server so this results in feature differences and environmental bugs.
These enviromental issues can be further compounded with the fact that large scale shops will employ primarily system administrators that have long backgrounds in that specific stack so in a .NET shop most system administrators are all most fluent in Windows Server, having to support multiple operating systems becomes a large cost especially in the risk management side when you're a large scale business.
To repeat what others have said. I work in a corporation and money, so to speak, does not matter that much when it comes to these matters. Decisions are made on the basis of "What kind of support can we get from the vendor", "How many skilled people are in the market", "What are the vendors reputation", etc.
I think there are two distinct groups for adopters of MySQL or SQL Server.
Large websites that are privately owned that does not have additional financial resources. These websites will typically run MySQL. Naturally.
Large websites built by corporations. These sites will run whatever is the accepted database technology within the corporation. Money does not dictate this decision, but more of who can support this software and development.
No Microsoft SQL Server Management Studio. For real. A lot of stuff is done there instead of raw SQL that happens in Open Source software world.
Things are much easier to deal with when the technology stack is homogeneous.
If you want MySQL support for Linq-to-SQL, good luck. It's still very much immature. With SQL Server, it's a matter of drag and drop. Literally.
You can also conduct Database queries from within Visual Studio for SQL Server. I've never tried it for any other database, but I'm not convinced you'd be able to.
It's great to say 'Oh, MySQL is so much cheaper than SQL Server.' Yes, it is. But I'm not sure the integration costs are worth it; not to mention having to rely on Yet Another Vendor to provide support if something goes wrong.
You use what you know...
(IMHO) The Microsoft tool stack is brilliant. It works well, we learn with it and grow with it, as the technology grows. It becomes easier to use as you become accustomed to it (its quirks and idiosyncrasies).
MySQL is also a brilliant tool. It works, and works well. We could all have religious wars as to what tool is best, but remember it is just a tool to get a job done.
Now let's factor in the cost of the software - Plenty of Fish 2 years ago made $7M, do you really think they care how much their database/server software costs? SO is on BizSpark $0 cost for 3 years (that's got to hurt).
For the sceptics, FaceBook runs MySQL on/for 30K servers and MySQL Enterprise Unlimited Licences cost $40k so this is not necessarily cheap either.
I don't know about you, but for me when I make a ton of cash, I really won't care how much it "costs", because I am making more with it, than without it!
I would say because of the following:
Microsoft is very well integrated while used with Microsoft products ;
Though using SQL Server, a free Express edition is available and can be used to host sites ;
With the .NET Framework coming through, Microsoft gained a lot of terrain over its competitors in schools an so, thus making SQL Server a well known database engine ;
Microsoft products works better with other Microsoft products ;
There are two ways of licensing SQL Server, per client (CAL), and per server processors or something like that. For sites hosts, perhaps is there an advantage of using SQL Server this way ;
Other database engines such as MySQL, PostgreSQL, Firebird, etc. all have their syntaxic differences, thus making SQL Server TSQL somehow a wise choice as for the number of persons being able to interact with SQL Server more easily ;
There might be some other politic related reasons for using SQL Server over other less costly solutions.
I would like to mention that some are using SQL Server, yes, but they use SQL Server Express Edition. Though they are whether aware or not that publishing or commercialising a solution with SQL Server Express Edition makes, according to the EULA of Microsoft for this product, your solution a free solution as well, as the EULA states that you need to provide your solution to your customer, and your customer is free to share your commercial solution with whom who wishes because it is sat on SQL Server Express. Although this is stated, some continue to use SQL Server Express without informing their customers about this information. Most of common clients won't know about this and they will respect their contract with the solution's supplier.
Furthermore, as I think I have above-written, some don't care about the price, but they have political reasons for using commercial products such as SQL Server and other software products. There are some places where the money isn't the most important factor, but service after sale, etc. They want specialized engineers or support teams directly, not necessarily what offers MySQL-like communities.
Hope this enlights a bit.
It's just culture. People group themselves. It's natural. People who prefer open-source, will naturally choose LAMP (Linux, Apache, MySQL and PHP) for the same kind of project that people who prefer corporate support choose Microsoft IIS, Microsoft SQL Server and Microsoft .NET for. There is a good deal of human psychology involved in this, make no mistake about it. There is nothing prohibiting one from using IIS with PHP and MySQL, or Apache with Microsoft SQL Server, but the way it goes is as described above.
Shorter put, large sites do use either, but yes, not often the two you mentioned together.
I believe George has it on mark: "homogeneous".
Most of Microsoft's technologies are built to work together. There are direct hooks between .NET and SQL Server to provide additional functionality like cache management that just don't exist between .NET and MySQL.
IIRC, MySQL doesn't have built in cache management which is why Ehcache and memcached exist.
re Joshua's comment: "A product like Oracle however would really allow you to scale seamlessly compared to MySQL." Years ago, Sabre picked MySQL over Oracle for some high scale projects based on cost and feature set. AFAIK, it's still picked over Oracle unless you can prove through cost/benefit analysis why Oracle is the better choice for a project.
I think it really boils down to functionality, user knowledge base and interoperability.
Sometimes SQL Server is the better match, sometimes it's MySQL, sometimes it's Oracle.
Less compatibility issues when you single source.
MS SQL Server is the "default" database for ASP.NET apps (see LINQ to SQL, ADO.NET, ApplicationServices etc)
Immature .NET tools for other databases. For example, you don't have to worry about a feature or functionality not being supported if you stick with MS SQL Server, other databases might not have full support (e.g. DbLInq, etc.).
MS SQL Server is also free to get started, (SQL Server Express) and once you're ready to go public, it's hard to change the Data layer.
I'm in the process of writing an ASP.NET MVC2 site with MySQL as the backend (mainly due to licensing costs) I've implemented DbLinq, but it also means writing a custom Membership/Role provider, and general tweaking of the data layer. It's definitely doable, but it's not as simple as sticking with MS SQL Server. I'm also hoping to move the site over to Mono 2.7 (once it's released) running on a Linux server to sidestep the server licensing issues as well.
The real reason is people usually go with MS SQL Server as .NET comes from the same brand. For instance PHP people always prefer MySQL over other databases. It's all mind set and people don't want to take any risk.
Any large enterprise site isn't going to care about licensing costs that much. What they want is fast, reliable data access and access to company technical support. They also want something that can easily be partioned to scale and that is designed for huge databases. They also want the easy availablity of performance tuning specialists, datawarehousing and Business Intelligence specialists, database developers, and database administrators. SQL Server and Oracle both meet these criteria. I really don't see MySQL as having as many people qualified to design and monitor large systems. I am Not sure how it stacks up on the partitioning and scalibility though.
Well, for one thing there are other, better, free databases (e.g., PostgreSQL). For another, the Microsoft ecosystem is designed to suck you in, getting you to spend more and more with the guys from Redmond.