Related
I'm very new to SQL and relational databases (just started learning last week) and I'm in the process of upgrading my website and currently keep all my data in XML files. It works, but the new site would be better suited from what I hear a relational database can do, and it looks like SQLite is best for me. One of my concerns is concurrency, even though 99% of the data will be read-only (which I understand SQLite is pretty good at) 99% of the time. Other things, like page view counters for certain pages will constantly require small writes. I'm still learning database design and want to do it right. Would it make sense to make separate databases for things that get written to a lot, that way making the main database far less susceptible to concurrency issues? Is it possible to do a "foreign key" type reference (I still haven't used foreign keys yet, but think I understand them) across databases? As each view count would point to some primary key in the main database. Thanks for any help!
SQLite is good to use in embedded systems (like mobile phones and tablets) and small desktop applications (Chrome, Firefox, Thunderbird, etc). However, when you need to have many concurrent readers and writers (typical for websites), you should not use it.
Even if you split your data in many databases, it has a lot of operational overhead. For example, it will be difficult to join data from different databases - you must use ATTACH, and by default you can only ATTACH up to 10 databases. And concurrency issues will still not go away 100%.
Instead, use real database like PostgreSQL or MySQL. Not only it will be faster, these databases provide real concurrent access to your data over the network, which SQLite cannot do.
My personal preference is PostgreSQL, but if your web hosting does not provide PostgreSQL, you can use MySQL, but then please use fully transactional engine like InnoDB.
I need to store the application settings somewhere, but can't find a satisfying solution. Read only settings are pretty easy to store in web.config, but what about settings for application administration that would should be accessible through web-page? Writing to web.config doesn't seem to be a good idea. I have considered storing the settings in custom xml file, but then if there is sensitive information involved in the settings, that seems to be problem, also if there are multiple users modifying the settings at the same time some kind of file locking has to be involved. Now I am inclined to store the app settings the MS-SQL database, it seems like a secure and well scale-able solution, however it feels wrong to have a table to store just one row - the setting. What's your opinion? How would you design that?
Are there any ready to go .NET solutions for storing dynamic web app settings?
Your question is so subjective that I don't even know why I am answering it instead of voting to close. But anyway, a database is a good place. And if you are bored and tired of relational data there are great NoSQL databases out there such as MongoDB and RavenDB that will make this very easy. And if you want a very fast database Redis could be worth checking out.
Storing things in files in a web application is far more difficult than it might look at the first place. If it is for readonly then web.config could indeed be a good place. But once you start writing you will have to take into account that a web application is a multithreaded environment where you will have to synchronize the access to this file. And what looked in the first place as an easy solution, could quickly turn into a nightmare if you want to design it properly. That's why I think that a database is a good solution as it gives you concurrency, security, atomicity, data integrity, ...
I absolutely think that storing settings of dynamic nature in database is the right way. Don't feel bad about having one simple table. This table can save you a lot of headaches. If you'll code it smart you can really benefit from it (but that depends on the type of values you want to store). The only problem with db is that someone might actually modify values directly in database. But it can be easily solved. For example I have a "configuration-values" class that I feed from database upon start and put it to cache with some timeout. Then after a while I can lazily feed it again, catching situations like I mentioned above. I hope it makes some sense.
We can precompile our (ASP.NET) websites and can publish only the IL code, so that the source code is not available to the customer.
But how do we do it for stored procedures written in SQL Server. I mean, when we give the customer the DB, he could see all my stored procedures and can modify the same... How could I protect it.
Thanks
Raja
An old problem. Here are a few answers I've picked up here and there:
Encrypt the stored procedures. As has
already been pointed out twice, this
doesn't really work, as 5 minutes of
Googling will find several hacks.
Write the stored procedures as CLR
procedures. Harder to hack than
"regular" stored procedures, probably
a lot more effort to produce and
support.
Submit all queries dynamically from
your compiled IL code. I understand
it can be done reasonably secure from
SQL injection attack, but make darn
sure before you release. (Maybe use Linq to do this?)
Convert all database object names
(tables, columns, procedures) to
guids or random gibberish. They could
read it, but that wouldn't help much.
I am not totally conversant on encryption within SQL 2005 and up. I really don't think you can use it on code-based objects (procedures, functions, etc.), but maybe you can?
But by and large, once you give a copy of your database to someone with SysAdmin rights, they can do pretty much anything they want with it.
use WITH ENCRYPTION
example
CREATE PROCEDURE prTest
WITH ENCRYPTION
AS
SELECT GETDATE()
Keep in mind that it can be cracked and also make sure you have the unencrypted source code backed up
CREATE PROCEDURE ... WITH ENCRYPTION
However note that this encryption is really more like obfuscation, and there are several ways to bypass it if your vendor is determined, including the DAC connection, some 3rd party products (including RedGate SQL Prompt), and code samples you can readily find online.
I'm about to build a new personal blog/portfolio site (which will be written in ASP.NET), and I'm going to run it against a SQLite database. There are a few reasons for this:
The site will not be getting a lot
of traffic, and from what I've read,
SQLite is able to support quite a
lot of concurrent users for reading
anyway
I can back up all the content
easily, just by downloading the db
over FTP
I don't have to pay my hosting
company every month for a huge
SQL2008 database that I'm hardly
using
So, should I go for it, or is this a crazy idea?
I'm not so sure about #2 (what happens if SQLite makes changes to the file while the FTP program is reading it?) but other than that, there is no reason to prefer one DB over the other (unless one of those DBs just can't do what you need).
[EDIT] Use an online backup to create the file for FTP download. That will make sure the file content is intact.
Even better, add a page (with password) to your site which creates the file at the press of a button, so your browser can download it.
It's just fine for a low traffic site as long as it's mostly read traffic. If it were me, I'd use SQL Compact Edition instead (same benefits as Sqlite- single file, no server), just because I'm a LINQ-head and the LINQ providers are "in the box" for it, but Sqlite has a decent LINQ library and managed support as well. Make sure your hosting company allows unmanaged code, or that you use the managed port of Sqlite (don't know its current stability though).
SQLite can handle this easily - go for it.
You should check, but I think that the Express version of SQL 2008 is free of charge.
Anyway, I've been working with SQLite from .NET environment, and it works quite fine (but I haven't done any load test).
And if you're not decided yet, you still can use a LINQ provider which will allow you later to switch from one database to another without rewriting your SQL code (I think to DbLinq, for example).
If you plan to backup you database, you must ensure first that it is not used at the moment.
SQLite answer this for you:
http://sqlite.org/whentouse.html
low-medium volume = okay,
high volume = don't use it
in your case its a-ok to use sqlite
Generally, yes.
But you should be aware of the fact that SQLite does not support everything that you might be used to from a 'real' DBMS. E.g. there are no constraints like foreign keys, unique indexes and the like, and AFAIK some (more advanced) datatypes are not available.
You should check for the various limitations here and here. If you can get along with that there's no reason to not use SQLite.
A rule of thumb is that if the site can run on one server
then SQLite is sufficient. That is what the creator of
SQLite, D. Richard Hipp, said at approximately 13 min
30 secs into episode 26 of the FLOSS Weekly
podcast.
Direct audio link (MP3 file, 24 MB, 51 min 15 sec).
I'd say no. First off, I don't know who you are using for a provider, but with my provider (goDaddy), it's pretty cheap at $2.99 a month or so. I get 1 sql server db and 10 mysql dbs.
I don't know how much cheaper this can get.
Secondly, why risk it? Most provider plans include at least MySQL database. You can hook up with that.
In general, SQLite isn't meant for a high-traffic website, but it can do quite well on websites getting 100,000 hits/day or less. The SQLite org website gets more than 500,000 hits/day, and generates 2 million or more DB interactions/day ... all handled by SQLite.
Here are some things that will dramatically speed up SQLite's performance:
Index your tables
Use transactions for multiple commands instead of executing one at a time.
Learn about write-ahead logging
Do a Google search on each of the above with SQLite ... your DB performance will improve dramatically.
An SQLite DB can actually be faster than a MySQL, PostGRE, MS SQL Server DB, or other hosted server-based DBs for 2 reasons:
1). SQLite is usually stored on the same machine as the website, rather than a separate server machine, eliminating round trip network latency response times.
2.) For smaller read/write requests, the SQLite engine is executing far less code, which can also be faster.
For a smaller website, a smaller DB engine like SQLite could actually be faster and more efficient.
Are you using any SQL functionality? SUM, AVG, SORT BY, etc, if yes go use SQLite. If not, just use plain txt files to store your data. Also make sure that the database is outside the httpdocs folder or it is not web accessible.
I don't know how authoritative this is but I found this:
http://www.sqlite.org/cvstrac/wiki?p=PerformanceConsiderations
and it doesn't seem good to have a lot of connections to sqlite. This seems to be bad for the web and most applications that have more than a few users. I'm having a hard time thinking of what sqlite would be used for when you don't need that many connections. Every program I can think of needs users, lots of them sometimes, so what would I use a database for that doesn't allow that many connections? I thought about prototypes but why would I use that when I can just connect to a larger database? Embedded apps maybe?
Thank you.
EDIT: Thanks everyone. I look at the page recommended below but an confused about something:
Under appropriate uses for sqlite it has:
Situations Where SQLite Works Well
•Websites
SQLite usually will work great as the database engine for low to medium traffic websites (which is to say, 99.9% of all websites). The amount of web traffic that SQLite can handle depends, of course, on how heavily the website uses its database. Generally speaking, any site that gets fewer than 100K hits/day should work fine with SQLite. The 100K hits/day figure is a conservative estimate, not a hard upper bound. SQLite has been demonstrated to work with 10 times that amount of traffic.
Situations Where Another RDBMS May Work Better
•Client/Server Applications
If you have many client programs accessing a common database over a network, you should consider using a client/server database engine instead of SQLite. SQLite will work over a network filesystem, but because of the latency associated with most network filesystems, performance will not be great. Also, the file locking logic of many network filesystems implementation contains bugs (on both Unix and Windows). If file locking does not work like it should, it might be possible for two or more client programs to modify the same part of the same database at the same time, resulting in database corruption. Because this problem results from bugs in the underlying filesystem implementation, there is nothing SQLite can do to prevent it.
A good rule of thumb is that you should avoid using SQLite in situations where the same database will be accessed simultaneously from many computers over a network filesystem.
The Question:
I'm going to show my ignorance here but what is the difference between these two?
This is answered well by sqlite itself : Appropriate use of sqlite
Another way to look at SQLite is this:
SQLite is not designed to replace Oracle. It is designed to replace fopen().
It's good for situations where you don't have access to a "real" database and still want the power of a relational db. For example, Firefox stores a bunch of information about your settings/history/etc in an SQLite database. You can't expect everyone that runs firefox to have MySQL or postgre installed on their machine.
It's also perfectly capable of running relatively-low traffic, read-heavy websites. The performance of it is overall very good, it's more than the large majority of websites need for their traffic levels.
It's often used for embedded applications.
It can be very handy to use a database like storage when you have no access to a database service. So SQLite is used since it's just a file you store somewhere.
I also find that using SQLite is good for getting a prototype application together pretty quickly without the overhead of having a seperate DB server or bogging a development environment with an instance of MySQL/Oracle/Whatever.
Also easy to pick up and move the database to a different machine if you need to.
The iPhone uses it for call history, SMS messages, contacts, and other type of data. Like Ólafur Waage said, good for embedded applications on mobile device because it's lightweight. I have used it also on stand alone applications. Easy to use and available on most platforms.
Think about simple client or desktop apps that could make use of a db, like as a poor example, an address book. Rather than bundling a huge db engine like mysql or postgre with your deliverable, sqlite is very lightweight and easy to include with your finished app.
This FLOSS Weekly podcast episode talks with the creator of SQLite and covers among other things goes over the type of things you would use it for. Everything from file systems for mobile phones to smallish web sites.
In the simplest terms, SQLite is a public-domain software package that provides a
relational database management system, or RDBMS. Relational database systems are
used to store user-defined records in large tables. In addition to data storage and management,
a database engine can process complex query commands that combine data
from multiple tables to generate reports and data summaries. Other popular RDBMS
products include Oracle Database, IBM’s DB2, and Microsoft’s SQL Server on the
commercial side, with MySQL and PostgreSQL being popular open source products.
The “Lite” in SQLite does not refer to its capabilities. Rather, SQLite is lightweight
when it comes to setup complexity, administrative overhead, and resource usage.
For detail info and solution about SQLite visit the link below:
http://blog.developeronhire.com/what-is-sqlite-sqlite/
Thank you.
What the above two answers say. Expanding slightly on Chad Birch's answer, its teh calls to the SQLite db, and a rather poor implementation of sync() that causes FF3 to be so slow in linux.