I'm programming site for ren-a-car company, and I need to save exactly three pictures for every car. What is better to store in database ( table description) path to images and images save in some folder OR to save pictures in table (MS SQL )?
The main question is: how big are those pictures on average??
A thorough examination by Microsoft Research (To BLOB or Not To BLOB: Large Object Storage in a Database or a Filesystem) has shown:
that it's better to store stuff inside your SQL tables as long as the data is typically smaller than 256 KB
that it's better to store on filesystem if the elements are typically larger than 1 MB
The in-between is a bit of a gray area....
So: are you pictures mostly more than 1 MB?? Then store on disk and keep a reference. Otherwise, I'd recommend storing them inside your database table.
Better to save the path to the picture (and probably even better just the filename usually). By storing the picture in the table, you are increasing the size of the tables and therefore increasing lookup time, insert time, etc for no apparent gain.
The other thing is that you say you need to save exactly 3 pictures for each posting, so this makes me think you're using fields in your posts table such as pic1, pic2, pic3. You might want to normalize this so that you have a post_pictures table that links each post with a picture.
For instance:
post_pictures:
post_picture_id | post_id | picture_filename. (you can even get away with having just two fields).
Depending on the horsepower of your server farm, save in SQL. I have set the up fields as varbinary(max).
What advantage do you have in storing the images in database? Can you think of any use of the image data being stored in a DB?
I suggest you store the images in file system rather than database. You can use the database to store meta data of the images (path,keywords etc) in the database.
Related
We can save an image with 2 way
upload image in Server and save image url in Database.
save directly image into database
which one is better?
There's a really good paper by Microsoft Research called To Blob or Not To Blob.
Their conclusion after a large number of performance tests and analysis is this:
if your pictures or document are typically below 256K in size, storing them in a database VARBINARY column is more efficient
if your pictures or document are typically over 1 MB in size, storing them in the filesystem is more efficient (and with SQL Server 2008's FILESTREAM attribute, they're still under transactional control and part of the database)
in between those two, it's a bit of a toss-up depending on your use
If you decide to put your pictures into a SQL Server table, I would strongly recommend using a separate table for storing those pictures - do not store the employee foto in the employee table - keep them in a separate table. That way, the Employee table can stay lean and mean and very efficient, assuming you don't always need to select the employee foto, too, as part of your queries.
For filegroups, check out Files and Filegroup Architecture for an intro. Basically, you would either create your database with a separate filegroup for large data structures right from the beginning, or add an additional filegroup later. Let's call it "LARGE_DATA".
Now, whenever you have a new table to create which needs to store VARCHAR(MAX) or VARBINARY(MAX) columns, you can specify this file group for the large data:
CREATE TABLE dbo.YourTable
(....... define the fields here ......)
ON Data -- the basic "Data" filegroup for the regular data
TEXTIMAGE_ON LARGE_DATA -- the filegroup for large chunks of data
Check out the MSDN intro on filegroups, and play around with it!
Like many questions, the ansewr is "it depends." Systems like SharePoint use option 2. Many ticket tracking systems (I know for sure Trac does this) use option 1.
Think also of any (potential) limitations. As your volume increases, are you going to be limited by the size of your database? This has particular relevance to hosted databases and applications where increasing the size of your database is much more expensive than increasing your storage allotment.
Saving the image to the server will work better for a website, given that these are incidental to your website, like per customer branding images - if you're setting up the next Flickr obviously the answer would be different :). You'd want to set up one server to act as a file server, share out the /uploaded_images directory (or whatever you name it), and set up an application variable defining the base url of uploaded images. Why is it better? Cost. File servers are dirt cheap commodity hardware. You can back up the file contents using dirt cheap commodity (even just consumer grade) backup software. And if your file server croaks and someone loses a day of uploaded images? Who cares. They just upload them again. Our database server is an enterprise cluster running on SSD SAN. Our backups and tran logs are shipped to remote sites over expensive bandwidth and maintained even on tape for x period. We use it for all the data where we need the ACID (atomicity, consistency, isolation, durability) benefits of a RDBMS. We don't use it for company logos.
Store them in the database unless you have a good reason not to.
Storing them in the filesystem is premature optimization.
With a database you get referential integrity, you can back everything up at once, integrated security, etc.
The book SQL Anti-Patterns calls storing files in the filesystem an anti-pattern.
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Storing Images in DB - Yea or Nay?
Friends,
I have a requirement to show dynamically changing images in DataList. what I did was I am storing the images in DB as image datatype and retrieving the images. Is that good technique to store images in DB?
FYI, user can upload the images.
Regards,
Abhi
The answer is - it depends... Studies have been done (http://research.microsoft.com/pubs/64525/tr-2006-45.pdf) which have basically concluded if objects are larger than one megabyte on average, NTFS has a clear advantage over SQL Server. If the objects are under 256 kilobytes, the database has a clear advantage. Inside this range, it depends on how write intensive the workload is, and the storage age of a typical replica in the system.
I would store them as physical files on the server, but store the file path in the database, not the actual image. Storing the image in the database will increase it's size dramatically over time.
Images sizes will increase your DB size unnecessarily so not good practice to store it, instead store the file path in your db, which is not that big.
Storing image in DB should be done if you have some strong requirement or use case.
The thing you have to address if you store paths etc. is maintaining referential integrity with your images. What if somebody moves files, what if somebody uploads a new file with the same name (I'd suggest uploads get renamed to reflect some kind of key rather than keeping their original name of bob.jpg). You'll need to look at segmenting your directories etc. to keep the list sensible. Finding the images may be harder than if you store them in a DB also.
However, on the up side, you can form a CDN based on distribution of your images over diverse servers, subdomains, cloud etc. if you don't jam them all in your database
Depends on the size of the images and the DB you use.
For SQL Server it is pretty bad idea if they are larger than 1MB and you do not use the NTFS Filestreams for storage of your BLOB fields.
See for example http://www.simple-talk.com/sql/learn-sql-server/an-introduction-to-sql-server-filestream/
If you have a document oriented database like Couch DB it might be ok.
I would store them as physical files on the server, but store the file path in the database, not the actual image. And search the file as per the location store into Databasse. Storing the image in the database will increase it's size dramatically over time.
Storing them in database is also useful if you need to scale your site across multiple web servers.
If they are static then there is no use as they can be deployed with your site but things like avatars are generally better stored in the DB so they are available to all cluster members.
I'm developing a quick side project that needs a users table, and I want them to be able to store profile data. I was already reaching for the ASP.NET profile provider when I realized that users will only ever have one profile.
I realize that frequently changing data will impact performance on things like indexes and stuff but how frequent is too frequent?
If I have one profile change per month per user happening for say 1000 users, is that a lot?
Or are we talking more like users changing profile data on an hourly basis?
I realize this isn't an exact science but I'm trying to gauge at what point the threshold starts to peak, and since my users profile data will probably rarely change if I should bother the extra work or just wait a few decades for it to be a problem.
One thing to consider is how adding a large text column to a table will affect the layout of the rows. Some databases will store the large columns inlined with the other fixed size columns; this will make the rows variable sized and that means more work for the database when it needs to pull a row off the disk. Other databases (such as PostgreSQL) store large text columns away from the fixed size columns; this leads to fixed sized rows with quick access during table scans and the like but an extra bit of work is needed to pull out the text columns.
1000 users isn't that much in database terms so there's probably nothing to worry about one way or the other. OTOH, little one-off side projects have a nasty habit of turning into real mission critical projects when you're not looking so doing it right from the beginning is a good idea.
I think Justin Cave has covered the index issue well enough.
As long as you structure your data access properly (i.e. all access to your user table goes through one isolated pile of code) then changing your data schema for users won't be much work anyway.
Does the profile information actually need to be indexed? Or are you just going to be retrieving it based on the USER_ID of the table or some other indexed USER column? If the profile data isn't indexed, which seems likely to me, than there are no performance impacts to other indexes on the table.
The only reason I can think of to be concerned about putting profile information in the table is if there is a lot of data compared to the necessary information to define a user and if the USER table needs to be full scanned for some reason. In that case, increasing the size of the table would adversely affect the performance of a table scan. Assuming that you don't have a use case where it's regularly going to make sense to do a full scan on the USERS table, and given that the table will only have 1000 rows, that's probably not a big deal.
I have a site where users can save images together with a couple of info fields about the image. The images are stored as binary data in the same table as the rest of the image info. So far pretty standard.
Now, the users should be able to upload a document, describing the image in more detail, along with the rest of the image and its' info. Size ~2MB per document.
My question is: Should I store this document (binary data) in the same table as the rest of the images or should I create a new table holding only the documents. There would of course be an id reference in the image table to the document in the document table.
I have a search function joining a couple of tables, including the image table, when searching for images. I need to know if there's a difference in efficiency between these two solutions.
I always fetch the document data separately but if I don't win anything in having the documents in a separate table I'll just put it with the rest of the image info.
I'm asking this since I don't really know how SQL Server handle tables when joing and searching in them.
I have about 10ยด000 users with a maximum of 10 images per user. (ASP.NET - SQL Server)
(I'm not asking if I should store documents in the database, but how ;)
Thanks in advance!
/zalk
Edit
Example 1:
columns in imageTable - id, title, dateAdded, image (binary data), document (binary data)
Searching for items from a specific user I would join the userTable and the imageTable and select title where image id equals user id.
So, will there be any difference in the performance if the document is in the imageTable or in an own table?
The metainformation about the image is information about the image. Hence; it should be part of the same table. Though there may be an argument if the metadata is used separately from the image.
Also, it is not uncommon to store the (relative) path to which the image file is stored in the database. This has the extra advantage that the path at which the image is stored can be served statically; rather then queried from a database and interpreted by ASP.
I've searched through the site and haven't found a question/answer that quite answer my question, the closest one I found was: Syncing objects between two disparate systems best approach.
Anyway to begun, because there is no RSS feeds available, I'm screen scraping a webpage, hence it does a fetch then it goes through the webpage to scrap out all of the information that I'm interested in and dumps that information into a sqlite database so that I can query the information at my leisure without doing repeat fetching from the website.
However I'm also storing various metadata on the data itself that is stored in the sqlite db, such as: have I looked at the data, is the data new/old, bookmark to a chunk of data (Think of it as a collection of unrelated data, and the bookmark is just a pointer to where I am in processing/reading of the said data).
So right now my current problem is trying to figure out how to update the local sqlite database with new data and/or changed data from the website in a manner that is effective and straightforward.
Here's my current idea:
Download the page itself
Create a temporary table for the parsed data to go into
Do a comparison between the official and the temporary table and copy updates and/or new information to the official table
This process seems kind of complicated because I would have to figure out how to determine if the data in the temporary table is new, updated, or unchanged. So I am wondering if there isn't a better approach or if anyone has any suggestion on how to architecture/structure such system?
Edit 1:
I'm not sure where to put the additional information, in an comment or as an edit, so I'm going to add it here.
This expands a bit on the metadata in regards of bookmarking, basically the data source can create new data/addition to the current data, so one reason why I was thinking of doing the temporary table idea was so that I would be able to determine if an data source that has been "bookmarked" has any new data or not.
Is it really important to determine if the data in the temporary table is new, updated or unchanged? Do you really need to keep an history of the changes?
NO: don't use the temporary table but just mark as old (timestamp) your old records, don't do updates, and just insert your new data.
YES: your idea seems correct to me but all depends on how much data you need to process each time; i don't think it is feasible with a large amount of data.