I have a site where users can save images together with a couple of info fields about the image. The images are stored as binary data in the same table as the rest of the image info. So far pretty standard.
Now, the users should be able to upload a document, describing the image in more detail, along with the rest of the image and its' info. Size ~2MB per document.
My question is: Should I store this document (binary data) in the same table as the rest of the images or should I create a new table holding only the documents. There would of course be an id reference in the image table to the document in the document table.
I have a search function joining a couple of tables, including the image table, when searching for images. I need to know if there's a difference in efficiency between these two solutions.
I always fetch the document data separately but if I don't win anything in having the documents in a separate table I'll just put it with the rest of the image info.
I'm asking this since I don't really know how SQL Server handle tables when joing and searching in them.
I have about 10ยด000 users with a maximum of 10 images per user. (ASP.NET - SQL Server)
(I'm not asking if I should store documents in the database, but how ;)
Thanks in advance!
/zalk
Edit
Example 1:
columns in imageTable - id, title, dateAdded, image (binary data), document (binary data)
Searching for items from a specific user I would join the userTable and the imageTable and select title where image id equals user id.
So, will there be any difference in the performance if the document is in the imageTable or in an own table?
The metainformation about the image is information about the image. Hence; it should be part of the same table. Though there may be an argument if the metadata is used separately from the image.
Also, it is not uncommon to store the (relative) path to which the image file is stored in the database. This has the extra advantage that the path at which the image is stored can be served statically; rather then queried from a database and interpreted by ASP.
Related
This is my first time using a NOSQL database and I'm really struggling to work out how to structure my data.
I have an app that predicts a users mood and then the user can select if that's right or not. So I need to save both the prediction and the actual result. I want to be able to pull the latest result from firebase and display it on the app.
I understand how I'd do this on an SQL DB and understand how to write an SQL query to get that data back out.
For my Firebase DB I thought of the following structure
the document name is the usersID and store multiple arrays based on the timestamp but I can't seem to user OrderBy on a document only a collection so not sure how to get this back.
The fact that this seems so difficult less me to believe I've implemented the DB wrong to begin with.
Structure of DB is as follows:
I should add that it all works fine for the USER_TABLE as its one document id and a single entry, so I've no problem retrieving that.
Thanks for your help!
orderBy is an instruction to the database to order documents on the server, before it returns them to your app. To store the fields inside the document, you can just do that inside your application code after it receives the document(s).
There is in itself nothing wrong with storing these entries in a single document, Just keep in mind that:
A document can be at most be 1MB in size, so make sure this fits your maximum number of entries.
Firestore only ever returns full documents, so you will either get all entries in a document, or none of them.
You won't be able to order or filter the entries inside a single document. If that is a requirement for you, consider storing each entry in its own document in a subcollection. Note that this will increase the number of documents each user reads though, which will increase the cost.
I need my conversion table (we integrate data from other system where their values mean another thing in our AX instance) to be encompassing all companies.
When I deploy the project, I'll upload that table's data through an Excel import but I don't want to do it for all 5 of our companies.
I know when code runs as Admin, it fetches data from tables regardless of company (unless you specify so in the where clause) but I want standard users to see the table's data regardless of the company they are in when they run the code.
Is this possible ?
Thanks.
Yes, there s a property on tables called SaveDataPerCompany. The default is yes but if you change it to no, then essentially the DataAreaId field is no longer applicable and the same data will be seen in all companies. You change the property by finding the table in the AOT (e.g Data Dictionary > Tables) and right-clicking it and choosing Properties towards the bottom.
I'm developing a quick side project that needs a users table, and I want them to be able to store profile data. I was already reaching for the ASP.NET profile provider when I realized that users will only ever have one profile.
I realize that frequently changing data will impact performance on things like indexes and stuff but how frequent is too frequent?
If I have one profile change per month per user happening for say 1000 users, is that a lot?
Or are we talking more like users changing profile data on an hourly basis?
I realize this isn't an exact science but I'm trying to gauge at what point the threshold starts to peak, and since my users profile data will probably rarely change if I should bother the extra work or just wait a few decades for it to be a problem.
One thing to consider is how adding a large text column to a table will affect the layout of the rows. Some databases will store the large columns inlined with the other fixed size columns; this will make the rows variable sized and that means more work for the database when it needs to pull a row off the disk. Other databases (such as PostgreSQL) store large text columns away from the fixed size columns; this leads to fixed sized rows with quick access during table scans and the like but an extra bit of work is needed to pull out the text columns.
1000 users isn't that much in database terms so there's probably nothing to worry about one way or the other. OTOH, little one-off side projects have a nasty habit of turning into real mission critical projects when you're not looking so doing it right from the beginning is a good idea.
I think Justin Cave has covered the index issue well enough.
As long as you structure your data access properly (i.e. all access to your user table goes through one isolated pile of code) then changing your data schema for users won't be much work anyway.
Does the profile information actually need to be indexed? Or are you just going to be retrieving it based on the USER_ID of the table or some other indexed USER column? If the profile data isn't indexed, which seems likely to me, than there are no performance impacts to other indexes on the table.
The only reason I can think of to be concerned about putting profile information in the table is if there is a lot of data compared to the necessary information to define a user and if the USER table needs to be full scanned for some reason. In that case, increasing the size of the table would adversely affect the performance of a table scan. Assuming that you don't have a use case where it's regularly going to make sense to do a full scan on the USERS table, and given that the table will only have 1000 rows, that's probably not a big deal.
I'm programming site for ren-a-car company, and I need to save exactly three pictures for every car. What is better to store in database ( table description) path to images and images save in some folder OR to save pictures in table (MS SQL )?
The main question is: how big are those pictures on average??
A thorough examination by Microsoft Research (To BLOB or Not To BLOB: Large Object Storage in a Database or a Filesystem) has shown:
that it's better to store stuff inside your SQL tables as long as the data is typically smaller than 256 KB
that it's better to store on filesystem if the elements are typically larger than 1 MB
The in-between is a bit of a gray area....
So: are you pictures mostly more than 1 MB?? Then store on disk and keep a reference. Otherwise, I'd recommend storing them inside your database table.
Better to save the path to the picture (and probably even better just the filename usually). By storing the picture in the table, you are increasing the size of the tables and therefore increasing lookup time, insert time, etc for no apparent gain.
The other thing is that you say you need to save exactly 3 pictures for each posting, so this makes me think you're using fields in your posts table such as pic1, pic2, pic3. You might want to normalize this so that you have a post_pictures table that links each post with a picture.
For instance:
post_pictures:
post_picture_id | post_id | picture_filename. (you can even get away with having just two fields).
Depending on the horsepower of your server farm, save in SQL. I have set the up fields as varbinary(max).
What advantage do you have in storing the images in database? Can you think of any use of the image data being stored in a DB?
I suggest you store the images in file system rather than database. You can use the database to store meta data of the images (path,keywords etc) in the database.
I've searched through the site and haven't found a question/answer that quite answer my question, the closest one I found was: Syncing objects between two disparate systems best approach.
Anyway to begun, because there is no RSS feeds available, I'm screen scraping a webpage, hence it does a fetch then it goes through the webpage to scrap out all of the information that I'm interested in and dumps that information into a sqlite database so that I can query the information at my leisure without doing repeat fetching from the website.
However I'm also storing various metadata on the data itself that is stored in the sqlite db, such as: have I looked at the data, is the data new/old, bookmark to a chunk of data (Think of it as a collection of unrelated data, and the bookmark is just a pointer to where I am in processing/reading of the said data).
So right now my current problem is trying to figure out how to update the local sqlite database with new data and/or changed data from the website in a manner that is effective and straightforward.
Here's my current idea:
Download the page itself
Create a temporary table for the parsed data to go into
Do a comparison between the official and the temporary table and copy updates and/or new information to the official table
This process seems kind of complicated because I would have to figure out how to determine if the data in the temporary table is new, updated, or unchanged. So I am wondering if there isn't a better approach or if anyone has any suggestion on how to architecture/structure such system?
Edit 1:
I'm not sure where to put the additional information, in an comment or as an edit, so I'm going to add it here.
This expands a bit on the metadata in regards of bookmarking, basically the data source can create new data/addition to the current data, so one reason why I was thinking of doing the temporary table idea was so that I would be able to determine if an data source that has been "bookmarked" has any new data or not.
Is it really important to determine if the data in the temporary table is new, updated or unchanged? Do you really need to keep an history of the changes?
NO: don't use the temporary table but just mark as old (timestamp) your old records, don't do updates, and just insert your new data.
YES: your idea seems correct to me but all depends on how much data you need to process each time; i don't think it is feasible with a large amount of data.