Let me start by explaining my scenario:
I am developing a quiz engine in C# Asp.net. There is an option to display one question at a time. Since there will be a lot of users accessing the quiz, I do not want to make database calls for each question. Ideally I want to load all the questions on page load, and somehow display one question at a time.
Operations to be done for each question:
The question controls will be generated dynamically depending on question type.
After each question I need to save the user response.
What is the ideal way to do this by conserving database calls? I would really like some directions. Thanks in advance.
Unless you are expecting transaction rates in the high hundreds per second I would not bother.
Premature optimisation will cause real problems and only solve imaginary ones.
Modern DBMSes have very efficient caching mechanisims of thier own, so why re-invent the wheel? Keep your app simple and to the point, lets the DBMS handle caching and solve any performance problems when you actually have them.
I the set of questions is same for all users then you may consider using application wide caching of questions and responses. For this you may get all the questions in one go or fetch questions first time and then cache it.
For per user stuff like saving responses to questions, you can use session object. As you have the questions and their related options in application-wide cache, you can only save question key and selected option in session object.
Ideally I want to load all the
questions on page load, and somehow
display one question at a time.
That implies caching of some kind. Don't use session state, because you can't share it from one user to another.
You probably want to use the Cache object. For example, something like (from your code behind):
DataSet quiz;
string quizName = "quiz#1";
lock (quizLock)
{
quiz = (DataSet)this.Cache[quizName];
if (quiz == null)
{
quiz = DAL.ReadQuiz(quizName);
this.Cache.Insert(quizName, quiz);
}
}
should be relatively easy to achieve using JavaScript.
You simply grab all question store them in a local hashtable (simple javascript object) and display then however you like.
There is a jQuery Quiz plug in which you might use/reuse or just look and learn some tips and tricks (see this page as well)
You could store the questions for a user in a session variable, but you should consider how that affects memory usage. By storing a lot of data for each user, the memory available will limit the number of users that your application can handle.
By getting each question from the database, each page request will take slightly longer to handle, but you will shift the bottle neck from how many users you can handle to how many pages per second you can handle. This is also a softer limit, i.e. the server will just take longer to respond when you are at capacity, instead of starting to throw away user sessions because the memory is running out.
Related
The amount of coding that goes into the making of a DataSet is often significant. Now I'm not sure what the industry standard or best practise when dealing with data requests from multiple ASP.NET pages. Should I use a cache/session to pass on the DataSet from page to page or should I fetch directly from the database for each page?
What's the most common approach here?
Here are my thoughts:
It depends on the database and the type of data that you're trying to get, as well as what may modify the data. Do you have backend processes that run concurrent with the data you're going to want? Is this data only updated because of the current page, or does it update at all? How many people are going to use said page?
I personally almost always call to the database, simply because there are so many what-ifs when it comes to this kind of thing. At any time the data can change; it's never as static as people would think it would be. I would personally trade correct data over performance any day.
But that's just me personally. This question is so open ended that it's impossible to take every single thing into consideration since I don't know your database structure, nor how expensive it is to retrieve it, nor what you're using it for.
Sorry I couldn't really be more help.
It depends upon you need. If data size is very large then don't save it in Session or Cahce, because Session or Cache is stored in server Memory. Session is user specific and it will store data for each user in the server, so avoid from it. I think you should directly fetch data each time you need, don't save it in session. If data is very small/limited then you can save it in session ( example UserName or UserId etc ). If you are using a gridview to showdata then use paging and on each page request fetch the data from the database.
Here's my issue, we have a large patient object that is used on multiple screens throughout the admin. Each screen contains different information about the same patient. It can't all be on one screen.
The only time I want to persist the patient is when the user clicks save. I need to have an in memory patient somewhere. A user may be in the admin, change patient information on various screens, run validation and decide to not save that patient. This is typical use.
Is it ok to store this patient in the session? Or, is there a better approach to do this? At most this admin would have 20 users with access.
Opinions may vary on this. Session is tricky, especially if you use something other than in-memory session. Distributed session will break a non-serializable object. If this object is a simple POCO or object you control, try your best to make it play with serialization. If it does you're set. For an admin tool without much load I'd say you'd be fine.
Hey I found this - know nothing about the site, but illustrates my point:
https://www.fortify.com/vulncat/en/vulncat/dotnet/asp_dotnet_bad_practices_non_serializable_object_stored_in_session.html
I had a similar situation with similar amount of users. I did it and it worked great.
My situation was about scheduling events.
Someone would create an event and through multiple web pages would modify and configure this event. When they were all done it would save all the details to SQL. In the end, I was surprised just how well it worked.
Session should be fine here. You have what appears to be a light user load... but you might want to check exactly how much memory the object takes up, multiply that by the maximum number of users, and see where you are.
If you want to avoid the session altogether, you could use System.Web.Caching to store the object instead, and key the stored object using the users identifier plus some constant string.
In either case, you'll want to be aware of how many web servers are running the application. If it's just one web server, no worries. If you have multiple web servers, you'll want to make sure they are "sticky" - then the user is guaranteed to have all requests processed by the same server. How this is done is entirely dependent on your flavor of load balancing... normally the "IT folks" handle this for you.
So there seems not be any pretty answer to the question of how pass data between multiple pages. After having done a little homework here's why (or at least what I've gleaned):
ViewState variables don't persist across pages.
Session variables are volatile and must be used sparingly.
Cookies have potential safety issues and take time and must be kept small.
Storing vars in the URL has limits to the amount of data and can be unsafe.
Storing vars temporarily in a db is a real pita because you add one table per object that might be potentially passed to another page.
So far it is looking like I will be using hidden fields to pass a keyid and unique id to the next page and then retrieve the data from the db. What are your thoughts on all of this? What is the best way to go about doing any of it? I am early in the development of this app, so making changes now is preferred.
edit: I am anticipating a lot of users using this application at any one time, does that affect whether or not I should be using SQL Server based Session?
If you want to persist state, yes store it in the database since you don't have to worry about an expiration. The Session is similar except you have to worry about the Session expiring. In both cases concurrent calls that write similar data to the same area could cause problems and need to be accounted for.
Session is good when you don't have to worry about multiple web servers or timeout issues. The database gives you more scalability but at a cost of doing lots of db read/writes and you have to consider clean up.
Personally I would try to use the following decision tree:
Is the data simple, short and not private -> query string
Is the data less simple but only needs to exist for a short time -> session
Will the data be needed across multiple area and be persistent for long period of time -> database
Of course there is more to it that this but that should give you a basic outline of considerations since you are just starting out. Keep it simple. Don't try to over engineer a solution if a simple query string will suffice. You can always over engineer late as long as you have kept it simple to start.
I think context is important here, e.g. what are you trying to pass between pages and why?
If you are dealing with complex, multi-part forms, then you can implement the form in a single page, simply showing or hiding relevant element. Use usercontrols and custom controls as much as possible to facilitate isolation and reusability. This makes life a lot easier across the board.
Anything that is user-generated is almost certainly going to end up in a database anyway - so #5 does not seem relevant. That is you shouldn't have to store data "temporarily" in a database- what data would need to be persisted between pages that isn't part of your application.
Anything else would seem to be session related and not that much data.
I could add some more thoughts if I knew what specifically you were dealing with.
Oh - "cookies have potential safety issues and take time" - you're going to use cookies, unless you don't want to be able to identify return visitors. Any potential safety issues would only be a result of bad implementation, and certainly passing data in hidden fields is no better. And you really don't want to get into writing an ASP.NET app that is designed around pages posting to forms other than itself. That's just a headache for many reasons and I can't think of a benefit of doing this as part of basic application design.
Session variables should work fine for your needs.
I would go with StateServer or SQLServer Session state mode. Using mode InProc is the fastest, but it has some issues (including all user sessions getting dropped when a new binary is pushed, web.config changes, etc). Sessions are volatile, but you can control the volatility in several ways. Sessions require cookies unless they are configured as cookieless (which I highly recommend you stay away from), but I think that is a reasonable requirement.
Also, you can create a struct or serializable class from which you create objects that you can store in a session variable. The struct or class will allow you to keep all of your data in one place - you only have one session variable to worry about.
There is going to be advantages and disadvantages for any method, it's all about finding the best method. I hope this helps.
All methods have their pros and cons. It would all depend on the scenario you are working in.
Session variables work quite well if used within reason. InProc sessions in traffic heavy sites can quickly drain your resources but you can always switch to SQL Server based session that does most of the DB work for you.
I'm doing a survey builder, and I think to store the survey in session with a unique guid key until the user creates it fully and saves it I'm thinking it is going to be an array of 100~200 objects (8 properties class)
That sounds like a fair use of Session.
Whether your data is too large depends on quite a few things, such as your web server's memory. The best thing to do is test the performance using Session. If you find your data is too heavy for Session have a look at ASP.NET Profile.
That doesn't sound like that much data unless we are talking about reams of text for each answer. I would not worry about it unless I was working on a web site expected to have thousands of these open at any given point in time.
IMHO I think the data should be stored in something other than the Session.
Session objects can disappear for a myriad of reasons. Would your users be annoyed if their answers are not persisted and needs to start afresh.
Remember to write the data to a persistent store (DB, files, etc) as soon as possible unless the users don't mind starting over.
I agree with ggonsalv. I would store the data somewhere just in case the session is lost. I have been to sites where I fill stuff out and then it looses it near the end. It's not fun to start over.
Background
I am writing a survey that is going to a large audience. It contains 15 questions and there are five possible answers to each question along with potential comments.
The user can cycle through all 15 questions answering them in any order and is allowed to leave the survey at any point and return to answer the remaining questions.
Once an answer has been attempted on all 15 questions a submit button appears which allows them to submit the questions as final answers. Until that stage all answers are required to be retrievable whenever the user loads the survey page up.
The requirement is that the user only sees one question on a page and 'Previous' and 'Next' buttons allow the user to scroll through the questions.
Requirement
I could request the question each time the user clicks a button and save the current response and so on but that would be a large number of hits to a database that is already heavily used. I don't have the time to procure a new server etc so I have to make do with what I have. Is there any way I can cache the questions on the user machine and/or responses? Obviously I need the response data to be secure and only known to the user so I feel a little bit stuck as for the best way of doing this. Any pointers?
I am prepared to offer a bounty of 100 points on this question if it means I get some good quality discussion and feedback going.
Unless there's a reason for using a database, you could always store the results in flat files on the server itself. It doesn't sound like the data you're storing is relational in any way. Worst comes to worst, you could always insert them back into a relational db as a batch job every night.
Another option would be the application cache. However, if your web server suddenly crashes on you, you risk losing information from there.
You could also store the values in the user's cookies.
Based on my personal experience (serving thousands of short survey pages per second) I suspect your fears are unfounded. Among other reasons, the DBMS will cache such small amounts of data far more efficiently that you can.
I've tested this, loading the questions and answers into an Application-scope collection at start up, and serving them from memory after that - often it made no difference at all.
Your alternative is to send everything at once to the browser, and write it as a javascript application, storing the data in (encrypted) cookies and only hitting the database when the whole thing is done. This is tedious but not difficult.
You have three requirements that need to be balanced:
users must be able to return to their survey at any time
answers entered by users must be saved with the least possible chance of data loss
need to minimize database hits
Any solution that involves caching answers in a volatile place (cookies, session, etc) will increase the risk of data loss. The final solution depends on how you rank the three requirements in importance. If the db issue is at the top, then you will either need to risk data loss, or spend a lot of extra time coding a solution using some temporary storage scheme (like Kevin's flat file idea).
A couple of folks suggested that you may be optimizing prematurely. I suggest you consider that idea first - maybe this whole thing is moot.
However, assuming that your db situation is a real problem, I think your best balance of requirements will be a system that saves answer to the db immediately (to prevent data loss) but carefully manages when you actually have to hit the db.
When the app starts up (or when the first user requests the survey) load the survey and its questions into application cache. If any of the questions have a pick list of possible answers, load these also. You will only have to hit the db once during the application lifetime (or your cache duration) to load survey data.
When a user starts their survey, run a single query to load any existing answers (in case they are a returning user) into an object in session - could be as simple as a <List>string. (If you can somehow identify a new user without having to hit the db, then you can skip this step for new users.)
Use the session answer object along with the survey question object in app cache to populate each page without hitting the db again.
When the user submits an answer, compare it to the session answer object to see if it has changed (she may be just clicking 'next' on a page with a previously entered answer). If the answer is new, or has changed, the save it to the db and to the session answer object.
When the user leaves the survey, you don't need to do anything - everything is saved already.
With this scheme, you hit the db once to load the survey, once for each user when they start (or restart) the survey, and once for each new or modified answer. Probably not as much of a reduction as you were hoping for, but it gives you the best data protection.
If the database trips are a problem, you can cache them in the web server (or wherever your application resides) but it sounds like each answer needs to be recorded as the user goes to the next question.
If the questions and possible answers are identical for everyone, I would definitely cache them in the application layer - this can be stored in the Application object. In any case, you could certainly optimize the database calls to return the results as efficiently as possible - i.e. multiple result sets or a joined result set from a single stored proc. If you don't mind multiple copes for each session (or if there is variation), you can stored it in the Session object. Storing it on the client (i.e. a cookie) is not really secure and kind of pointless from a web server-client bandwidth saving persepective.
This sounds a lot like premature optimization to me, though.
Your scenario is a perfect candidate for Predictive Fetch Pattern. I would suggest that you cache all your questions. When the user signs in use the pattern to fetch the first 5 answers (if they have given any answers) and based on their navigation (where their current question is) get the information from the Response object or from the DB.
HTH
Not sure of the languages etc you are using, but most have an application cache. I would store the questions there, and retrieve them from the database and store them when they are not in the cache (when the application recycles).
As for the answers, are the users logging in some how? Is it feasible to save answers in a cookie until all questions are answered?
Edit:
If cookies aren't reliable enough, you could store (in the application cache) a list of queries (inserts/updates) to be executed, they would not be executed until an a query limit was reached or under certain conditions (i.e. execute the query list when a user requests answers that are in the list, execute list when the application recycles, etc).
Pretty crude, but you get the idea:
if (function == "get question" && userQuestionIsInQueue) || function == "finish survey"
execute(Application["querylist"]);
continue as normal...
if function == "submit answer"
if Application["querylist"] == null
Application["querylist"] = newAnswerQuery;
else
Application["querylist"] += newAnswerQuery;
You'd also need to add execute(Application["querylist"]) to the recycle event, I believe you can hook it in the global.asax
Edit 2:
I would also accumulate all database transactions for a request into 1, you if you did have to execute the list, then followed by getting the answer for the user, do them in the same transaction and save a trip. Common practice when optimizing.
This is a classic problem to do with maintaining state between pages in a browser based system. Im also assuming that we want this data to persist even if the user logs out and comes back later. Here are the options:
With a high availability server we can keep a single collection of 15 answers in memory (not session) for this user (probably not a good idea and not easily load balanced)
We denormalise the 15 answers into 1 row of a sql table
We persist the data on the client using a cookie or localStorage (IE8).
My feeling is that the first two options are probably not what you are looking for, so lets explore the last option.
You could quite simply store the answers in a cookie. There is a small chance that this could get lost, and that the user may log in from another machine, but this may be an acceptable risk. With with latest browsers that support HTML5 (inc IE8 afaik) you get the benefit of localStorage which is not as easily deleted as a cookie. You could fall back to cookies if this wasnt available.
Cookies can be encrypted if required.
I would like to offer you the new feature of HTML5 which is called Dom Storage but since only the new browsers are supporting it, it could be a problem using it at this point.
With DOM Storage, you can store data on user browser. Since it can store up to 5MB per domain in Mozilla Firefox[3], Google Chrome, and Opera, 10MB per storage area in Internet Explorer, you can store answers and question ids in the DOM Storage.
Even with DOM Storage, let alone Database hit, you can reduce server hits as well.
Since we all know working with cookies is hassle sometimes and it can store 4kb, the easiest way is now to store key-value information in DOM Storage.
You can store key-value information specifically for sessions as well as locally. When session ends, the session based info will be wiped off from the browser but if you store local based values, even the user closes the tab, the key-value will remain for a while.
Example Code:
<p>
You have viewed this page
<span id="count">an untold number of</span>
time(s).
</p>
<script>
var storage = window.localStorage;
if (!storage.pageLoadCount) storage.pageLoadCount = 0;
storage.pageLoadCount = parseInt(storage.pageLoadCount, 10) + 1;
document.getElementById('count').innerHTML = storage.pageLoadCount;
</script>
You can learn more about DOM Storage from the links below :
https://developer.mozilla.org/en/DOM/Storage
http://en.wikipedia.org/wiki/Web_Storage
http://msdn.microsoft.com/en-us/library/cc197062%28VS.85%29.aspx
do you mean...a cookie?