I have a web application at work that is similar to a ticket working system. Some users enter new issues. Other workers choose and resolve issues. All of the data is maintained in MS SQL server 2005.
The users working to resolve issues go to a page where they can view open issues. Because up to twenty people can be looking at this page at the same time, one potential problem I had to address was what happens if someone picks an issue that someone else picked just after their page loaded.
To address this, I did two things. First, the gridview displaying the issues to select uses an AJAX timer to update every second. Once an issue has been selected, it disappears one second later at most. In case they select one within this second, they get a message asking them to choose another.
The problem is that the AJAX part of this is sending too many updates (this is what I am assuming) and it is affecting the performance of the page and database. In addition, the updates are not performing every second. I find the timer to be unreliable when working to trigger stored procedures.
There has to be a better way, but I can't seem to find one. Does anyone have experience with a situation like this or have suggestions to keep multiple users from selecting the same record to maintain? I really do not want to disable the AJAX part entirely because I feel the message alone would make the application frustrating to use.
Thanks,
Put a lock timestamp field on the row in the database. Write a stored proc that returns true or false if the expiration timsetamp is older than a specific time. Set your sessions on your web app to expire in the same time, a minute or two. When a user select a row they hit the stored proc which helps the app to decide if it should let the user to modify it.
Hope that makes sense....
Two things can help mitigate your problem.
First, after-selection notification that the case has been taken is needed regardless of your ajax update time frame. Even checking every second doesn't mean two people cannot click the same case at what they perceive to be the same time. In such cases, one of the users needs to be notified that their selection is invalid even though it appeared valid when selected. This notification doesn't need to be elaborate; keeping a light, helpful tone can improve user perception even in the light of disappointment. And if you identify the user who selected that record already, that will not only help your users coordinate in future but also divert attention from your program to the user who snaked the juicy case. (indeed, management may like giving your users the occasional collision as it will motivate them to select cases faster)
Second, a small tweak to how you display your cases can reduce selection collisions. Adding a random element to display order and/or filtering out every other case on display will help your users select different cases naturally. Human pattern recognition and task selection isn't really random so small changes to presentation can equal big changes to selection behavior. Reductions in collision chance keeps your collision notifications rare (and thus less frustrating to your users). This is even better if your users can be separated into classifications that can help determine useful case ordering/filtering.
Okay, a third thing that will help you over time is if you keep a log of when collisions occur (with helpful meta data about the collision—like who was involved and selection timing). Armed with solid collision data, you can find what works and what doesn't. Over time, you can hone your application to your actual use cases as well as identify potential problems early. Nothing reassures your users more than being on top of a problem (and able to explain your plans to solve it) before they're even aware it exists.
With these mitigating patterns, you'll probably find you can safely reduce your ajax query timeframe without affecting user experience. And with useful logging, you'll have the assurance that any tweaks you put in place are actually working (or not—which is maybe even more useful to know).
I did something similar where once a user opened a ticket (row) it assigned that ticket to that user and set a value on that record, like and FK to that particular user, so if anyone else tried to open that ticket (row) it would let them know it has already been assigned to someone else.
If possible limit the system so that they just get the next open issue off the work queue as opposed having them be able choose from all open issues.
If that isn't possible, I suppose you could check upon the choosing of an issue to see if it is still available. If it's not available, then make it disappear after the user clicks on it. This way you are only requesting when they actually click on something as opposed to constant polling of the data.
Have you tried increasing the time between refreshes. I would expect that once per 30 seconds would be sufficient. 40 requests/minute is a lot less load than 1200/minute. Your users may not even notice the difference.
If they do, how about providing a refresh button on the page so the users can manually refresh the list just prior to selecting an item to avoid the annoying message if they choose.
I'm missing to see the issue, specially after you mentioned you are already flagging tickets as in progress/being maintained and have a timestamp/version of the item.
Isn't the following enough:
User browses the tickets and sees a list of available tickets i.e. this excludes ones that are in the db as in progress. If you want the users to also see tickets in progress, you indicate it clearly in the ticket status and disable the option to take it.
User either flags a ticket as in progress explicitly or implicitly by opening the ticket (depends on the user experience / how its presented to the users).
User explicitly moves the ticket to a different status i.e. completed, invalid, awaiting for feedback, etc.
When the items are retrieved at 1, you include a timestamp/version. When 2 happens, you use a optimistic concurrency approach to make sure that if 2 persons try to update the take the ticket at the same time only the first one will be successful.
What will happen is that for the second person, the update ... where ... timestamp = #timestamp will not find any records to update and you will report back that the ticket was already taken.
If you want, you can build on top of the above to update the UI as tickets are grabbed. This could be by just doing a full refresh of the current page of tickets after x time (maybe alerting/prompting the user), or even by retrieving a list of tickets changed for the page of tickets being showed with ajax. You still have the earlier steps in place, as this modification its just a convenience for the users.
Related
Can anyone help or provide me with some suggestions for the below query.
I have a web form (Minutes of Meeting) and 8 users that need to access this web page and update their area. A user may have more than one area to update and essentially i would like to some how lock down the web page if possible when a user is using it so that no other user can update this web page till joe bloggs has finished with it.
I have a Active Directory security group set up to restrict the site to that group of users only, but i need to think of a solution to the above?
Is there a way i can do this via a web control or via SQL?
There must be better ways to do it. However, Is it possible for you to introduce a sql table column similar to "UpdateInProgress" (bit). Any update process sees that column, If 0 then It updates to 1 and after It saves the changes and updates back to 0 so that the form is available for other to update. If update process sees 1, It can't update the web form because update is in progress.
I also suggest to introduce another column named "UpdateInProgressBy" to check who has opened it for editing.
First of all we must note that there is a big time from the moment the user reads the data, get it in a page, change them and then try to write them back. So we are not talking for the lock command on SQL, nether any other lock that happens in milliseconds and help to synchronize threads, but here we must synchronize people and what they write.
There is also a problem if the user leave the page for any reason and this can make the data lock for ever.
This problem can solve with two approaches.
the easy one, when a user try to save data you must check if the same data have been change in the middle, and warn him, or show a merge dialog, or merge programmaticall, or something similar - I do not know what you won.
the difficult way is to constantly monitor the page that read and change the data, and keep this monitor results on a common table in the data base, and there if a user have been and stay on page, the rest users get a warning and read only data, until the user go.
This monitor must be made with javasript and must know even if a user abandon the page.
SET TRANSACTION ISOLATION LEVEL as SERIALIZABLE
for more information check this link:
http://msdn.microsoft.com/en-us/library/ms173763.aspx
Background
I am writing a survey that is going to a large audience. It contains 15 questions and there are five possible answers to each question along with potential comments.
The user can cycle through all 15 questions answering them in any order and is allowed to leave the survey at any point and return to answer the remaining questions.
Once an answer has been attempted on all 15 questions a submit button appears which allows them to submit the questions as final answers. Until that stage all answers are required to be retrievable whenever the user loads the survey page up.
The requirement is that the user only sees one question on a page and 'Previous' and 'Next' buttons allow the user to scroll through the questions.
Requirement
I could request the question each time the user clicks a button and save the current response and so on but that would be a large number of hits to a database that is already heavily used. I don't have the time to procure a new server etc so I have to make do with what I have. Is there any way I can cache the questions on the user machine and/or responses? Obviously I need the response data to be secure and only known to the user so I feel a little bit stuck as for the best way of doing this. Any pointers?
I am prepared to offer a bounty of 100 points on this question if it means I get some good quality discussion and feedback going.
Unless there's a reason for using a database, you could always store the results in flat files on the server itself. It doesn't sound like the data you're storing is relational in any way. Worst comes to worst, you could always insert them back into a relational db as a batch job every night.
Another option would be the application cache. However, if your web server suddenly crashes on you, you risk losing information from there.
You could also store the values in the user's cookies.
Based on my personal experience (serving thousands of short survey pages per second) I suspect your fears are unfounded. Among other reasons, the DBMS will cache such small amounts of data far more efficiently that you can.
I've tested this, loading the questions and answers into an Application-scope collection at start up, and serving them from memory after that - often it made no difference at all.
Your alternative is to send everything at once to the browser, and write it as a javascript application, storing the data in (encrypted) cookies and only hitting the database when the whole thing is done. This is tedious but not difficult.
You have three requirements that need to be balanced:
users must be able to return to their survey at any time
answers entered by users must be saved with the least possible chance of data loss
need to minimize database hits
Any solution that involves caching answers in a volatile place (cookies, session, etc) will increase the risk of data loss. The final solution depends on how you rank the three requirements in importance. If the db issue is at the top, then you will either need to risk data loss, or spend a lot of extra time coding a solution using some temporary storage scheme (like Kevin's flat file idea).
A couple of folks suggested that you may be optimizing prematurely. I suggest you consider that idea first - maybe this whole thing is moot.
However, assuming that your db situation is a real problem, I think your best balance of requirements will be a system that saves answer to the db immediately (to prevent data loss) but carefully manages when you actually have to hit the db.
When the app starts up (or when the first user requests the survey) load the survey and its questions into application cache. If any of the questions have a pick list of possible answers, load these also. You will only have to hit the db once during the application lifetime (or your cache duration) to load survey data.
When a user starts their survey, run a single query to load any existing answers (in case they are a returning user) into an object in session - could be as simple as a <List>string. (If you can somehow identify a new user without having to hit the db, then you can skip this step for new users.)
Use the session answer object along with the survey question object in app cache to populate each page without hitting the db again.
When the user submits an answer, compare it to the session answer object to see if it has changed (she may be just clicking 'next' on a page with a previously entered answer). If the answer is new, or has changed, the save it to the db and to the session answer object.
When the user leaves the survey, you don't need to do anything - everything is saved already.
With this scheme, you hit the db once to load the survey, once for each user when they start (or restart) the survey, and once for each new or modified answer. Probably not as much of a reduction as you were hoping for, but it gives you the best data protection.
If the database trips are a problem, you can cache them in the web server (or wherever your application resides) but it sounds like each answer needs to be recorded as the user goes to the next question.
If the questions and possible answers are identical for everyone, I would definitely cache them in the application layer - this can be stored in the Application object. In any case, you could certainly optimize the database calls to return the results as efficiently as possible - i.e. multiple result sets or a joined result set from a single stored proc. If you don't mind multiple copes for each session (or if there is variation), you can stored it in the Session object. Storing it on the client (i.e. a cookie) is not really secure and kind of pointless from a web server-client bandwidth saving persepective.
This sounds a lot like premature optimization to me, though.
Your scenario is a perfect candidate for Predictive Fetch Pattern. I would suggest that you cache all your questions. When the user signs in use the pattern to fetch the first 5 answers (if they have given any answers) and based on their navigation (where their current question is) get the information from the Response object or from the DB.
HTH
Not sure of the languages etc you are using, but most have an application cache. I would store the questions there, and retrieve them from the database and store them when they are not in the cache (when the application recycles).
As for the answers, are the users logging in some how? Is it feasible to save answers in a cookie until all questions are answered?
Edit:
If cookies aren't reliable enough, you could store (in the application cache) a list of queries (inserts/updates) to be executed, they would not be executed until an a query limit was reached or under certain conditions (i.e. execute the query list when a user requests answers that are in the list, execute list when the application recycles, etc).
Pretty crude, but you get the idea:
if (function == "get question" && userQuestionIsInQueue) || function == "finish survey"
execute(Application["querylist"]);
continue as normal...
if function == "submit answer"
if Application["querylist"] == null
Application["querylist"] = newAnswerQuery;
else
Application["querylist"] += newAnswerQuery;
You'd also need to add execute(Application["querylist"]) to the recycle event, I believe you can hook it in the global.asax
Edit 2:
I would also accumulate all database transactions for a request into 1, you if you did have to execute the list, then followed by getting the answer for the user, do them in the same transaction and save a trip. Common practice when optimizing.
This is a classic problem to do with maintaining state between pages in a browser based system. Im also assuming that we want this data to persist even if the user logs out and comes back later. Here are the options:
With a high availability server we can keep a single collection of 15 answers in memory (not session) for this user (probably not a good idea and not easily load balanced)
We denormalise the 15 answers into 1 row of a sql table
We persist the data on the client using a cookie or localStorage (IE8).
My feeling is that the first two options are probably not what you are looking for, so lets explore the last option.
You could quite simply store the answers in a cookie. There is a small chance that this could get lost, and that the user may log in from another machine, but this may be an acceptable risk. With with latest browsers that support HTML5 (inc IE8 afaik) you get the benefit of localStorage which is not as easily deleted as a cookie. You could fall back to cookies if this wasnt available.
Cookies can be encrypted if required.
I would like to offer you the new feature of HTML5 which is called Dom Storage but since only the new browsers are supporting it, it could be a problem using it at this point.
With DOM Storage, you can store data on user browser. Since it can store up to 5MB per domain in Mozilla Firefox[3], Google Chrome, and Opera, 10MB per storage area in Internet Explorer, you can store answers and question ids in the DOM Storage.
Even with DOM Storage, let alone Database hit, you can reduce server hits as well.
Since we all know working with cookies is hassle sometimes and it can store 4kb, the easiest way is now to store key-value information in DOM Storage.
You can store key-value information specifically for sessions as well as locally. When session ends, the session based info will be wiped off from the browser but if you store local based values, even the user closes the tab, the key-value will remain for a while.
Example Code:
<p>
You have viewed this page
<span id="count">an untold number of</span>
time(s).
</p>
<script>
var storage = window.localStorage;
if (!storage.pageLoadCount) storage.pageLoadCount = 0;
storage.pageLoadCount = parseInt(storage.pageLoadCount, 10) + 1;
document.getElementById('count').innerHTML = storage.pageLoadCount;
</script>
You can learn more about DOM Storage from the links below :
https://developer.mozilla.org/en/DOM/Storage
http://en.wikipedia.org/wiki/Web_Storage
http://msdn.microsoft.com/en-us/library/cc197062%28VS.85%29.aspx
do you mean...a cookie?
I have a requirement that my site always display the number of users currently online. For example, "35741 Users Currently Online". This is not based on a log in, simply how many users are currently on my site. I have tried using Session Start/Session End for this, however session end is not reliable. Therefore I get inflated numbers, as my session start adds numbers but session end doesn't remove them because it doesn't fire.
There is no additional information to be gathered from this (reporting, etc), it's simply requested that the number show up. Very simple request that's turning into a huge deal. Any help is appreciated.
EDIT:
I should specify that I have also tried using a database for this. Simple table that contains a session ID and a last activity column. With each page hit, I check to see if the session is in my database. If not, insert. If so, update with activity time. Then I run a procedure that sweeps the database looking for sessions with no activity in the last 20 minutes. This approach seemed to kill my SQL server and/or IIS. Had to restart the site.
Best way is like you do, but time it out via activity. If a given session doesn't access a page within 5 minutes or so, you may consider them no longer active.
If you're using ASP.Net membership, take a look at GetNumberOfUsersOnline.
For every user action that you can record, you need to consider them "online" for a certain window of time. Depending on the site, you may set that to 5 minutes. The actual web request should take less than a second. You have to make some assumption about how long they might stay on that page and do nothing but be considered online.
This approach requires that you keep track of the time of each users last activity.
Use Performance Counters:
State Server Sessions Active: The
number of active user sessions.
Expanding what silky said in his answer - since really http is stateless to determine if the user is currently 'online' you can really only track how long since the user last accessed your site and make a determination on how long between requests your consider to still be active.
Since you stated that this isn't based upon users logging in may it's a simple of how many different IP addresses you received requests from in the past 5 minutes (or however long you consider the 'online' timeout to be).
Don't use sessions for this unless you also need sessions for something else; it's overkill otherwise.
Assuming a single-server installation, do something like this:
For each user, issue a cookie that contains a unique ID
Maintain a static table of unique IDs and their last access time
In an HttpModule (or Global.asax), enter new users into the table and update their access times (use appropriate locking to prevent race conditions)
Periodically, either from a background thread or in-line with a user request, remove entries from the table that haven't made a request within the last N minutes. You might also want to support an explicit "log out" feature.
Report the number of people online as the size of the table
If you do use sessions, you can use the Session ID as the unique identifier. However, keep in mind that Session IDs aren't issued until you store something in the Session dictionary, unless you have a Session_Start() event configured.
In a load balanced or web garden scenario, it gets a little more complicated, but you can use the same basic idea, just persisting the info in a database instead of in memory.
When the user logs in write his user name into the HttpContext.Current.Cache with a sliding expiration (say 20 minutes).
Then in the Global.asax.cs in the Application_PreRequestHandlerExecute "touch" the cache entry for the current users so it resets the sliding expiration.
When a user explicitly logs out, remove his username from HttpContext.Current.Cache.
If you do this, at any given time HttpContext.Current.Cache.Count will give you the # of current users.
Note: this is assuming you aren't using the Cache for other purposes.
I have an ASP.NET application that uses Session.SessionID to prevent multiple users viewing the same data at the same time.
I have a table that contains a set of images (stored in BLOB) that require processing. Only one user is supposed to be able to view the same image at the same time. To achieve this, as each record is retrieved by a user the record is updated with the Session.SessionID. This update occurs inside a ReaderWriterLock.
I have done a test to ensure the ReaderWriterLock is working correctly and can confirm that only one session can execute the code inside that block at once.
My current theory is that two different users are getting the same SessionID at the same time. A user of this application is allowed to view records they have locked or any unlocked images.
I have modified the application to display the SessionID in the footer of every page so that if the problem happens again I can check the SessionID value.
I've seen some articles online suggesting that SessionID is not unique and some saying that the SessionID is unique. I understand that SessionID is not unique forever but can the SessionID value be considered unique for active sessions?
This forum describes a similar problem
I have also read some suggestions that a Guid should be stored in the Session object and used as a unique ID instead of the Session ID.
Thanks for the responses so far. Here is a clarification based on the answers so far:
"Locked forever" - we prevent this by a lock timeout of 5 minutes. Before a user locks an image, while inside the ReaderWriterLock, we do a "cleanup" of old locks (which unlocks images locked for more than 5 minutes), a query to get the oldest unlocked image and an update statement to "lock" that image to the current session.
A possible cause of the problem would be if one user "locks" an image but then leaves the PC for a short break. If they did nothing for 5 minutes, that image on there screen would be unlocked and potentially opened by another user. I mentioned this scenario when the problem was reported and I was assured that the users had been working continuously.
"Different Window/Tab" - I haven't actually seen the error with my own eyes but the person who reported the problem has told me that it is two different PC's and two different usernames of the logged in user.
Hopefully now that I am displaying the Session ID on the page, next time it happens I will be able to say with certainty whether it is the same Session ID on two machines or if it is some other problem. This issue has never occurred during the testing phase so it appears to be a symptom of a larger number of concurrent users.
Thanks for the responses so far and I will update this question as more information comes to hand.
It seems that the user didn't give me the full story. Session ID is unique in our case as per the accepted answer. Two users were able to see the same image at the same time because user 1 was idle for the 5 minute "abandoned image" unlock process. The "abandoned image" timeout has been raised to match the session timeout to avoid this problem.
Session ID is unique per user as far as ASP.NET assigns them, though that is not a guarantee against malicious users (a user could manually copy the ID they've been assigned and give it to someone else).
What you are likely seeing here is multiple tabs or windows from the same user, as it is perfectly valid for a user to be making more then one request at a time.
To do what you want I would have to ask - how do you know when a user has stopped looking at an image, so you can unlock it. What if I view an image, and then just close my browser instead of going to another page/image, does it remain locked to my (lost forever) session id?
If you are using some kind of checkout scheme - the user must intentionally check out and check in an image, then you should perhaps be using a unique number for that checkout (new Guid), rather then the whole user.
Red flag here people.
Yes you can but it's absolutely essential that you can't or at the very least make it as difficult as possible to re-use them in the future.
This is dependent on how well your server side application is handling Session ID's and maintaining State. You should (actually MUST) think about limiting session_ID lifetime, how you look after authentication /authorisation state.
I saw in a critical trading platform at a top 10 investment bank whereby you could capture Session ID's on the fly (that contained authorised permissions) and re-use them (through a tool such as Paros from ParosProxy.org) to perform $multi million trades on someone else's behalf. In the current climate - is this an issue? ;-) Sorry - as much as I'd love to name & shame these clowns, I won't.
How likely is this scenario? Can you capture Session ID's on a Switched Network? Certainly within a local VLAN using the hacking tool CAIN and taking advantage of ARP poisioning you can.
In a poorly written Server Side application, you can also predict Session ID's. Check the tool WebScarab (which is in any Pen Testers armoury). This will detect the randomness of ID's. At the same bank with another critical application you could generate your own Session ID's to access & trade applications. Their focus was on low latency (which is business critical) rather than Security.
An intro can be found at Owasp.org
Noelie Dunne
This link says session ids are unique - http://support.microsoft.com/kb/899918
Session ids are unique in that only 1 session will ever have that id at any given time.
If this wasn't the case I think there would be a lot of people shouting very loudly about it.
The Session ID is almost certainly unique to the user. Failure modes where several users share a SessionID are very, very rare today. However, a user can do several things to create the effect you are seeing.
For instance the user can have several tabs open in her browser. Those tabs will all share the same Session ID. So if she switches back and forth between those browser tabs it might give the effect you are seeing.
Another issue is that users frequently doubleclick buttons and links. This means a processing request may get issued twice with the same Session ID. I would check for this possibility first.
I would like some advice from anyone experienced with implementing something like "pessimistic locking" in an asp.net application. This is the behavior I'm looking for:
User A opens order #313
User B attempts to open order #313 but is told that User A has had the order opened exclusively for X minutes.
Since I haven't implemented this functionality before, I have a few design questions:
What data should i attach to the order record? I'm considering:
LockOwnedBy
LockAcquiredTime
LockRefreshedTime
I would consider a record unlocked if the LockRefreshedTime < (Now - 10 min).
How do I guarantee that locks aren't held for longer than necessary but don't expire unexpectedly either?
I'm pretty comfortable with jQuery so approaches which make use of client script are welcome. This would be an internal web application so I can be rather liberal with my use of bandwidth/cycles. I'm also wondering if "pessimistic locking" is an appropriate term for this concept.
It sounds like you are most of the way there. I don't think you really need LockRefreshedTime though, it doesn't really add anything. You may just as well use the LockAcquiredTime to decide when a lock has become stale.
The other thing you will want to do is make sure you make use of transactions. You need to wrap the checking and setting of the lock within a database transaction, so that you don't end up with two users who think they have a valid lock.
If you have tasks that require gaining locks on more than one resource (i.e. more than one record of a given type or more than one type of record) then you need to apply the locks in the same order wherever you do the locking. Otherwise you can have a dead lock, where one bit of code has record A locked and is wanting to lock record B and another bit of code has B locked and is waiting for record A.
As to how you ensure locks aren't released unexpectedly. Make sure that if you have any long running process that could run longer than your lock timeout, that it refreshes its lock during its run.
The term "explicit locking" is also used to describe this time of locking.
I have done this manually.
Store the primary-key of the record to a lock table, and mark record
mode attribute to edit.
When another user tries to select this record, indicate the user's
ready only record.
Have a set-up maximum time for locking the records.
Refresh page data for locked records. While an user is allowed to
make changes, all other users are only allowed to check.
Lock table should have design similar to this:
User_ID, //who locked
Lock_start_Time,
Locked_Row_ID(Entity_ID), //this is primary key of the table of locked row.
Table_Name(Entity_Name) //table name of the locked row.
Remaining logic is something you have to figure out.
This is just an idea which I implemented 4 years ago on special request of a client. After that client no one has asked me again to do anything similar, so I haven't achieved any other method.