I am working on a web app in ASP.NET/C# which needs to be scalable to handle the high user load (will probably run in a web farm). Since it will cater to a high number of users, around 1 Million plus, but number of online users would be around 30K-50K. I plan to use caching (provider based), and was wondering:
Is it a good idea to cache ALL users for performance? I plan to cache all other generic data, like settings etc, but how efficient would it be to cache ALL users in memory? If a user changes his/her profile, I will reload only that particular user in cache (having a collection of all the users). Any suggestions on this approach?
Do I need to worry about locking when using this above users cache? Only one editing the profile would be the user himself, that would be one atomic operation, though there will be multiple read oeprations in different threads. So while fetching users from cache, or updating a particualr user, should I use lock?
Thanks
Asif
Putting anything in Global Cache that is only useful to a single user is usually a bad idea and a performance killer. Optimize your database queries, and you will be in much better shape.
As a general rule of thumb you should only keep things in cache that are expensive to get from the database, and more than one user will want to see that information at once. Such as a list of the top 100 products or something. Small amounts of data that are relatively cheap to grab from the database, and that are only useful to a single person should stay where they are.
Caching increases complexity tremendously, and even more so in a web farm. Don't introduce needless complexity unless you absolutely have to. Wait until you have an actual performance problem before trying to solve it.
Caching users is probably a good idea. But it depends on how much data you are going to cache for each user, and the cost of retrieving that data from wherever it is stored.
For locking - can anyone else edit a user's profile (like an admininstrator)? Would that be a common occurrence? If so, you may want to do some locking. Otherwise, if only the user can edit their own stuff, I wouldn't bother.
Related
I have taken over development on an ASP.NET web forms app that is 100% designed to be used on mobile devices. Currently, the app makes very heavy use of session variables. The mode is default.
<sessionState timeout="600"></sessionState>
It won't surprise you that the app is buggy, and much of it's problems have to do with session data being lost, or, the entire session being lost.
Just some overview of the app in general:
No sensitive (security risk) data is stored in the session
The data in the session is small, but mission critical. No huge objects. Just a few INT's (Id's, etc), some strings, etc.
We have to assume that the user's bandwidth will be low, and that connectivity will be spotty
The data is unique per user
One of my suggestions to management about how to make this app better is to stop using Session variables and to use something else instead. I'm of the opinion that switching to cookies to store all of this data is a better way to handle this. I'm aware that some users may disable cookies (even on their mobile devices), but if that's the case, we can simply inform them that to use the site, they need to turn cookies back on.
I thought about using query strings, but simply from a programming stand point (building links, etc) query strings are a pain to work with.
I've come to this conculsion after reading this MSDN article about the various ways to persist State data, and each of their pro's and con's:
https://msdn.microsoft.com/en-us/library/z1hkazw7.aspx
My question here is: Are cookies the best alternative?
Cookies are limited in size and aren't really the best place if you've lots of variables. Also people don't like cookies, so may restrict them.
The ViewState is a great place to store data though. This is sent as a postback from your web forms and can be used to store reasonable amounts of data. For example if your application uses several pages to complete a process (e.g. ordering something), then you can persist data in the ViewState and this will be available when the page is posted back.
Used in the right way, the ViewState is very effective.
In a new project, I'm planning to use ActiveDirectoryMembershipProvider and SqlRoleProvider to provide authentication and authorization, respectively.
One thing that isn't clear to me is how maintenance is handled -- when users that have logged in and been assigned roles are removed from Active Directory, how do you remove orphaned records in the mapping table used by SqlRoleProvider? I believe this is the aspnet_UsersInRoles table.
One could query Active Directory periodically for disabled users, then iterating through that list calling Roles.RemoveUserFromRoles(UserId, Roles.GetRolesForUser(UserId)) where UserId is also in aspnet_UsersInRoles. Hugely slow, I would imagine, for a large organization.
Or, alternatively, for each distint UserId in UsersInRoles, query ActiveDirectory and ensure the userAccountControl attribute's bitmask doesn't indicate the account is disabled. Also very inefficient, for a large number of application users.
An even more ugly but much more efficient approach would be to store last login date and periodically purge role associations for users that haven't logged in for, say, six months. This might cause headaches.
I'd love to hear suggestions.
Yes, you have to manually do the cleanup. Do you need instantaneous update? If you can do a batch process that runs nightly, that would be efficient since it isn't running during core operational hours. Or, it might make sense to kick off a process in another thread to handle the deletion of the role as soon as you are aware of it. Removing roles per user access shares the hit across users and makes them think that the application is slow.
How many times are roles removed? If a lot, then consider a batch process, if once in a few years, then it probably isn't as much of an issue to work it into the application during some process.
As far as how too, you can use the API, but the aspnet_UsersInRoles and aspnet_roles tables could be easily wiped on their own accord too via SQL Script.
HTH.
My question is how to best handle temporary data for an session. The scenario is similar to a shopping cart or like a bet slip. While the user is navigating the site and adding items with unique ID's. I'm only interested in the data collected this way if the user wants to commit it.
I'm developing in ASP .Net 3.5 with jQuery,JSON and a MS SQL DB.
As I see it there are a few possible ways to do this.
Perform a full post back to the server. Store every selections, update page controls accordingly.
Send selections via a Ajax request back to the server and update displaying control.
Build all functionality in JavaScript and store all values in a session cookie. Nothing being sent to server until user choose to commit.
I really want to consider performance here but I don't want to end up with 1000's of lines of JavaScript code..
Any suggestions of the best implementation with pro's and con's?
Cheers,
Stefan
Storing things in a session cookie is not a good idea, because that will be sent back to the server with every request. If you could find a way to store the state on the client without using a cookie, then you might have a viable client-centric option, but i can't think of anything portable off the top of my head. There are things in HTML5 and Flash that can do it, but you don't want to go there - yet, in the case of the former, and at all, in the case of the latter.
I'd use AJAX to post back to the server (with graceful degradation to a full post for browsers that can't handle that), then store the information in volatile memory there - ie not in the database. Write it to the database only when you need to. This is very easy to do in Java (you can associate information with the session), so i assume ASP.net has some way to do it too.
All three possibilities look good to me. The question, however, is: how much traffic do you expect?
Each of the options you presented suits better to a given scenario. Let's say you will have A LOT (thousand of thousands) users and not a lot of hardware available then you should probably try to minimize the number of requests to your app and store data in the client as much as possible before sending it to the server.
If it is smaller application then using Session or some other central database storage would be fine.
It all depends on your requirements.
There are 6 techniques to manage states in ASP.NET 3.5 (as far as I know).
(1) View State
(2) Cross Page Posting
(3) Query String
(4) Session State
(5) Application State
(6) Cookies
Can anyone give me some appropriate examples of situations where I should use these techniques?
For example:
(*) Session State: Personalization, Buy Cart, etc.
(*) Cookies: Saving User Credentials, etc.
There's a lot of factors that can influence this, so I won't comment on all of them. But here are a few pointers:
ViewState - This is useful when you'll be posting back to the same page frequently (something you're practically forced into doing by ASP.Net Webforms). How useful it is exactly changes depending on what kind of app you're building. For public internet sites, it should be used very sparingly. You may even want to turn it off by default. For local intranet sites, it's a great tool — especially for the fewer, heavier, webforms pages.
Query String - Use this to store state that you need to allow the user to bookmark a page or process and come back to much later. Even then, you might want to keep it down to some kind of hash that you can use as a key in a database lookup to avoid a really huge url (though hashes have their own problems). Also, a lot of users like to fiddle with your query string directly, so it can be dangerous to put too much here. It's easy to accidentally expose data to users who aren't supposed to see it this way.
Application State - Remember that this is shared by all users, so use appropriately. Things like view counts can go here.
Cookies - Don't use cookies to store user credentials. They're just plain unencrypted text files. Use cookies to store a key into the session (even here you can and should now use cookie-less sessions) and simple personalization settings that will be specific to that user and browser. For example, my monitor size at work is different from home, and so putting display size/layout settings into a cookie is nice because the settings stick for each computer, but it isn't going to compromise my security any if someone else reads that information.
Now I want to highlight this concept from the "Query String" section:
you might want to keep it down to some kind of hash that you can use as a key in a database lookup
Again, hashes have their own problems, but I want to point out that several items on my list talk (including Query String) about uploading data from the client web browser to the web server: ViewState, Query String, Cookie, and Cross-Page Post. You want to minimize the data that you move from client to server. This concept applies to all of these, and for several reasons:
Pulling data from the client is slow for public internet sites. Even broadband connections typically cripple the bandwidth available for upload. 512Kpbs (still a typical broadband upload rate in many areas) is nothing when compared to the Gigabit Ethernet (or faster) connection that likely sits between your database and your web server. As much as you might think of a database query as slow (and it is), it's still likely a much better way to go than waiting for the same data to arrive from the client.
Keeping the data on the server is cheaper, because you don't pay for the bandwidth required to push it to or from the client, and bandwidth often costs as much or more than your server hardware.
It's more secure, because if done right even when a client's computer or connection is compromised all the hacker has access to initially is a hash key that likely expires by the time he can decrypt it. Of course, if done wrong he can use that key directly immediately, so you still need to be careful.
So for most things, what I recommend is to start out by keeping a database key in the Session and then have code to easily pull what you need from a database based on that key. As you experience bottlenecks, profile to find out where they are and start caching those pages or controls, or keep that data/query result in the session directly.
State management option
View state:
Use when you need to store small amounts of information for a page that will post back to itself. Using the ViewState property provides functionality with basic security.
Control state:
Use when you need to store small amounts of state information for a control between round trips to the server.
Hidden fields:
Use when you need to store small amounts of information for a page that will post back to itself or to another page, and when security is not an issue.
You can use a hidden field only on pages that are submitted to the server.
Cookies:
Use when you need to store small amounts of information on the client and security is not an issue.
Query string:
Use when you are transferring small amounts of information from one page to another and security is not an issue.
You can use query strings only if you are requesting the same page, or another page via a link.
Server Side Management Options
Application state
Use when you are storing infrequently changed, global information that is used by many users, and security is not an issue. Do not store large quantities of information in application state.
Session state
Use when you are storing short-lived information that is specific to an individual session and security is an issue. Do not store large quantities of information in session state. Be aware that a session-state object will be created and maintained for the lifetime of every session in your application. In applications hosting many users, this can occupy significant server resources and affect scalability.
Profile properties
Use when you are storing user-specific information that needs to be persisted after the user session is expired and needs to be retrieved again on subsequent visits to your application.
Database support
Use when you are storing large amounts of information, managing transactions, or the information must survive application and session restarts. Data mining is a concern, and security is an issue.
Not sure if you mean the Cache object by Application State.
The Cache object is a great way to manage application wide state, e.g. to record source and count access to your website (to prevent DDOS attacks for example).
(3) Query String
(4) Session State
(5) Application State
(6) Cookies
1. Viewstate
Disclaimer: Use as little as possible. Good point is to always have each state reachable by an url, if possible.
F.e. Paging should use the URL (so /url/?p=2 instead of storing the page in Viewstate)
Use to persist control state between page-cycles.
F.e. Store the selected item in a checkbox, so you can determine whether it has changed.
2. Cross Page Posting
Don't. See the disclaimer for viewstate. Use the URL for this, or store the data in a session / cookie / profile if loads of properties need to be kept around.
Major downside of CPP is that the user cannot use the 'Back' and 'Forward' buttons in it's webbrowser. When a user clicks the back button it wants to undo everything on that page and retry the last one. When using CPP to click them through a wizard; this behavior is not possible without a lot of 'Are you sure you want to resend blablablabl'.
3. Query String
Use alot. Every visible state that a page could reach should be accessible by URL. People with screenreaders will thank you for this. And by using the query string there is no need to use javascript-only solutions.
/url/?page=2 // when doing paging, don't use postback for this
/url/?tab=advanced-search // when having tabs on top of your page
etc.
4. Session state
Use this for short-living objects, that only make sense this time the visitor visits your site. For example:
Which step of a certain wizard was reached
Pages a user had visited before
Small objects you want to put in cache, but that are user-bound
Don't use sessions but profiles for things like:
Preferences
Selected language
Because those things also make sense the next time the user visits your site.
5. Application state
Never. Use ASP.NET cache, or memcached, or any caching framework for this.
6. Cookies
Session ID, Profile ID for authenticated users; user preferences for anonymous users (everything listed in the second list under 4.).
I'm really asking this by proxy, another team at work has had a change request from our customer.
The problem is that our customer doesn't want their employees to login with one user more than one at the same time. That they are getting locked out and sharing logins.
Since this is on a web farm, what would be the best way to tackle this issue?
Wouldn't caching to the database cause performance issues?
You could look at using a distributed cache system like memcached
It would solve this problem pretty well (it's MUCH faster than a database), and is also excellent for caching pretty much anything else too
It's just a cost of doing business.
Yes, caching to a database is slower than caching on your webserver. But you've got to store that state information in a centralized location, otherwise one webserver isn't going to know what users are logged into another.
Assumption: You're trying to prevent multiple concurrent log-ins by a single user.
A database operation at login and logout won't cause a performance problem.
If you are using a caching proxy, that will cause a problem:
a user will log out, but won't be able to log back in until the logout reaches the cache
Your biggest potential problem might be:
if the app/box crashes without a chance for the user to log out, the user's state in the database will remain "logged in".
It depends on how the authentication is done. If you store the last successful login datetime (whatever the backend), so maybe you can change the schema to store a flag "logged_in" and that won't involve an extra performance cost. (ok, it's not clean at all)