I have an Azure Web App with autoscaling configured with a minimum of 2 instances. Database is SQL Azure.
User will make changes to the data e.g. edit a product's price. The change will make it to the database and I can see it in SSMS. However, after user refreshes the page, the data may or may not get updated.
My current theory is something to do with having multiple instances of the web app, because if I turn off autoscale and just have 1 instance, the issue is gone.
I haven't configured any sort of caching in Azure at all.
It sounds like what is happening is the data may or may not appear because it is stored in memory on the worker server (at least temporarily). When you have multiple worker servers, a different one may serve the request, in which case that server would not have the value in memory. The solution is to make sure that your application's code is re-fetching the value from the database in every case.
Azure Web Apps has some built in protection against this, called the ARR affinity cookie. Essentially each request has a cookie which keeps sessions "sticky". i.e. if a worker server is serving requests to a certain user, that user should receive subsequent requests from that server as well. This is the default behavior, but you may have disabled it. See: https://azure.microsoft.com/en-us/blog/disabling-arrs-instance-affinity-in-windows-azure-web-sites/
Related
We have an ASP.NET WebForms website running in an Azure WebApp with automatic "Scale Out" enabled. I can see there are currently two instances running. We have a test page with the following code:
Request.ServerVariables["LOCAL_ADDR"]
If we constantly refresh this page, the IP randomly switches between two different values (presumably for the two different website instances that are running). This is breaking functionality that relies on Sessions, as the Sessions are server-specific.
Is there a way to keep users on the same server instead of connecting them to a random server for each request? (I understand this wouldn't fully solve the problem, so my next question is...)
Is it not viable to use Session objects with the "Scale Out" feature enabled? If not, what are some other options? If we use cookies, I'm concerned about reaching data limits since we occasionally use Sessions to preserve large data sets for short periods. If we use something like a Redis cache, it adds to our operating costs. Is there a better way to do this?
In Azure App Service we need to enable ARR Affinity to keep the session active in one Server.
The Application requesting routing identifies the user and assigns a Affinity cookie. Client establishes the session with the current instance and keeps the instance active until the session expires.
ARR affinitywill not work when we scale out the app service instances, when we scale out, new instances of our app services server will be created and the ARR Affinity will fail if the request goes to new server.
Thanks #ajkuma-MSFT
If our application is stateful, scaling up would be the best option, If our application is stateless, scaling out gives the greater flexibility and a better scalability potential.
Instead of Scaling out we can Scale up App Services plan by increasing the size and SKU of our existing App Service plan to higher tier with more compute, features and then enable the ARR affinity, which helps the sessions remain active and persistent in one Server.
If we use something like a Redis cache, it adds to our operating costs.
Thanks #Praveen Kumar Sreeram
when you configure Load Balancer utilizing the auto-scaling capability, "Sessions" wouldn't work as planned in the Azure App Service.
Another option is to use Redis Cache.
Currently I am using the Standard Tier S1.
With auto Scaling Rule, we can scale up and down when it is not required.
Scale up to Standard S3.
One affinity cookie will be associated to the app server for each request, even if the app receives repeated requests; this will maintain the session's persistence and activity.
And as new instance of app server won’t be created, the application session will remain active due to ARR affinity.
Reference taken from Microsoft Q&A
I have a single .NET web app running in an ARR cluster (IIS) with multiple machines.
Each machine must keep a cache for user access permissions in memory. That is, when the app must determine whether the user has permission to access a resource, it queries a memory cache to avoid database access (there's a lot of queries per user request).
The problem is that, in certain situations, this cache must be invalidated. But, as there are multiple machines, when one decides the cache must be invalidated, it has to be propagated to the other machines.
What is the best practice to solve this problem? We have an ASP.NET MVC 3 app running on a IIS ARR cluster.
Message queues are the normal solution.
You can have a queue that is subscribed by all your nodes and when you need invalidate a cache you send a message to the queue - the other nodes see this message and invalidate their queues.
MSMQ is the Microsoft message queue, but there are many third party ones. You may want to take a look at nServiceBus as an alternative.
Invalidate all cache related to permissions, or only relevant parts of the cache (ex particular user, particular role etc?)
Anyway, I'm thinking along the lines of some pub/sub pattern, ex http://redis.io/topics/pubsub - a server that the apps subscribe to (sub), and the apps can request invalidation of cache by publishing a request to all subscribers (pub)
My organisation (a small non-profit) currently has an internal production .NET system with SQL Server database. The customers (all local to our area) submit requests manually that our office staff then input into the system.
We are now gearing up towards online public access, so that the customers will be able to see the status of their existing requests online, and in future also be able to create new requests online. A new asp.net application will be developed for the same.
We are trying to decide whether to host this application on-site on our servers(with direct access to the existing database) or use an external hosting service provider.
Hosting externally would mean keeping a copy of Requests database on the hosting provider's server. What would be the recommended way to then keep the requests data synced real-time between the hosted database and our existing production database?
Trying to sync back and forth between two in-use databases will be a constant headache. The question that I would have to ask you is if you have the means to host the application on-site, why wouldn't you go that route?
If you have a good reason not to host on site but you do have some web infrastructure available to you, you may want to consider creating a web service which provides access to your database via a set of well-defined methods. Or, on the flip side, you could make the database hosted remotely with your website your production database and use a webservice to access it from your office system.
In either case, providing access to a single database will be much easier than trying to keep two different ones constantly and flawlessly in sync.
If a webservice is not practical (or you have concerns about availability) you may want to consider a queuing system for synchronization. Any change to the db (local or hosted) is also added to a messaging queue. Each side monitors the queue for changes that need to be made and then apply the changes. This would account for one of the databases not being available at any given time.
That being said, I agree with #LeviBotelho, syncing two db's is a nightmare and should probably be avoided if you can. If you must, you can also look into SQL Server replication.
Ultimately the data is the same, customer submitted data. Currently it is being entered by them through you, ultimately it will be entered directly by them, I see no need in having two different databases with the same data. The replication errors alone when they will pop-up (and they will), will be a headache for your team for nothing.
I am looking at implementing caching in a .net Web App. Basically... I want to cache some data that is pulled in on every page, but never changes on the database.
Is my Cache Element unique to each:
Session?
App Pool?
Server?
If it is session, this could get out of hand if thousands of people are hitting my site and each cache is ~5k.
If App Pool, and I had several instances of one site running (say with a different DB backend, all on one server, though) then I'd need individual App Pools for each instance.
Any help would be appreciated... I think this data is probably out there I just don't have the right google combination to pull it up.
By default it is stored in memory on the server. This means that it will be shared among all users of the web site. It also means that if you are running your site in a web farm, you will have to use an out-of-process cache storage to ensure that all nodes of the farm share the same cache. Here's an article on MSDN which discusses this.
"One instance of this class is created per application domain, and it remains valid as long as the application domain remains active" - MSDN
What issues do I need to be aware of when I am deploying an ASP.NET application as a web farm?
All session state information would need to be replicated accross servers. The simplest way would be to use the MSSQL session state provider as noted.
Any disk access, such as dynamic files stored by users, would need to be on an area avialable to all servers. Such as by using some form of Network Attached storage. Script files, images and html etc would just be replicated on each server.
Attempting to store any information in the application object or to load information on application startup would need to be reviewed. The events would fire each time the user hit a new machine in the farm.
Machine keys across each server is a very big one as other people have suggested. You may also have problems if you are using ssl against an ip address rather than a domain.
You'll have to consider what load balancing strategy your going to go through as this could change your approach.
Sessions is a big one, make sure you use SQL Server for managing sessions and that all servers point to the same SQL Server instance.
One of the big ones I've run across is issues with different machineKeys spread across the different servers. ASP.NET uses the machineKey for various encryption operations such as ViewState and FormsAuthentication tickets. If you have different machineKeys you could end up with servers not understanding post backs from other servers. Take a look here if you want more information: http://msdn.microsoft.com/en-us/library/ms998288.aspx
Don't use sessions, but use profiles instead. You can configure a SQL cluster to serve them. Sessions will query your session database way too often, while profiles just load themselfs, and that's it.
Use a distributed caching store like memached for caching data, and ASP.Net cache for stuff you'll need alot
Use a SAN or an EMC to serve your static content
Use S3 or something similar to have a fallback on 3.
Have some decent loadbalancer, so you can easily update per server, without ever needing to shut down the site
HOW TO: Set Up Multi-Server ASP.NET Web Applications and Web Services
Log aggregation is easily overlooked - before processing HTTP logs, you might need to combine them to create a single log that includes requests sent to across servers.