I have a client-server web application - the client is HTML/JS and the server is ASP.Net. web application hosted by Azure Web Role
In this application the client can save a document on the server by calling a web service method on the server side.
After calling the saving method, the client might save the document again while the server processes his previous save request. In this case I want the newly initiated saving to be queued until the previous saving operation is completed.
If I had a single web role instance, it would be easy for me to implement this by thread synchronization, but since the client request might be handled by different web role instances, the synchronization is problematic.
My question is - how can I implement such synchronization mechanism, or is there a better way to get the same result, that I'm not aware of.
Thanks.
I would consider combination of storage or service bus queues to queue up the requests to process the documents AND using BLOB leases to mark the work as in progress.
Queuing would be important since the requests might be delayed in processing if there is a previous request for the same job that's on going.
BLOB Leasing is a way to put a centralized lock in storage. Once you start processing of a request, you can put a blob with a lease on it and release the lease once you're done. Requests for the same work would check first if the lease is available before kicking off. Otherwise, they could just wait. More info on leases here: http://blog.smarx.com/posts/leasing-windows-azure-blobs-using-the-storage-client-library
Have you looked into using the Windows Azure Cache - Co-Located on your roles?
This is a shared caching layer that can use excess memory on your roles (Or have it's own worker role if your web roles are already close to capacity) to create a key / value store which can be accessed by any role on the same deployment.
You could use the cache to store a value indicating a document is currently being processed and block it until the document has finished uploaded. As it is a shared caching layer the value will be persisted across your instances (Though the cache will not persist during an upgrade deployment).
Here's a good introductary article to using Caching in Azure with configuration examples and sample code.
Related
We have an ASP.NET WebForms website running in an Azure WebApp with automatic "Scale Out" enabled. I can see there are currently two instances running. We have a test page with the following code:
Request.ServerVariables["LOCAL_ADDR"]
If we constantly refresh this page, the IP randomly switches between two different values (presumably for the two different website instances that are running). This is breaking functionality that relies on Sessions, as the Sessions are server-specific.
Is there a way to keep users on the same server instead of connecting them to a random server for each request? (I understand this wouldn't fully solve the problem, so my next question is...)
Is it not viable to use Session objects with the "Scale Out" feature enabled? If not, what are some other options? If we use cookies, I'm concerned about reaching data limits since we occasionally use Sessions to preserve large data sets for short periods. If we use something like a Redis cache, it adds to our operating costs. Is there a better way to do this?
In Azure App Service we need to enable ARR Affinity to keep the session active in one Server.
The Application requesting routing identifies the user and assigns a Affinity cookie. Client establishes the session with the current instance and keeps the instance active until the session expires.
ARR affinitywill not work when we scale out the app service instances, when we scale out, new instances of our app services server will be created and the ARR Affinity will fail if the request goes to new server.
Thanks #ajkuma-MSFT
If our application is stateful, scaling up would be the best option, If our application is stateless, scaling out gives the greater flexibility and a better scalability potential.
Instead of Scaling out we can Scale up App Services plan by increasing the size and SKU of our existing App Service plan to higher tier with more compute, features and then enable the ARR affinity, which helps the sessions remain active and persistent in one Server.
If we use something like a Redis cache, it adds to our operating costs.
Thanks #Praveen Kumar Sreeram
when you configure Load Balancer utilizing the auto-scaling capability, "Sessions" wouldn't work as planned in the Azure App Service.
Another option is to use Redis Cache.
Currently I am using the Standard Tier S1.
With auto Scaling Rule, we can scale up and down when it is not required.
Scale up to Standard S3.
One affinity cookie will be associated to the app server for each request, even if the app receives repeated requests; this will maintain the session's persistence and activity.
And as new instance of app server won’t be created, the application session will remain active due to ARR affinity.
Reference taken from Microsoft Q&A
i have a bit theoretical question.
When creating web applications, there is difference to desktop applications with working and active connection to database. So im curious if there is some solution, which can provide more desktop-like access to database e.g. transactions on asynchronous requests from client (web browser)?
edit:
So i figured out, that there can be a transaction process of asynchronous request, from client. Is there solution, which can provide it in web apps?
e.g I have assynchronou ajax call, which consist of multiple operations, and i wana to process them as transaction. If everything is okay, operations will be all done. But if one of them fail, just rollback it. Like its in DB. Is it possible?
edit2: maybe im wrong and the issue is not about ajax, but about whole web applications, but i dont think there is a way how to make a asynchronnous request from web client.
Transaction need continuous connection to database. To make it work with web application you need a platform which allow the application to run continuously independent of client request. Java servlet is best fit, php is a no-no. So I asume you will use java servlet.
In java servlet, you can create a db transaction, create an id for it, and then store them in a static variable or in the provided application-wide object, context. Then, return the id to the client.
When the client want to send another request, make it send the id. The application then can locate the transaction variable based on the id. As long as the application doesn't restarted between the two requests, the transaction is still there and active.
Because web application don't know when the user leave the application, you must create a mechanism to check the transactions periodically, and then rollback it if the user leave them for a specified time period.
The database has no knowledge of who is connected outside of authentication.
I have an Azure Web App with autoscaling configured with a minimum of 2 instances. Database is SQL Azure.
User will make changes to the data e.g. edit a product's price. The change will make it to the database and I can see it in SSMS. However, after user refreshes the page, the data may or may not get updated.
My current theory is something to do with having multiple instances of the web app, because if I turn off autoscale and just have 1 instance, the issue is gone.
I haven't configured any sort of caching in Azure at all.
It sounds like what is happening is the data may or may not appear because it is stored in memory on the worker server (at least temporarily). When you have multiple worker servers, a different one may serve the request, in which case that server would not have the value in memory. The solution is to make sure that your application's code is re-fetching the value from the database in every case.
Azure Web Apps has some built in protection against this, called the ARR affinity cookie. Essentially each request has a cookie which keeps sessions "sticky". i.e. if a worker server is serving requests to a certain user, that user should receive subsequent requests from that server as well. This is the default behavior, but you may have disabled it. See: https://azure.microsoft.com/en-us/blog/disabling-arrs-instance-affinity-in-windows-azure-web-sites/
I have a single .NET web app running in an ARR cluster (IIS) with multiple machines.
Each machine must keep a cache for user access permissions in memory. That is, when the app must determine whether the user has permission to access a resource, it queries a memory cache to avoid database access (there's a lot of queries per user request).
The problem is that, in certain situations, this cache must be invalidated. But, as there are multiple machines, when one decides the cache must be invalidated, it has to be propagated to the other machines.
What is the best practice to solve this problem? We have an ASP.NET MVC 3 app running on a IIS ARR cluster.
Message queues are the normal solution.
You can have a queue that is subscribed by all your nodes and when you need invalidate a cache you send a message to the queue - the other nodes see this message and invalidate their queues.
MSMQ is the Microsoft message queue, but there are many third party ones. You may want to take a look at nServiceBus as an alternative.
Invalidate all cache related to permissions, or only relevant parts of the cache (ex particular user, particular role etc?)
Anyway, I'm thinking along the lines of some pub/sub pattern, ex http://redis.io/topics/pubsub - a server that the apps subscribe to (sub), and the apps can request invalidation of cache by publishing a request to all subscribers (pub)
I want to create a queue where clients can put in requests, then server worker threads can pull them out as they have resources available.
I'm exploring how I could do this with a Firebase repository, rather than an external queue service that would then have to inject data back into Firebase.
With security and validation tools in mind, here is a simple example of what I have in mind:
user pushes a request into a "queue" bucket
servers pull out the request and deletes it (how do I ensure only one server gets the request?)
server validates data and retrieves from a private bucket (or injects new data)
server pushes data and/or errors back to the user's bucket
A simplified example of where this might be useful would be authentication:
user puts authentication request into the public queue
his login/password goes into his private bucket (a place only he can read/write into)
a server picks up the authentication request, retrieves login/password, and validates against the private bucket only the server can access
the server pushes a token into user's private bucket
(certainly there are still some security loopholes in a public queue; I'm just exploring at this point)
Some other examples for usage:
read only status queue (user status is communicated via private bucket, server write's it to a public bucket which is read-only for the public)
message queue (messages are sent via user, server decides which discussion buckets they get dropped into)
So the questions are:
Is this a good design that will integrate well into the upcoming security plans? What are some alternative approaches being explored?
How do I get all the servers to listen to the queue, but only one to pick up each request?
Wow, great question. This is a usage pattern that we've discussed internally so we'd love to hear about your experience implementing it (support#firebase.com). Here are some thoughts on your questions:
Authentication
If your primary goal is actually authentication, just wait for our security features. :-) In particular, we're intending to have the ability to do auth backed by your own backend server, backed by a firebase user store, or backed by 3rd-party providers (Facebook, twitter, etc.).
Load-balanced Work Queue
Regardless of auth, there's still an interesting use case for using Firebase as the backbone for some sort of workload balancing system like you describe. For that, there are a couple approaches you could take:
As you describe, have a single work queue that all of your servers watch and remove items from. You can accomplish this using transaction() to remove the items. transaction() deals with conflicts so that only one server's transaction will succeed. If one server beats a second server to a work item, the second server can abort its transaction and try again on the next item in the queue. This approach is nice because it scales automatically as you add and remove servers, but there's an overhead for each transaction attempt since it has to make a round-trip to the firebase servers to make sure nobody else has grabbed the item from the queue already. But if the time it takes to process a work item is much greater than the time to do a round-trip to the Firebase servers, this overhead probably isn't a big deal. If you have lots of servers (i.e. more contention) and/or lots of small work items, the overhead may be a killer.
Push the load-balancing to the client by having them choose randomly among a number of work queues. (e.g. have /queue/0, /queue/1, /queue/2, /queue/3, and have the client randomly choose one). Then each server can monitor one work queue and own all of the processing. In general, this will have the least overhead, but it doesn't scale as seamlessly when you add/remove servers (you'll probably need to keep a separate list of work queues that servers update when they come online, and then have clients monitor the list so they know how many queues there are to choose from, etc.).
Personally, I'd lean toward option #2 if you want optimal performance. But #1 might be easier for prototyping and be fine at least initially.
In general, your design is definitely on the right track. If you experiment with implementation and run into problems or have suggestions for our API, let us know (support#firebase.com :-)!
This question is pretty old but in case someone makes it here anyway...
Since mid 2015 Firebase offers something called the Firebase Queue, a fault-tolerant multi-worker job pipeline built on Firebase.
Q: Is this a good design that will integrate well into the upcoming security plans?
A: Your design suggestion fits perfectly with Firebase Queue.
Q: How do I get all the servers to listen to the queue, but only one to pick up each request?
A: Well, that is pretty much what Firebase Queue does for you!
References:
Introducing Firebase Queue (blog entry)
Firebase Queue (official GitHub-repo)