I have a client application which is adding items to a cart. An "add" operation is firing an update request via a HTTP REST call to a remote endpoint. Please note: this request contains the complete cart as a whole, not solely the item being added. This request is then load-balanced between two servers using a round-robin alogrithm.
The problem I'm trying to tackle is that the client does not wait for the "add" request to return before to launch another "add" request, if the user does so. This is good from an end-user perspective because the user doesn't have to wait. But this is a nightmare from a server perspective : you can't be sure in which order the requests will be processed because of the load-balancer.
Here is an example:
The user adds the item #1 to the cart. The request A is sent.
The user adds the item #2 to the cart. The request B is sent. Please note that the request B is sent before the request A has received a response.
The request A is load-balanced on server 1, and the request B is load-balanced on server 2
For some reasons, the server 1 is slower than server 2, so that the request B is processed first => the cart has item #1 and #2
The server 1 processes the request A => the cart has item #1 only (reminder: each update request contains the wole cart)
I'm not so sure about how to handle this. So far, the possible solutions I can think of are:
Send a timestamp with the request and keep the timestamp in database. Before to update the cart, check that the timestamp of the request is higher. Otherwise drop the request. But this rely heavily on client-side behaviour.
Set a version number on the cart itself, and increment it at each update. This would force the client to wait for the response before to send another update. Not very satisfactory from an end-user perspective, because he has to wait.
Set a "session affinity" on the load-balancer so that the requests from a particular client are piped to the same server each time. The problem is that it affects the balance on the server load.
I'm pretty sure I'm not the first one to face such issue, but I failed at finding a similar case, quite surprisingly. I must probably have asked the wrong question or keywords! Anyway, I'd be very interested to share your thoughts and experience on this problem.
The easiest way would be to send the operation details (added #1, added #2…) to be able to reconstruct the cart in an incremental way on the servers. With this information, you don't rely on the requests being processed in a specific order at all.
If you can't modify the API, your third solution (handling a same session on a same server during its whole duration) would probably be the way to go, without more information on your expected load by customer/customer count.
Related
A play Controller with Actions for POST requests, may need to ignore HTTP Request Automatic Retries to prevent Controller code being ran multiple times.
What is the best way to do this in Play?
I would recommend the following:
add a unique ID to every post request as part of the query string.
Extend the DefaultHttpRequestHandler, as explained here
In your extension check if this is a request for which you want to
prevent retries and read the request's ID using getQueryString on the RequestHeader, see docs here.
Check if you have already seen the ID by querying a datastore such as Redis. Save the ID to Redis if this is the first time you have seen it.
Drop the POST request if you have already seen the ID, otherwise forward it to the router.
I have an application where requests to a controller will take a while to process. The controller starts a thread per request and eventually returns some data to a database. I need to limit how many requests can be processed. So let's say our limit is 100, if the controller is already processing 100 requests, the 101st request will return 503 status till at least one request is completed.
I could use an application wide static counter to keep count of current processes but is there a better way to do this ?
EDIT:
The reason why the controller takes a while to respond is because the controller calls another API, which is a large database spanning several TB of geostationary data. Even if I could optimize this in theory, its not something I have control over. To make matters worse, the third party API simply times out if I have more than 10 concurrent requests. I am already dropping incoming requests to a servicebus queue. I just need a good way on my api controller to keep a global count of how many requests are coming in and returning 503 whenever it exceeds a set number of requests.
The requests to the API controller should not be limited. An idea would be to take requests and store the list of processes that need completing (database, queue etc)
Then create something outside the web request that processes this work, this is where you can manage how many are processed at once using parallel processing/multi-threading etc. (using windows service /Worker Role / Hangfire etc)
Once processed, you could communicate back to the page via SignalR to then get the data required to display once processed, or show status.
The benefit of this is that you can always go back to the page or refresh and get some kind of status, without re-running the whole process.
As we know, a POST method is not idempotent. Should a user send multiple requests in a short time, the operation would be repeated.
For example:
POST request to create a submission is called by the user.
Similar submissions are allowed by the server, so if a user sends the same request twice, the second doesn't fail due to a unique constraint violation.
The user sends two requests by accident, creating two identical resources (save for the auto generated resource ID).
How do I avoid this, whilst ensuring my server can be scaled? If I only had one instance, I could add a temporary timeout list to the server which would prevent the user completing similar requests on the same server temporarily. However if the request got passed to a different server instance, this server would not have the user on the timeout list and would process the second request just fine.
I'm aware that I can implement something on the client side such as a temporary submission button disable so users can only send one request at a time, however is there a way to handle this on the server side?
Caveat: I don't have much production experience in Web development.
If the goal is to safeguard from "Submit" key being pressed twice by accident, then I'd add a generated GUID to the POST web page somehow (a form field?). The posted contents and GUID form a unique key. It prevents the same contents from being submitted twice from the same generated page.
When POSTing data - either using AJAX or from a mobile device or what have you - there is often a "retry" condition, so that should something like a timeout occue, the data is POSTed again.
Is this actually a good idea?
POST data is meant to be idempotent, so if you
make a POST to the server,
the server receives the request,
takes time to execute and
then sends the data back
if the timeout is hit sometime after 3. then the next retry will send data that was meant to be idempotent.
The question then is that should a retry (when calling from the client side) be set for POST data, or should the server be designed to always handle POST data appropriately (with tokens and so on), or am i missing something?
update as per the questions - this is for a mobile app. As it happens, during testing it was noticed that with too short a timeout, the app would retry. Meanwhile, the back-end server had in fact accepted and processed the initial request, and got v. upset when the new (otherwise identical) re-request came in.
nonce's are a (partial) solution to this. The server generates a nonce and gives it to the client. The client sends the POST including the nonce, the server checks if the nonce is valid and unused and if so, acts on the POST and invalidates the nonce, if not, it reports back that the nonce is used and discards the data. Also very usefull to avoid the 'double post' problem by users clicking a submit button twice.
However, it is moving the problem from the client to a different one on the server. If you invalidate the nonce before the action, the action might still fail / hang, if you invalidate it after, the nonce is still valid for requests during the processing. So, a possible scenario on the server becomes on receiving.
Lock nonce
Do action
On any processing error preventing action completion, rollback, release lock on nonce.
On no errors, invalidate / remove nonce.
Semaphores on the server side are most helpfull with this, most backend languages have libraries for these.
So, implementing all these:
It is safe to retry, if the action is already performed it won't be done again.
A reply that the nonce has already been used can be understood as a a confirmation that the original POST has been acted upon.
If you need the result of an action where the second requests shows that the first came through, a short-lived cache would be needed server-sided.
Up to you to set a sane limit on subsequent tries (what if the 2nd fails? or the 3rd?).
The issue with automatic retries is the server needs to know if the prior POST was successfully processed to avoid unintended consequences. For example, if each POST inserts a DB record, but the last POST timed out after the insert, the automatic re-POST will cause a duplicate DB record, which is probably not what you want.
In a robust setup you may be able to catch that there was a timeout and rollback any POSTed updates. That would safely allow an automatic re-POST. But many (most?) web systems aren't this elaborate, so you'll typically require human intervention to determine if the POST should happen again.
If you can't avoid the long wait until the server responds, a better practice would be to return an immediate response (200OK with the message "processing") and have the client, later on, send a new request that checks if the action was performed.
AJAX was not designed to be used in such a way ("heavy" actions).
By the way, the default HTTP timeout is 7200 secs so I don't think you'll reach it easily - but regardless, you should avoid having the user wait for long periods of time.
Providing more information about the process (like what exactly you're trying to do) would help in suggesting ways to avoid such obstacles.
If your requests are failing enough to instigate something like this, you have major problems that have nothing to do with implementing a retry condition. I have never seen a Web app that needed this type of functionality. Programming your app to automatically beat an overloaded sever with the F5 hammer is not the solution to anything.
If this ajax request is triggered from a button click, disable that button until it returns, successful or not. If it failed, let the user click it again.
I have an Ajax request to a web service that typically takes 30-60 seconds to complete. In some cases it could take as long as a few minutes. During this time the user can continue working on other tasks, which means they will probably be on a different page when the task finishes.
Is there a way to tell that the original request has been completed? The only thing that comes to mind is to:
wrap the web service with a web service of my own
use my web service to set a flag somewhere
check for that flag in subsequent page requests
Any better ways to do it? I am using jQuery and ASP.Net, if it matters.
You could add another method to your web service that allows you to check the status of a previous request. Then you can use ajax to poll the web service every 30 seconds or so. You can store the request id or whatever in Session so your ajax call knows what request ID to poll no matter what page you're on.
I would say you'd have to poll once in a while to see if request has ended and show some notifications, like this site does with badges for example.
At first make your request return immediately with something like "Started processing...". Then use a different request to poll for the result. It is not good neither for the server nor the client's browser to have long open HTTP sessions. Moreover the user should be informed and educated that he is starting a request that could take some time to complete.
To display the result you could have a"notification area" in all of your web pages. Alternatively you could have a dedicated page for this and instruct the user to navigate there. As others have suggested you could use polling to get the result.
You could use frames on your site, and perform all your long AJAX requests in an invisible frame. Frames add a certain level of pain to development, but might be the answer to your problems.
The only other way I could think of doing it is to actually load the other pages via an AJAX request, such that there are no real page reloads - this would mean that the AJAX requests aren't interrupted, but may cause issues with breaking browser functionality (back/forward, bookmarking, etc).
Since web development is stateless (you can't set a trigger/event on a server to update the client), the viable strategy is to setup up a status function that you can intermittently call using a javascript timer to check whether your code has finished executing. When it finishes, you can update your view.