I want a client to be able to request (via HTTP) whichever document is at the head of a server's output queue, with the understanding that if retrieval is successful, the document will then automatically be deleted from the queue. There is never more than one client per server, but the client could be multithreaded. There is no random access to the queue; only the head item can be retrieved (and deleted). I have not found this scenario discussed either here or elsewhere on the web. Here are the various approaches I can think of:
(1) The client could send a GET request. But GET is not supposed to have side effects, so this doesn't seem like a good idea.
(2) The client could send two requests, a GET to retrieve the document at the head of the queue and a DELETE (with an empty or ignorable URL) to delete the document at the head of the queue. But this requires two calls, which could cause various problems, especially if more than one thread/process in the client is trying to retrieve files.
(3) The client could send a POST request with an empty body; if there is a document at the head of the queue, the server will return a response whose body contains the document, and will also delete the document from the queue. This is somewhat counterintuitive in that it doesn't match the mental model of posting data and receiving a simple return code, but otherwise I like it. I'm not worried about the response getting lost in transit and the document going missing; I expect the connection to be safe enough to prevent this.
It would be nice if there were another HTTP method to handle this situation, but since there isn't, I think (3) is the best approach. Agree? Disagree?
UPDATE: Added (4) after reading Dan675's post below.
(4) The client could send a DELETE request, to which the server could send a response with the document in the body (and delete the document from the queue, of course). Again, this is slightly counterintuitive (you don't usually say "delete the item on top of the stack for me, please" when you want to retrieve it), but would work.
It should be done in two calls the first to GET it then one to DELETE it.
If the delete succeeds then the clients request is valid otherwise just treat it as if the whole request failed and try to get what's on the top of the queue again. This will cause some additional overhead due to failed requests but I would not recommend doing either of the other options.
I guess another way of doing this would be to first maybe do a PUT to mark the top most item as 'reserved' in some way then do a GET and DELETE. In doing it this way it may be possible to traverse this server side queue and look for the top-most item that is not 'reserved'.
Related
I've got POST method which responds with a file. And right after file download I also want to show the message on the UI, that was generated during the file creation in the code (were there any errors, are all items included in the file etc.).
My question is how to implement such behaviour in most appropriate way?
now I've got 2 ideas:
sending another POST method right after download, which will return
JSON with message contents. Which seems to be quite complicated
since I should be storing on the server objects with messages for
each created file
sending a cookie with the same response, that will contain message
contents
I think both of your solutions should work in theory. I am assuming here that cookie headers sent back with an octet stream response will be honoured and instantly accessable via javascript. I've not tried this out myself.
I have two more ideas which might give you some inspiration:
Simple
In the POST handler before you output the file to the user, you could set a session variable containing the message you want to display. Then after the file is downloaded, make a 2nd request to GET that message from the session. Since it would get overwritten each time, it might get a bit unreliable if the user made lots of POSTs at once.
More complicated
For a totally asynchronous solution, in the past I have used services in this way:
POST /process_this -> "{'status':'ok', 'id':123}"
GET /status?id=123 -> "{'status':'processing', 'progress':'45%'}"
GET /status?id=123 -> "{'status':'processing', 'progress':'85%'}"
GET /status?id=123 -> "{'status':'ready', 'meta':{'data':'message'}}"
GET /download?id=123 -> Binary file stream
It gives you the ability to always respond instantly and do some cool progress bar on the frontend. Also you could have a queue system if the processing is intensive. But on the downside, you need to write the processor as a seperate module and have it talking to a database or something that saves state.
Other
Some other thoughts I had included:
Storing the meta data within the file itself. (What is the response file format?)
Using multipart responses (not sure how compatible browsers are on this yet)
Using a combination of your two ideas if cookies are not big enough to hold all of the message.
I have an object:
Account
{
Id,
Name,
CurrentBalance
}
Id is an immutable key, Name is a mutable string, and CurrentBalance is calculated from all of the transactions associated with the account.
I am stuck on the fact that GET \Accounts\{Id} will not be idempotent because changes to a transaction will cause a change in CurrentBalance. Should I remove this field from the object and make a request like
POST \Accounts\{Id}\CurrentBalance
But now I have to make multiple calls to the server to get the CurrentBalance of all objects:
GET \Accounts
POST \Accounts\{Id1}\CurrentBalance
POST \Accounts\{Id2}\CurrentBalance
POST \Accounts\{Id3}\CurrentBalance
....
I guess I am just looking to see if there is already a standard way to handle this that I am missing?
UPDATE
Part 2 if the original object is ok via GET. My only way to update the Account.Name is via a PATCH as I cannot allow an update to CurrentBalance, correct?
NOTE
I realize I could put this on the client to have to get all transactions and calculate it, but I would prefer to do this on the server for multiple reasons
Idempotency does not mean that you must always get the same response back.
Consider the resource /TodaysWeather. It would be pretty useless if it always returned the same value.
Idempotency simply states that if a client makes the same request multiple times instead of just once, the impact on the system (from the client's perspective) will be the same.
I just re-read the HTTP specs and realized that if I want to be truly RESTful I have to make multiple calls because GET has to be safe.
In particular, the convention has been established that the GET and
HEAD methods SHOULD NOT have the significance of taking an action
other than retrieval.
I am not deleting this question because I think it could help others in the future, but if the majority disagree I will delete it
If it's important that you're able to PUT some data, then immediately retrieve the same data via GET, then you could simply treat it as a different resource entirely, e.g.:
# Change an account name
PUT \Accounts\{id}
# Get accounts/names/balances
GET \AccountDetails
# Get balance of an account
GET \AccountDetails\{id}\CurrentBalance
However, there's really no good reason to go through the trouble of doing that. Your PUT is idempotent as long as making the same request multiple times doesn't change the state of the system. Not changing the system's state if some spurious value is submitted is the correct behavior. In fact, if someone does try a PUT including CurrentBalance, you might want to return a 400 (Bad Request) status explaining that CurrentBalance can't be updated.
While the HTTP 1.1 spec seems to allow message bodies on DELETE requests, it seems to indicate that servers should ignore it since there are no defined semantics for it.
4.3 Message Body
A server SHOULD read and forward a message-body on any request; if the
request method does not include defined semantics for an entity-body,
then the message-body SHOULD be ignored when handling the request.
I've already reviewed several related discussions on this topic on SO and beyond, such as:
Is an entity body allowed for an HTTP DELETE request?
Payloads of HTTP Request Methods
HTTP GET with request body
Most discussions seem to concur that providing a message body on a DELETE may be allowed, but is generally not recommended.
Further, I've noticed a trend in various HTTP client libraries where more and more enhancements seem to be getting logged for these libraries to support request bodies on DELETE. Most libraries seem to oblige, although occasionally with a little bit of initial resistance.
My use case calls for the addition of some required metadata on a DELETE (e.g. the "reason" for deletion, along with some other metadata required for deletion). I've considered the following options, none of which seem completely appropriate and inline with HTTP specs and/or REST best practices:
Message Body - The spec indicates that message bodies on DELETE have no semantic value; not fully supported by HTTP clients; not standard practice
Custom HTTP Headers - Requiring custom headers is generally against standard practices; using them is inconsistent with the rest of my API, none of which require custom headers; further, no good HTTP response available to indicate bad custom header values (probably a separate question altogether)
Standard HTTP Headers - No standard headers are appropriate
Query Parameters - Adding query params actually changes the Request-URI being deleted; against standard practices
POST Method - (e.g. POST /resourceToDelete { deletemetadata }) POST is not a semantic option for deleting; POST actually represents the opposite action desired (i.e. POST creates resource subordinates; but I need to delete the resource)
Multiple Methods - Splitting the DELETE request into two operations (e.g. PUT delete metadata, then DELETE) splits an atomic operation into two, potentially leaving an inconsistent state. The delete reason (and other related metadata) are not part of the resource representation itself.
My first preference would probably be to use the message body, second to custom HTTP headers; however, as indicated, there are some downsides to these approaches.
Are there any recommendations or best practices inline with REST/HTTP standards for including such required metadata on DELETE requests? Are there any other alternatives that I haven't considered?
Despite some recommendations not to use the message body for DELETE requests, this approach may be appropriate in certain use cases. This is the approach we ended up using after evaluating the other options mentioned in the question/answers, and after collaborating with consumers of the service.
While the use of the message body is not ideal, none of the other options were perfectly fitting either. The request body DELETE allowed us to easily and clearly add semantics around additional data/metadata that was needed to accompany the DELETE operation.
I'd still be open to other thoughts and discussions, but wanted to close the loop on this question. I appreciate everyone's thoughts and discussions on this topic!
Given the situation you have, I would take one of the following approaches:
Send a PUT or PATCH: I am deducing that the delete operation is virtual, by the nature of needing a delete reason. Therefore, I believe updating the record via a PUT/PATCH operation is a valid approach, even though it is not a DELETE operation per se.
Use the query parameters: The resource uri is not being changed. I actually think this is also a valid approach. The question you linked was talking about not allowing the delete if the query parameter was missing. In your case, I would just have a default reason if the reason is not specified in the query string. The resource will still be resource/:id. You can make it discoverable with Link headers on the resource for each reason (with a rel tag on each to identify the reason).
Use a separate endpoint per reason: Using a url like resource/:id/canceled. This does actually change the Request-URI and is definitely not RESTful. Again, link headers can make this discoverable.
Remember that REST is not law or dogma. Think of it more as guidance. So, when it makes sense to not follow the guidance for your problem domain, don't. Just make sure your API consumers are informed of the variance.
What you seem to want is one of two things, neither of which are a pure DELETE:
You have two operations, a PUT of the delete reason followed by a DELETE of the resource. Once deleted, the contents of the resource are no longer accessible to anyone. The 'reason' cannot contain a hyperlink to the deleted resource. Or,
You are trying to alter a resource from state=active to state=deleted by using the DELETE method. Resources with state=deleted are ignored by your main API but might still be readable to an admin or someone with database access. This is permitted - DELETE doesn't have to erase the backing data for a resource, only to remove the resource exposed at that URI.
Any operation which requires a message body on a DELETE request can be broken down into at it's most general, a POST to do all the necessary tasks with the message body, and a DELETE. I see no reason to break the semantics of HTTP.
I suggest you include the required metadata as part of the URI hierarchy itself. An example (Naive):
If you need to delete entries based on a date range, instead of passing the start date and end date in body or as query parameters, structure the URI such a way that you pass the required information as part of the URI.
e.g.
DELETE /entries/range/01012012/31122012 -- Delete all entries between 01 January 2012 to 31st December 2012
Hope this helps.
I would say that query parameters are part of the resource definition, thus you can use them to define the scope of your operation, then "apply" the operation.
My conclusion is that Query Parameters as you defined it is the best approach.
Let's say I have a resource called "Session". The client would call PUT to create and begin a new session. When the client is finished with the session, it should no longer be accessible, but should persist for historical/accountability reasons.
To end the session, would it be more appropriate to issue a DELETE request, which would seem semantically closer to the desired effect, or POST, seeing as the resource isn't actually removed permanently?
The question here is: Is the request idempotent? If you execute the same request twice, does it have a side effect? Like when you order an article, executing the order request twice would get you the article twice.
In that case, POST is the method you want. If not, then you want either PUT or DELETE.
As you don't seem to be deleting the session, only altering its state, PUT would be a better method, because it means that the resource is altered, and not deleted, which is the case in your case.
Edit:
If the resource appears to be deleted from the client, DELETE seems more appropriate. How things are implemented in the back doesn't matter for the client.
POST request will be better here as you don't actually delete the session. POST requests are often used to change state of an object. It is your case I think.
I have REST services which should receive really long queries via GET. Say for example I want to query a service with many geographical coordinates to find out something about all this coordinates.
1) My first thought was to use long URIs and increase the max URI length of the servlet container.
It would look like this:
GET http://some.test/myresource?query={really big JSON object}
But it seems that URIs longer than 2 KB are not reliable due to old proxy servers (is that right?).
2) My workaround is to create a temporary resource via POST first and use the URI of this resource as parameter in the actual GET request. That would look like this:
POST http://some.test/temp
Request Body: {really big JSON object}
201 Created Location: http://some.test/temp/12309871
GET http://some.test/myresource?query=http://some.test/temp/12309871
3) Use body of GET request. I've read the answers to the question whether it is a good idea to use the body of a GET request for the query, and the consensus is: no. Even Roy Fielding says that this is a bad idea.
4) Another approach could be to interpret POST as "create query result resource" and delete this resource after the request. But I consider that to be not RESTful and to be a bad idea.
Is there a better way to handle big queries with GET requests?
Use PUT.
Why? For the following reasons:
Just because the verb PUT 'may update' the resource, doesn't mean it will or must alter underlying state of the resource.
No new resource identifier (url) should be created by the API side of a PUT. Yes, technically a PUT with a client specified identifier is possible, but in this case you're hitting an existing resource.
PUT is like GET in the fact that it should be idempotent, meaning the results of the request will always be the same regardless of how often you call it and it has no side effects.
PUT means you're putting resource data to an existing resource. In terms of a article or post in the document / blog post worlds, it would be like uploading a new revision of some document to an existing resource URL. If you upload the same revision to the same URL, nothing should change in the resource you get back.
In your case, the geo data is some new resource data you're uploading and the result you get back should be the same every time you make the same request.
A more purist method to use the GET verb for the request might be:
Create an endpoint for a query resource type
POST the JSON set of query details to a query resource endpoint and get an identifier for the query resource (say it returns a query id of 123)
Submit to the get request a query identifier http://some.test/myresource?query_id=123
Delete the query resource 123
I see the pure method much more overhead than using PUT with query resource data in the body.
I thought that the whole point in REST was to work on "documents" (or something alike). The URI part of a request is there to identify uniquely the resource to work on. The body part in contrast is there for the "contents" part of the document.
Hence, use the "body" part of the request.
Also note that the semantics of a "GET" request isn't supposed to be used for "PUTTING" or "POSTING" documents (comment in relation to your "query" example above which seems to "create" an object).
In any case, as you have pointed out, the URI part is limited (for good reason I am sure).
If you are concerned with caching, then the use of ETag/Last-Modified fields (in conjunction with "conditional GET" helps for this purpose.
Here is a slight variation on your second option. Create yourself a processor resource called QueryMaker. POST your parameters to it and let it redirect you to a temporary query resource that will return your results.
POST /QueryMaker
Body: Big Json representation of parameters
303: See Other
Location: http://example.org/TemporaryQueries/123213
If you are using a GET request to send large objects, you are not using REST correctly.
GET should be used for retrieving
resources (via some sort of unique
identifier)
POST should be used for
creating resources (with the contents
in the body)
PUT should be used for
updating a resource (with the
contents in the body)
DELETE should be used for deleting a resource
If you follow these guidelines you will never have to have overly long URIs.
Some best practice REST guidelines are here: http://www.xml.com/pub/a/2004/08/11/rest.html
The biggest limitation of URL lengths on the open Web is actually IE, which constraints them to 2083 characters.
Some proxies (e.g., all but the most recent versions of Squid) will limit them to about 4k, although this is moving towards 8k slowly.
Your #2 workaround is a good approach, depending on your use case.
Sending bodies on GETs may be allowed by some implementations, and disallowed by others, so it's a bad idea for interoperability as well as theoretical reasons. Most importantly, how will a cache know what to use as a key?
Can't you just send the big JSON data with the GET request body, instead of creating the temp resource?
Although it's not 100% kosher, i've found it works nicely with firefox and IE and IMO, the querystring is inelegant and usually exposes implementation details that don't belong in the URI. Just make sure to add a cache buster querystring parameter if you need up-to-date data because the server will ignore the data when determining whether it can return a cached response.
See here for a discussion of pros and cons of stuffing data in the GET request body.