My colleagues and I are implementing a number of RESTful HTTP services, and we're trying to make sure we are a) following the spec, and b) doing the "right" thing where the spec is short of detail.
Here is a particular situation that we have come to and are looking for opinions from the community on:
Suppose you have a resource /People/Bob, and your client is going to update it with a PUT. The server can produce representations for /People/Bob in application/json and text/html. The server can interpret representations for /People/Bob in application/json.
Given this request:
PUT /People/Bob
Content-Type: application/json
Accept: application/xml
{ name: "Still Bob" }
The server can't produce an XML representation, but it can process the incoming JSON. So we know the correct answer is for the server to return status 406.
The question is: should the server have performed the update to /People/Bob?
+1 for Philosophy of REST.
Without detailed knowledge of the HTTP spec, I would simply choose one of the options and document the quandary and the choice.
My preference would be that the server cannot respond as requested, then it should not process any of the request at all.
But that may not work in some scenarios, so you might have to do the opposite.
The question is: should the server have performed the update to /People/Bob?
From the HTTP spec, a 406 means:
The resource identified by the request is only capable of generating response entities which have content characteristics not acceptable according to the accept headers sent in the request.
Unless it was a HEAD request, the response SHOULD include an entity containing a list of available entity characteristics and location(s) from which the user or user agent can choose the one most appropriate. The entity format is specified by the media type given in the Content-Type header field. Depending upon the format and the capabilities of the user agent, selection of the most appropriate choice MAY be performed automatically. However, this specification does not define any standard for such automatic selection.
Note: HTTP/1.1 servers are allowed to return responses which are
not acceptable according to the accept headers sent in the
request. In some cases, this may even be preferable to sending a
406 response. User agents are encouraged to inspect the headers of
an incoming response to determine if it is acceptable.
If the response could be unacceptable, a user agent SHOULD temporarily stop receipt of more data and query the user for a decision on further actions.
That note in the middle about HTTP/1.1 may be your answer. I read it as saying "you may return a 200 in response to the PUT request to /People/Bob when the user agent specifies application/xml in the Accept header, selecting any suitable content-type, and that this outcome may be preferable to returning 406."
Under this scenario, the PUT would succeed on the server, return a 200, but the client would get an application/json representation. The client needs to be able to handle that possibility by making sure that it understands the media type given in the Content-type header, and behaving in a well-defined manner if it doesn't.
But this is always true anyway.
One more thing: you may want to consider not using plain-vanilla media types like application/xml and application/json, but instead define your own custom media types, maybe based on XHTML or JSON. All of the client-server coupling in a RESTful application happens through media types. Without media types rich enough to capture your domain concepts, you're incompletely specifying your REST API.
I would argue 'yes' in theory, but 'no' for real-world application.
I see the logic in not processing if there's an error. Since you return a 406, not a 500, I would know that it's not an error in the data I provided, but rather in the way the result is being presented to me.
That said, some applications won't check for error codes; they will just see that it came back with an error rather than the XML it asked for, and assume the transaction failed.
I assume your not handling application/xml is not an actual problem, but for the purposes of the question - if this is actually being deployed as a real-world service, you'd almost certainly want to be able to have an XML representation, as that's (I suspect) the most common RESTful interaction, and many callers would probably be hard-coded to use XML.
To sum up: if you actually aren't providing application/xml, then I would say, don't perform the update. If you're handling all the standards, but you're planning for the contingency where a user will ask for application/fooSomethingNonStandard, then go ahead and perform the update, but be sure you respond with a 406.
One way out of your conundrum is to have a successful PUT return a 204 (No Content). That way the client's Accept header is irrelevant to the issue of whether the update is performed.
A "RESTful" (or at least "HTTP-embracing") client will know not to update its current "page", and that it will have to do a GET in order to refresh its view of the just-PUT resource. The Accept header on that GET is now, of course, a separate concern from the update atomicity.
I would either succeed and return a 200 using the method Rich suggests above or a 406 and fail. The protocol does not allow for a more nuanced approach mixing 2xx (Success) with 4xx (Error) codes so 4xx can be read to imply NOT Success.
Related
Let's say a web application gets the following request:
POST /some/endpoint HTTP/1.1
Host: <something>
Accept: application/json
Accept-Language: pt
Content-Type: application/json
If-Match: "blabla"
Some body
If the server doesn't support HTTP 1.1 and the endpoint /some/endpoint does not exist, the former problem should likely be checked first, and a 505 rather than 404 should be returned.
If it just so happens that none of the endpoints of the server accept POST and the endpoint /some/endpoint doesn't exist, the latter should get priority, and 404 should be returned rather than 405.
If the Accept can't be provided and the body can't be appropriately decoded/validated, probably 406 should take precedence over 400.
These are cases where intuition might suffice. But there are a myriad other ones where it is not clear which of two non-2XX status codes should be preferred/checked first. For example, should Content-Type (resulting in 415) or Accept-Language (406) be returned if both would fail? 415 or 412? And on it goes...
Much of the time errors are pairwise independent: if the aspect that is relevant to one error being thrown (such as a particular header value) is fixed, the success/error status of another will not be affected. In those cases, the wrong error "priority" is perhaps only a nuisance. But sometimes it may be the case that these errors are not independent: I might have a resources as HTML in Portuguese, but in JSON only in English (humour me), so that if a client expects me to prioritise Accept-Language over Accept, and I do the opposite, the result will be quite bad.
The question should be evident now: are there any standards about which errors should be prioritised?
I haven't come across any relevant RFCs, or even much serious and general discussion. I know of the webmachine diagram, which sort of helps, but primarily just seems to describe a particular (well thought out) implementation rather than any standard.
Obviously, you can’t expect this question to be answered “no,” even though that’s probably the correct answer.
So let me address a particular point of yours instead:
I might have a resources as HTML in Portuguese, but in JSON only in English (humour me), so that if a client expects me to prioritise Accept-Language over Accept, and I do the opposite, the result will be quite bad.
In your example, you tell the server that Portuguese JSON is good, but all other combinations are equally bad. If that’s not the case, you can elaborate your preferences like this:
Accept: text/json
Accept-Language: pt, en;q=0.1
The server can then multiply your weights, getting 1×0.1=0.1 for English JSON and 0×1=0 for Portuguese HTML, and choosing the former.
(Sidenote 1: there is no text/json media type in the registry. You probably want application/json.)
(Sidenote 2: 415 Unsupported Media Type is not a correct response code for the scenarios you mention. It concerns the request body. If you cannot honor the Accept header, you can respond with 406 Not Acceptable, just as with Accept-Language.)
TL;DR: The specifications give the server ultimate authority in how it honors the request, even allowing the server to ignore the acceptable formats the client requests. However, the specifications instruct the server to make a best effort and to respond in a way that best helps the client recover from errors.
The specifications provide guidance, even if they don't (or can't) prioritize all possible error modes.
RFC 2616 § 10.4.7 says:
HTTP/1.1 servers are allowed to return responses which are
not acceptable according to the accept headers sent in the
request. In some cases, this may even be preferable to sending a
406 response. User agents are encouraged to inspect the headers of
an incoming response to determine if it is acceptable.
RFC 7231 § 3 says:
An origin server might be provided with, or be capable of generating,
multiple representations that are each intended to reflect the
current state of a target resource. In such cases, some algorithm is
used by the origin server to select one of those representations as
most applicable to a given request, usually based on content
negotiation.
RFC 7231 § 3.4 says:
Note that, in all cases, HTTP is not aware of the resource semantics.
The consistency with which an origin server responds to requests ... is determined entirely by whatever entity or algorithm selects
or generates those responses. HTTP pays no attention to the man
behind the curtain.
RFC 7231 § 3.3 says:
Response messages with an error status code
usually contain a payload that represents the error condition, such
that it describes the error state and what next steps are suggested
for resolving it.
RFC 2616 § 14.46 says:
The Warning general-header field is used to carry additional information about the status or transformation of a message which might not be reflected in the message. This information is typically used to warn about a possible lack of semantic transparency from caching operations or transformations applied to the entity body of the message.
(Emphases all mine.)
Section 3 of RFC 7231 gives the origin server ultimate authority to decide the appropriate response, even if that response is repugnant. Simultaneously, section 3 encourages the origin server to satisfy the request, or provide notice that it satisfied some of the request (Vary), or provide selectable options ("Passive negotiation").
Even though the server has ultimate authority, the specification makes clear to me that the responses should help the user resolve the problem. In my mind, the best error code is the one that helps the user best fix the problem!
Considering your pair-wise examples:
"If the server doesn't support HTTP 1.1 and the endpoint /some/endpoint does not exist, the former problem should likely be checked first, and a 505 rather than 404 should be returned."
No. Per the spec, an HTTP 1.1 client can GET from 1.0 server by protocol downgrade, so this kind of version negotiation is handled by the specification. Send a 404 (or a 301 if that's known) so the user can correct it.
"If it just so happens that none of the endpoints of the server accept POST and the endpoint /some/endpoint doesn't exist, the latter should get priority, and 404 should be returned rather than 405."
Yes, 404. If you're not getting to a resource, the method hardly matters.
"If the Accept can't be provided and the body can't be appropriately decoded/validated, probably 406 should take precedence over 400."
Never send 400 when you know 406 applies. You're giving the client less information, which is less helpful. However, the origin server is free to ignore the Accept header per RFC 7231 § 5.3.2:
If the [Accept] header field is
present in a request and none of the available representations for
the response have a media type that is listed as acceptable, the
origin server can either honor the header field by sending a 406 (Not
Acceptable) response or disregard the header field by treating the
response as if it is not subject to content negotiation.
"I might have a resources as HTML in Portuguese, but in JSON only in English (humour me), so that if a client expects me to prioritise Accept-Language over Accept, and I do the opposite, the result will be quite bad."
I disagree that the result will be bad. See RFC 7231 § 5.3.5:
the origin server can either disregard the [Accept-Language] header field by treating the response as if it is not subject to content negotiation or honor the header field by sending a 406 (Not Acceptable) response. However, the latter is not encouraged, as doing so can prevent users from accessing content that they might be able to use (with translation software, for example).
This pattern of specification language occurs more than once. "The server may disregard [whatever the client requested] by treating the response as if it's not subject to [this part of the specification], or the server may honor [the client request] and send [an applicable error code]. But, it's better to [send something intelligible] than only send [an inscrutable error code]."
At the end of the day, it's your API. HTTP provides only a window into your semantics. Document what you accept, how you respond, and with what. Send intelligible responses (HATEOAS is good) and, when applicable, the most specific error codes available.
What status code should be returned if a client sends an HTTP request and specifies a Content-Encoding header which cannot be decoded by the server?
Example
A client POSTs JSON data to a REST resource and encodes the entity body using the gzip coding. However, the server can only decode DEFLATE codings because it failed the gzip class in server school.
What HTTP response code should be returned? I would say 415 Unsupported Media Type but it's not the entity's Content-Type that is the problem -- it's the encoding of the otherwise supported entity body.
Which is more appropriate: 415? 400? Perhaps a custom response code?
Addendum: I have, of course, thoroughly checked rfc2616. If the answer is there I may need some new corrective eyewear, but I don't believe that it is.
Update:
This has nothing to do with sending a response that might be unacceptable to a client. The problem is that the client is sending the server what may or may not be a valid media type in an encoding the server cannot understand (as per the Content-Encoding header the client packaged with the request message).
It's an edge-case and wouldn't be encountered when dealing with browser user-agents, but it could crop up in REST APIs accepting entity bodies to create/modify resources.
As i'm reading it, 415 Unsupported Media Type sounds like the most appropriate.
From RFC 2616:
10.4.16 415 Unsupported Media Type
The server is refusing to service the request because the entity of the request is in a format not supported by the requested resource for the requested method.
Yeah, the text part says "media type" rather than "encoding", but the actual description doesn't include any mention of that distinction.
The new hotness, RFC 7231, is even explicit about it:
6.5.13. 415 Unsupported Media Type
The 415 (Unsupported Media Type) status code indicates that the
origin server is refusing to service the request because the payload
is in a format not supported by this method on the target resource.
The format problem might be due to the request's indicated
Content-Type or Content-Encoding, or as a result of inspecting the
data directly.
They should make that the final question on Who Wants To Be a Millionaire!
Well the browser made a request that the server cannot service because the information the client provided is in a format that cannot be handled by the server. However, this isn't the server's fault for not supporting the data the client provided, it's the client's fault for not listening to the server's Acccept-* headers and providing data in an inappropriate encoding. That would make it a Client Error (400 series error code).
My first instinct is 400 Bad Request is the appropriate response in this case.
405 Method Not Allowed isn't right because it refers to the HTTP verb being one that isn't allowed.
406 Not Acceptable looks like it might have promise, but it refers to the server being unable to provide data to the client that satisfies the Accept-* request headers that it sent. This doesn't seem like it would fit your case.
412 Precondition Failed is rather vaguely defined. It might be appropriate, but I wouldn't bet on it.
415 Unsupported Media Type isn't right because it's not the data type that's being rejected, it's the encoding format.
After that we get into the realm of non-standard response codes.
422 Unprocessable Entity describes a response that should be returned if the request was well-formed but if it was semantically incorrect in some way. This seems like a good fit, but it's a WebDAV extension to HTTP and not standard.
Given the above, I'd personally opt for 400 Bad Request. If any other HTTP experts have a better candidate though, I'd listen to them instead. ;)
UPDATE: I'd previously been referencing the HTTP statuses from their page on Wikipedia. Whilst the information there seems to be accurate, it's also less than thorough. Looking at the specs from W3C gives a lot more information on HTTP 406, and it's leading me to think that 406 might be the right code after all.
10.4.7 406 Not Acceptable
The resource identified by the request is only capable of generating
response entities which have content characteristics not acceptable
according to the accept headers sent in the request.
Unless it was a HEAD request, the response SHOULD include an entity
containing a list of available entity characteristics and location(s)
from which the user or user agent can choose the one most appropriate.
The entity format is specified by the media type given in the
Content-Type header field. Depending upon the format and the
capabilities of the user agent, selection of the most appropriate
choice MAY be performed automatically. However, this specification
does not define any standard for such automatic selection.
Note: HTTP/1.1 servers are allowed to return responses which are
not acceptable according to the accept headers sent in the
request. In some cases, this may even be preferable to sending a
406 response. User agents are encouraged to inspect the headers of
an incoming response to determine if it is acceptable.
If the response could be unacceptable, a user agent SHOULD temporarily
stop receipt of more data and query the user for a decision on further
actions.
While it does mention the Content-Type header explicitly, the wording mentions "entity characteristics", which you could read as covering stuff like GZIP versus DEFLATE compression.
One thing worth noting is that the spec says that it may be appropriate to just send the data as is, along with the headers to tell the client what format it's in and what encoding it uses, and just leave it for the client to sort out. So if the client sends a header indicating it accepts GZIP compression, but the server can only generate a response with DEFLATE, then sending that along with headers saying it's DEFLATE should be okay (depending on the context).
Client: Give me a GZIPPED page.
Server: Sorry, no can do. I can DEFLATE pack it for you. Here's the DEFLATE packed page. Is that okay for you?
Client: Welllll... I didn't really want DEFLATE, but I can decode it okay so I'll take it.
(or)
Client: I think I'll have to clear that with my user. Hold on.
As far as I know what GET can do, the same can be achieved by POST. So why was GET required in first place while defining HTTP protocol. If GET is only for fetching the resource, people can still update resources by sending the parameters values in URL. Why this loophole? Or the guy who did the coding on server side to update the resource on GET request has written a bad code?
HTTP specified different methods for different purposes. The GET method is intended to be used to “retrieve whatever information (in the form of an entity) is identified by the Request-URI”. Especially, it is intended to be a safe and idempotent method. That means a GET request should not have side effects (i.e. changing data):
In particular, the convention has been established that the GET and HEAD methods SHOULD NOT have the significance of taking an action other than retrieval.
And sending an identical request multiple times results in the same as sending it just once:
Methods can also have the property of "idempotence" in that (aside from error or expiration issues) the side-effects of N > 0 identical requests is the same as for a single request. The methods GET, HEAD, PUT and DELETE share this property.
Practically, no browser implements POSTing by clicking links (without intercepting the click event in JavaScript), nor bookmarking POST data. Furthermore, semantically POST and GET serve different purposes. One is for POSTing data to an application, the other is for GETting data from the application. These semantics have practical implications, but they also have theoretical design implications that speak to the quality of your application's design: an application that doesn't handle GET differently from POST probably has a great deal of security problems and workflow bugs.
From RFC 2616:
9.3 GET
The GET method means retrieve whatever
information (in the form of an entity)
is identified by the Request-URI. If
the Request-URI refers to a
data-producing process, it is the
produced data which shall be returned
as the entity in the response and not
the source text of the process, unless
that text happens to be the output of
the process.
The semantics of the GET method change
to a "conditional GET" if the request
message includes an If-Modified-Since,
If-Unmodified-Since, If-Match,
If-None-Match, or If-Range header
field. A conditional GET method
requests that the entity be
transferred only under the
circumstances described by the
conditional header field(s). The
conditional GET method is intended to
reduce unnecessary network usage by
allowing cached entities to be
refreshed without requiring multiple
requests or transferring data already
held by the client.
The semantics of the GET method change
to a "partial GET" if the request
message includes a Range header field.
A partial GET requests that only part
of the entity be transferred, as
described in section 14.35. The
partial GET method is intended to
reduce unnecessary network usage by
allowing partially-retrieved entities
to be completed without transferring
data already held by the client.
The response to a GET request is
cacheable if and only if it meets the
requirements for HTTP caching
described in section 13.
See section 15.1.3 for security
considerations when used for forms.
9.5 POST
The POST method is used to request
that the origin server accept the
entity enclosed in the request as a
new subordinate of the resource
identified by the Request-URI in the
Request-Line. POST is designed to
allow a uniform method to cover the
following functions:
- Annotation of existing resources;
- Posting a message to a bulletin board, newsgroup, mailing
list,
or similar group of articles;
- Providing a block of data, such as the result of submitting a
form, to a data-handling process;
- Extending a database through an append operation. The actual
function performed by the POST method
is determined by the server and is
usually dependent on the Request-URI.
The posted entity is subordinate to
that URI in the same way that a file
is subordinate to a directory
containing it, a news article is
subordinate to a newsgroup to which it
is posted, or a record is subordinate
to a database.
The action performed by the POST
method might not result in a resource
that can be identified by a URI. In
this case, either 200 (OK) or 204 (No
Content) is the appropriate response
status, depending on whether or not
the response includes an entity that
describes the result.
If a resource has been created on the
origin server, the response SHOULD be
201 (Created) and contain an entity
which describes the status of the
request and refers to the new
resource, and a Location header (see
section 14.30).
Responses to this method are not
cacheable, unless the response
includes appropriate Cache-Control or
Expires header fields. However, the
303 (See Other) response can be used
to direct the user agent to retrieve a
cacheable resource.
POST requests MUST obey the message
transmission requirements set out in
section 8.2.
See section 15.1.3 for security
considerations.
As stated, the response may change with GET if the request message has conditionals based on certain criteria. The POST requires that the server accept the request, no matter what.
Anytime you do a web search and you want to link someone to it, you can easily do it through:
http://www.google.com/search?q=lol
Can you imagine telling someone to do a POST request instead? A POST request isn't really bookmarkable like that, which is why GET is useful.
They simply have different purposes, as stated in other answers. GET is for GETing, POST is for POSTing.
Everything can also be achieved using raw TCP connections. Yet we often use HTTP rather than raw TCP connections because HTTP offers a layer of abstraction and, therefore, convenience and conforming implementations. Likewise, we use HTTP correctly (GETs, POSTs, PUTs, DELETEs, etc) rather than dumbly (POSTs only) because these verbs offer an additional layer of abstraction and, therefore, convenience and conforming implementations.
Lets say I want to send a variable to a page via a link, can I do that with POST? Nope, but with GET, I can send something over by doing ?variableName=someValue
You're right, everything can be tunnel through an HTTP POST. In fact, SOAP web services do exactly that. Everything is a POST using SOAP web services.
In that case, you are tunneling through HTTP, and not using HTTP to its fullest. If that's all you want to do, then that's fine.
However, if you wish to leverage HTTP for the features and benefits that it provides beyond simple message transport, then you should read the RFC and learn the rest of the HTTP protocol including GET, PUT, POST, DELETE, and all of the headers, cache management and result codes.
If a clients sends data in an unsupported media type to a HTTP server, the server answers with status "415 unsupported media type". But how to tell the client what media types are supported? Is there a standard or at least a recommended way to do so? Or would it just be written to the response body as text?
There is no specification at all for what to do in this case, so expect implementations to be all over the place. (What would be sensible would be if the server's response included something like an Accept: header since that has pretty much the right semantics, if currently in the wrong direction.)
I believe you can do this with the OPTIONS Http verb.
Also the status code of 300 Multiple Choices could be used if your scenario fits a certain use case. If they send a request with an Accept header of application/xml and you only support text/plain and that representation lives at a distinct URL then you can respond with a 300 and in the Location header the URL of that representation. I realize this might not exactly fit your question, but it's another possible option.
And from the HTTP Spec:
10.4.7 406 Not Acceptable
The resource identified by the request is only capable of generating response entities which have content characteristics not acceptable according to the accept headers sent in the request.
Unless it was a HEAD request, the response SHOULD include an entity containing a list of available entity characteristics and location(s) from which the user or user agent can choose the one most appropriate. The entity format is specified by the media type given in the Content-Type header field. Depending upon the format and the capabilities of the user agent, selection of the most appropriate choice MAY be performed automatically. However, this specification does not define any standard for such automatic selection.
Note: HTTP/1.1 servers are allowed to return responses which are
not acceptable according to the accept headers sent in the
request. In some cases, this may even be preferable to sending a
406 response. User agents are encouraged to inspect the headers of
an incoming response to determine if it is acceptable.
tl;dr;
Edited the generated proxy class to inherit from Microsoft.Web.Services3.WebServicesClientProtocol**.
I came across this question when troubleshooting this error, so I thought I would help the next person who might come through here, although not sure if it answers the question as stated. I ran into this error when at some point I had to take over an existing solution which was utilizing WSE and MTOM encoding. It was a windows client calling a web service.
To the point, the client was calling the web service where it would throw that error.
Something that contributed to resolving that error for me was to check the web service proxy class that apparently is generated by default to inherit from System.Web.Services.Protocols.SoapHttpClientProtocol.
Essentially that meant that it didn't actually use WSE3.
Anyhow I manually edited the proxy and changed it to inherit from Microsoft.Web.Services3.WebServicesClientProtocol.
BTW, to see the generated proxy class in VS click on the web reference and then click the 'Show All Files' toolbar button. The reference.cs is da place of joy!
Hope it helps.
In his book "HTTP Developer's Handbook" on page 81 Chris Shiflett explains what a 415 means, and then he says, "The media type used in the content of the HTTP response should be indicated in the Content-Type entity header."
1) So is Content-Type a possible answer? It would presumably be a comma-separated list of accepted content types. The obvious problem with this possibility is that Content-Type is an entity header not a response header.
2) Or is this a typo in the book? Did he really mean to say "the HTTP request"?
I'm designing a Web service. The request is idempotent, so I chose the GET method. The response is relatively expensive to calculate and not small, so I want to get caching (on the protocol level) right. (Don't worry about memoisation at my part, I have that already covered; my question here is actually also paying attention to the Web as a whole.)
There's only one mandatory parameter and a number of optional parameter with default values if missing. For example, the following two map to the same representation of the response. (If this is a dumb way to go about it the interface, propose something better.)
GET /service?mandatory_parameter=some_data HTTP/1.1
GET /service?mandatory_parameter=some_data;optional_parameter=default1;another_optional_parameter=default2;yet_another_optional_parameter=default3 HTTP/1.1
However, I imagine clients do not know this and would treat them separate and therefore waste cache storage. What should I do to avoid violating the golden rule of caching?
Make up a canonical form, document it (e.g. all parameters are required after all and need to be sorted in a specific order) and return a client error unless the required form is met?
Instead of an error, redirect permanently to the canonical form of a request?
Or is it enough to not mind how the request looks like, and just respond with the same ETag for same responses?
First, don't use semicolons as a delimiter in a query string. You should be using ? to begin a query string and & to delimit variable/value pairs. RFC 3986 doesn't explicitly say you have to use &, but the vast majority of existing code uses this delimiter because of the application/x-www-form-urlencoded precedent.
Second, you're right, in that parameters in a query string result in a different URI, and thus, as far as caches are concerned, a different resource. Assuming you want optimal caching performance, if you know that an optional parameter has been specified, and its inclusion is unnecessary and does not affect the representation that will be transmitted, you should be making a redirect to a canonical representation that omits the parameter. (i.e., An optional parameter is given with a value that is set to the default value. For example, if you have http://example.com:80/, you can normalize to http://example.com/ because 80 is the default value for the port with HTTP. You can do the same for query parameters since you control the URI space.) If you have parameters included (optional or otherwise) that appear in an order other than the canonical order, you should redirect for that too. A 301 redirect would be preferred if you know that the relationship between URIs will be stable. Otherwise, do a 302/307 redirect as appropriate. I would recommend defining your canonical form the same way that OAuth does: Sort each parameter alphabetically, first by key, then by value. Other normalization operations will also help out here. RFC 3986 has an entire section on URI normalization that will be relevant to you. This technique will really only work for GET, and redirects on PUT/POST/DELETE are not generally recommended.
Third, ETags are great, and they provide a huge performance improvement if implemented well by both the client and server. However, it's unfortunately rare for both sides to do it right. Ditto for Last-Modified. You should pursue these, because the CPU and bandwidth savings are significant when it works, but they are not sufficient on their own. Other headers like Cache-Control are also frequently necessary. It's worth familiarizing yourself with Section 13 of RFC 2616 if you're planning on going into great detail on this stuff.
Finally, a word of warning — there is an issue with these redirects you need to be aware of: Clients trying to access your resources may frequently be redirected to other locations. This introduces overhead that only gives you an overall savings if the clients make subsequent requests against the same resource, maintaining state to avoid the subsequent redirect. Unless you've open-sourced a reference client implementation that takes advantage of your caching optimizations, you may never benefit from these tweaks.
I would pick option (2) in your list - I would make the request RESTful, rather than RPC like.
I.e. in this case, if you make all of the parameters parts of the request path:
/service/mandatory_parameter/some_data/optional_parameter/default1/another_optional_parameter/default2/yet_another_optional_parameter/default3
In the case where not all of the optional parameters are specified, return a 301 (Permanent redirect) to the full resource name with the defaults filled in. This will (or should) be cached by clients and web caches appropriately, and even if it gets to your backend then making the 301 should be very cheap for you.
At which point, you have one canonical form for the URI, and caching will work as normal/expected.
This does mean that every combination of parameters will be cached separately (as a 301), however that's fine really as the non-canonical requests will have an independent cache policy to the full request and clients which are worried about the extra round trip can fill in all the parameters themselves.
Your option (3) won't work as you expect - each form will be cached independently as they're different URIs.
It should also be noted that a lot of downstream caches / software won't cache your response at all due to the query parameters, which is why I suggest turning it into a 'proper' resource..
First it's a good thing you choice GET since other methods don't have as good caching support. As far as I know browsers do cache URIs with respect to the parameters so I don't think It's a good idea to use a canonical form.
One thing that you don't state here is how this service is going to be used. If those requests are made from a browser (and it looks to me that those are probably issued from a script) requests will probably look the same even if they are asked for more than once. So make sure that whatever generate the URI end up with the same URI for equal input data (remove default parameters or always include them).
When it comes to the ETag I recommend you to have this, though I would like to clarify how it works; You get the request, you process all your "expensive calculations" and then if there were a If-None-Match header with the same hash (ETag) as your processed response you may return 304 Not-Modified. So ETag is used to avoid transmitting the response if the client already have it. (Sure you may implement caching on server-side, but this is better to do based on input parameters).
To further improve cache hits on client side you may want to set proper caching headers in you response.
I asked almost the same question for me some month ago. My answer I describe on an example of my realization.
On the server side I have WFC service which receive requests in one of the following forms
GET /Service/RequestedData?param1=data1¶m2=data2…
GET /Service/RequestedData/IdOfData?param1=data1¶m2=data2…
PUT /Service/RequestedData/IdOfData // with param1=data1¶m2=data2… in body
POST /Service/RequestedData/IdOfData // with param1=data1¶m2=data2… in body
DELETE /Service/RequestedData/IdOfData
So requests are in REST for, but GET requests have some optional parameters. Especially this part is a port of your interest.
Because WFC support a URL templates, the prototype of functions which reply to a client request looks like
[WebGet (UriTemplate = "RequestedData?param1={myParam1}¶m2={myParam2}",
ResponseFormat = WebMessageFormat.Json)]
[OperationContract]
MyResult GetData (string myParam1, int myParam2);
All requests like
GET /Service/RequestedData?param1=¶m2=data2
GET /Service/RequestedData?param2=data2¶m1=
GET /Service/RequestedData?param2=data2
will be mapped to the same call from the side of my WCF service. So I have one problem less.
Now at the beginning of implementation of every method which response to HTTP GET request I set in the HTTP header "Cache-Control: max-age=0". It means that client always try to verify client browser cache and no ajax requests will be not easy responded from the local cache like it can do Internet Explorer.
Next I calculate always an ETag based on my data. The exact algorithm is a subject of separate discussion, but important is, that in all responses to HTTP GET requests exist ETag in the HTTP header.
So clients every time verify his local cache and send GET request to server. They send the ETag, which come from its local cache, inside of "If-None-Match" HTTP header. Server computes the ETag which has data, which will be sending back to this GET request. It ETag of data is the same as in the client request server send back response with empty body and the code "304 Not Modified" back. In this case browser gives data from the local cache.
If the same client from a unknown reason create a new version of URL request, which will be interpret from the web browser as a new URL, then web browser will not find old server response in the local cache and send one more time the same request to the server. Is it a real problem? The server send the data one more time. If you have a server side caching you can makes a little more optimization. In the most cases, the URL of GET requests will be produced by a client side JavaScript so you will be no time have such situation.
Calculation of ETag and setting of "Cache-Control: max-age=0" and Etag header as well as setting "304 Not Modified" code should do WFC service, but it is very easy.
The most important is that my implementation of ETag calculation is not as expansive as getting the whole data from the database server and calculation MD5 cache from there. I use permanently rowversion data type in every row of data in the SQL Server database. This rowversion is nothing other as a counter of changes in the database. If one change a row of data rowversion value in the corresponding row will be incremented. So if one makes SELECT statement from maximum value of rowversion value, and this value is not changed comparing with the previous requests, one can be sure that the data were not changed in the time period. The algorithm of calculation of ETag should be only sensitive to deleting of data from the table. But it is also a solved problem. A little more about this you can read in Concurrency handling of Sql transactrion.
I don’t want suggest my ETag calculation as a best choice, I want only say, that calculation of ETag can be much cheaper as calculation MD5 from the whole data.
In case of errors Server throws an exception which will be mapped to a HTTP code, which I define in the throw statement. As a body WFC sends a standard JSON object {"description":"My error text"}. A custom error object is also possible (see Is WebProtocolException included in .net 4.0?). On the client side I use jQuery and in the corresponding jQuery.ajax inside of error event handler the error message will be decoded and displayed to the user.
So my recommendation: usage of ETag together with "Cache-Control: max-age=0" for all HTTP GET requests. For all other requests I’ll recommend you implement RESTfull service. For the error implementation you should look at the most native way which is supported by the software used for server and client implementation and use this.
UPDATED: To clear the URL structure I should add following. In my service the main part like GET /Service/RequestedData/IdOfData describes data objects requested. Parameters param1=data1¶m2=data2 corresponds mostly the information about sorting, paging and filtering of data. I use active jqGrid plugin for jQuery and if the end-user scroll in the grid to the next page, click on the column header (sorting of data) or if he set a filter with respect of searching feature, all these follows to different optional parameters appended the main URL.