I need to create an API that has four possible HTTP query parameters. Either parameter one or parameter two is required. The others are optional. From the official RAML version 1.0 specification on Github, I found an almost exact scenario in the RAML queryString example.
I loaded it into Mulesoft Design Center to test it out. The RAML produces no errors in Design Center, and everything looks okay. According to the first example in the RAML, the following URL should produce a success (200 OK):
GET https://(mocking URL)/locations?start=2&lat=12&long=13
When I send it via Postman, it reaches the mocking service, but I get the following error:
{
"code": "REQUEST_VALIDATION_ERROR",
"message": "Error validating query string: expected type: JSONObject, found: Null"
}
I'm not sure if this is a Design Center limitation or if there's something off in my URL. Does anyone know what I'm doing wrong?
Here's the RAML sample from the official spec:
#%RAML 1.0
title: Illustrate query parameter variations
types:
lat-long: # lat & long required; mutually exclusive with location
properties:
lat: number
long: number
loc: # location required; mutually exclusive with lat & long
properties:
location:
paging: # each is optional, not exclusive with anything
properties:
start?: number
page-size?: number
/locations:
get:
queryString:
type: [paging, lat-long | loc ]
examples:
first:
value:
start: 2
lat: 12
long: 13
second:
value:
start: 2
page-size: 20
location: 1,2
third: # not valid
value:
lat: 12
location: 2
strict: false # because it's not valid
The RAML specification explicitly does not define validation for object types:
RAML does not define validation when a query parameter declaration
specifies any of the following types for the value of the query
parameter: an object type, a union of non-scalar types, or an array
type if the underlying type of the array is an object type or union of
non-scalar types. Processors MAY default to treating the format of the
query parameter value as JSON in applying the type to instances of
that query parameter, or they MAY allow other treatments based on
annotations.
Even if the Mocking Service implements the validation eventually, it might be better to use simple types for query parameters, like strings and numbers. It makes sense because usually query parameters are just used that way.
This looks like a bug in the MuleSoft Anypoit mocking tool.
The RAML spec is correct.
You may wish to raise a defect with MuleSoft support.
Related
In a 406 Not Acceptable response:
The server SHOULD generate a payload containing a list of available
representation characteristics and corresponding resource identifiers
from which the user or user agent can choose the one most appropriate.
A user agent MAY automatically select the most appropriate choice from
that list. However, this specification does not define any standard
for such automatic selection, as described in RFC7231 Section 6.4.1.
Is there a preferred format for that "list of available representation characteristics and corresponding resource identifiers"?
I can send a response like:
{ Acceptable: ["application/json", "application/pdf"] }
But then I am assuming a default Content-Type for the 406 payload (JSON in this case).
Or should I send a very simple, almost format-less, payload like:
application/json,application/pdf
Is there a preferred format for that "list of available representation characteristics and corresponding resource identifiers"?
There's no standard for such payload.
You could choose any format that can be easily parsed by the user agent. In practice, both JSON or text should be fine:
{ "acceptable" : [ "application/json", "application/pdf" ] }
application/json,application/pdf
See the following quote from the section 6.4.1 of the RFC 7231, which is referenced in the 406 status code definition:
[...] A specific format for automatic selection is not defined by
this specification because HTTP tries to remain orthogonal to the
definition of its payloads. In practice, the representation is
provided in some easily parsed format believed to be acceptable to
the user agent, as determined by shared design or content
negotiation, or in some commonly accepted hypertext format. [...]
MDN Web Docs from Mozilla suggests the following:
[...] In reality, this error is very rarely used: instead of responding using this error code, which would be cryptic for the end user and difficult to fix, servers ignore the relevant header and serve an actual page to the user. It is assumed that even if the user won't be completely happy, they will prefer this to an error code. [...]
A query using the extended metadata entities does not work. Do i need to pass these parameters using a different URL Parameter other than "attributes"
For example, the following query does not work (returns 400)
GET https://api.projectoxford.ai/academic/v1.0/evaluate?expr=Composite(J.JN=='computer')&model=latest&count=10&offset=0&attributes=Id,Ti,Y,D,CC,AuN,AA.AfN,J.JN,W,E,D,E,S.U HTTP/1.1
Host: api.projectoxford.ai
Ocp-Apim-Subscription-Key: ••••••••••••••••••••••••••••••••
Simply removing "S.U" from the list of entities makes it work
Thanks for your interest in Microsoft Academic!
Attribute “E” is a JSON string which consists of extended metadata attributes.
You can get only full contents of E, i.e. all extended metadata attributes.
You can use only entity attributes in “attributes” parameter of URI, not extended metadata attributes.
https://www.microsoft.com/cognitive-services/en-us/academic-knowledge-api/documentation/entityattributes>
So, if you need to get S.U you should get the value of attribute "E" and parse corresponding JSON.
Example (here I use only three attributes: Id,Ti, E in order to make output shorter):
https://api.projectoxford.ai/academic/v1.0/evaluate?expr=Composite(J.JN=='computer')&model=latest&count=10&offset=0&attributes=Id,Ti,E
Please, let us know if you have any questions or suggestions.
We would love to hear what you think about Cognitive Services:
http://cognitive.uservoice.com/
My application has a resource at /foo. Normally, it is represented by an HTTP response payload like this:
{"a": "some text", "b": "some text", "c": "some text", "d": "some text"}
The client doesn't always need all four members of this object. What is the RESTfully semantic way for the client to tell the server what it needs in the representation? e.g. if it wants:
{"a": "some text", "b": "some text", "d": "some text"}
How should it GET it? Some possibilities (I'm looking for correction if I misunderstand REST):
GET /foo?sections=a,b,d.
The query string (called a query string after all) seems to mean "find resources matching this condition and tell me about them", not "represent this resource to me according to this customization".
GET /foo/a+b+d My favorite if REST semantics doesn't cover this issue, because of its simplicity.
Breaks URI opacity, violating HATEOAS.
Seems to break the distinction between resource (the sole meaning of a URI is to identify one resource) and representation. But that's debatable because it's consistent with /widgets representing a presentable list of /widget/<id> resources, which I've never had a problem with.
Loosen my constraints, respond to GET /foo/a, etc, and have the client make a request per component of /foo it wants.
Multiplies overhead, which can become a nightmare if /foo has hundreds of components and the client needs 100 of those.
If I want to support an HTML representation of /foo, I have to use Ajax, which is problematic if I just want a single HTML page that can be crawled, rendered by minimalist browsers, etc.
To maintain HATEOAS, it also requires links to those "sub-resources" to exist within other representations, probably in /foo: {"a": {"url": "/foo/a", "content": "some text"}, ...}
GET /foo, Content-Type: application/json and {"sections": ["a","b","d"]} in the request body.
Unbookmarkable and uncacheable.
HTTP does not define body semantics for GET. It's legal HTTP but how can I guarantee some user's proxy doesn't strip the body from a GET request?
My REST client won't let me put a body on a GET request so I can't use that for testing.
A custom HTTP header: Sections-Needed: a,b,d
I'd rather avoid custom headers if possible.
Unbookmarkable and uncacheable.
POST /foo/requests, Content-Type: application/json and {"sections": ["a","b","d"]} in the request body. Receive a 201 with Location: /foo/requests/1. Then GET /foo/requests/1 to receive the desired representation of /foo
Clunky; requires back-and-forth and some weird-looking code.
Unbookmarkable and uncacheable since /foo/requests/1 is just an alias that would only be used once and only kept until it is requested.
I would suggest the querystring solution (your first). Your arguments against the other alternatives are good arguments (and ones that I've run into in practise when trying to solve the same problem). In particular, the "loosen the constraints/respond to foo/a" solution can work in limited cases, but introduces a lot of complexity into an API from both implementation and consumption and hasn't, in my experience, been worth the effort.
I'll weakly counter your "seems to mean" argument with a common example: consider the resource that is a large list of objects (GET /Customers). It's perfectly reasonable to page these objects, and it's commonplace to use the querystring to do that: GET /Customers?offset=100&take=50 as an example. In this case, the querystring isn't filtering on any property of the listed object, it's providing parameters for a sub-view of the object.
More concretely, I'd say that you can maintain consistency and HATEOAS through these criteria for use of the querystring:
the object returned should be the same entity as that returned from the Url without the querystring.
the Uri without the querystring should return the complete object - a superset of any view available with a querystring at the same Uri. So, if you cache the result of the undecorated Uri, you know you have the full entity.
the result returned for a given querystring should be deterministic, so that Uris with querystrings are easily cacheable
However, what to return for these Uris can sometimes pose more complex questions:
returning a different entity type for Uris differing only by querystring could be undesirable (/foo is an entity but foo/a is a string); the alternative is to return a partially-populated entity
if you do use different entity types for sub-queries then, if your /foo doesn't have an a, a 404 status is misleading (/foo does exist!), but an empty response may be equally confusing
returning a partially-populated entity may be undesirable, but returning part of an entity may not be possible, or may be more confusing
returning a partially populated entity may not be possible if you have a strong schema (if a is mandatory but the client requests only b, you are forced to return either a junk value for a, or an invalid object)
In the past, I have tried to resolve this by defining specific named "views" of required entities, and allowing a querystring like ?view=summary or ?view=totalsOnly - limiting the number of permutations. This also allows for definition of a subset of the entity that "makes sense" to the consumer of the service, and can be documented.
Ultimately, I think that this comes down to an issue of consistency more than anything: you can meet HATEOAS guidance using the querystring relatively easily, but the choices you make need to be consistent across your API and, I'd say, well documented.
I've decided on the following:
Supporting few member combinations: I'll come up with a name for each combination. e.g. if an article has members for author, date, and body, /article/some-slug will return all of it and /article/some-slug/meta will just return the author and date.
Supporting many combinations: I'll separate member names by hyphens: /foo/a-b-c.
Either way, I'll return a 404 if the combination is unsupported.
Architectural constraint
REST
Identifying resources
From the definition of REST:
a resource R is a temporally varying membership function MR(t), which for time t maps to a set of entities, or values, which are equivalent. The values in the set may be resource representations and/or resource identifiers.
A representation being an HTTP body and an identifier being a URL.
This is crucial. An identifier is just a value associated with other identifiers and representations. That's distinct from the identifier→representation mapping. The server can map whatever identifier it wants to any representation, as long as both are associated by the same resource.
It's up to the developer to come up with resource definitions that reasonably describe the business by thinking of categories of things like "users" and "posts".
HATEOAS
If I really care about perfect HATEOAS, I could put a hyperlink somewhere in the /foo representation to /foo/members, and that representation would just contain a hyperlink to every supported combination of members.
HTTP
From the definition of a URL:
The query component contains non-hierarchical data that, along with data in the path component, serves to identify a resource within the scope of the URI's scheme and naming authority (if any).
So /foo?sections=a,b,d and /foo?sections=b are distinct identifiers. But they can be associated within the same resource while being mapped to different representations.
HTTP's 404 code means that the server couldn't find anything to map the URL to, not that the URL is not associated with any resource.
Functionality
No browser or cache will ever have trouble with slashes or hyphens.
Actually it depends on the functionality of the resource.
If for example the resource represents an entity:
/customers/5
Here the '5' represents an id of the customer
Response:
{
"id": 5,
"name": "John",
"surename": "Doe",
"marital_status": "single",
"sex": "male",
...
}
So if we will examine it closely, each json property actually represents a field of the record on customer resource instance.
Let's assume consumer would like to get partial response, meaning, part of the fields. We can look at it as the consumer wants to have the ability to select the various fields via the request, which are interesting to him, but not more (in order to save traffic or performance, if part of the fields are hard to compute).
I think in this situation, the most readable and correct API would be (for example, get only name and surename)
/customers/5?fields=name,surename
Response:
{
"name": "John",
"surename": "Doe"
}
HTTP/1.1
if illegal field name is requested - 404 (Not Found) is returned
if different field names are requested - different responses will be generated, which also aligns with the caching.
Cons: if the same fields are requested, but the order is different between the fields (say: fields=id,name or fields=name,id), although the response is the same, those responses will be cached separately.
HATEOAS
In my opinion pure HATEOAS is not suitable for solving this particular problem. Because in order to achieve that, you need a separate resource for every permutation of field combinations, which is overkill, as it is bloating the API extensively (say you have 8 fields in a resource, you will need permutations!).
if you model resources only for the fields but not all the permutations, it has performance implications, e.g. you want to bring the number of round trips to minimum.
If a,b,c are property of a resource like admin for role property the right way is to use is the first way that you've suggested GET /foo?sections=a,b,d because in this case you would apply a filter to the foo collection. Otherwise if a,b and c are a singole resource of foo collection the the way that would follow is to do a series of GET requests /foo/a /foo/b /foo/c. This approach, as you said, has a high payload for request but it is the correct way to follow the approach Restfull. I would not use the second proposal made by you because plus char in a url has a special meaning.
Another proposal is to abandon use GET and POST and create an action for the foo collection like so: /foo/filter or /foo/selection or any verb that represent an action on the collection. In this way, having a post request body, you can pass a json list of the resource you would.
you could use a second vendor media-type in the request header application/vnd.com.mycompany.resource.rep2, you can't bookmark this however, query-parameters are not cacheable (/foo?sections=a,b,c) you could take a look at matrix-parameters however regarding this question they should be cacheable URL matrix parameters vs. request parameters
I'm wondering whether to use matrix or query parameters in my URLs. I found an older discussion to that topic not satisfying.
Examples
URL with query params: http://some.where/thing?paramA=1¶mB=6542
URL with matrix params: http://some.where/thing;paramA=1;paramB=6542
At first sight matrix params seem to have only advantages:
more readable
no encoding and decoding of "&" in XML documents is required
URLs with "?" are not cached in many cases; URLs with matrix params are cached
matrix parameters can appear everywhere in the path and are not limited to its end
matrix parameters can have more than one value: paramA=val1,val2
But there are also disadvantages:
only a few frameworks like JAX-RS support matrix parameters
When a browser submits a form via GET, the params become query params. So it ends up in two kinds of parameters for the same task. To not confuse users of the REST services and limit the effort for the developers of the services, it would be easier to use always query params - in this area.
Since the developer of the service can choose a framework with matrix param support, the only remaining disadvantage would be that browsers create by default query parameters.
Are there any other disadvantages? What would you do?
The important difference is that matrix parameters apply to a particular path element while query parameters apply to the request as a whole. This comes into play when making a complex REST-style query to multiple levels of resources and sub-resources:
http://example.com/res/categories;name=foo/objects;name=green/?page=1
It really comes down to namespacing.
Note: The 'levels' of resources here are categories and objects.
If only query parameters were used for a multi-level URL, you would end up with
http://example.com/res?categories_name=foo&objects_name=green&page=1
This way you would also lose the clarity added by the locality of the parameters within the request. In addition, when using a framework like JAX-RS, all the query parameters would show up within each resource handler, leading to potential conflicts and confusion.
If your query has only one "level", then the difference is not really important and the two types of parameters are effectively interchangeable, however, query parameters are generally better supported and more widely recognized. In general, I would recommend that you stick with query parameters for things like HTML forms and simple, single-level HTTP APIs.
In addition to Tim Sylvester's answer I would like to provide an example of how matrix parameters can be handled with JAX-RS .
Matrix parameters at the last resource element
http://localhost:8080/res/categories/objects;name=green
You can access them using the #MatrixParam annotation
#GET
#Path("categories/objects")
public String objects(#MatrixParam("name") String objectName) {
return objectName;
}
Response
green
But like the Javadoc states
Note that the #MatrixParam annotation value refers to a name of a matrix parameter that resides in the last matched path segment of the Path-annotated Java structure that injects the value of the matrix parameter.
... what brings us to point 2
Matrix parameters in the middle of an URL
http://localhost:8080/res/categories;name=foo/objects;name=green
You can access matrix parameters anywhere using path variables and #PathParam PathSegment.
#GET
#Path("{categoryVar:categories}/objects")
public String objectsByCategory(#PathParam("categoryVar") PathSegment categorySegment,
#MatrixParam("name") String objectName) {
MultivaluedMap<String, String> matrixParameters = categorySegment.getMatrixParameters();
String categorySegmentPath = categorySegment.getPath();
String string = String.format("object %s, path:%s, matrixParams:%s%n", objectName,
categorySegmentPath, matrixParameters);
return string;
}
Response
object green, path:categories, matrixParams:[name=foo]
Since the matrix parameters are provided as a MultivaluedMap you can access each by
List<String> names = matrixParameters.get("name");
or if you only need the first one
String name = matrixParameters.getFirst("name");
Get all matrix parameters as one method parameter
http://localhost:8080/res/categories;name=foo/objects;name=green//attributes;name=size
Use a List<PathSegment> to get them all
#GET
#Path("all/{var:.+}")
public String allSegments(#PathParam("var") List<PathSegment> pathSegments) {
StringBuilder sb = new StringBuilder();
for (PathSegment pathSegment : pathSegments) {
sb.append("path: ");
sb.append(pathSegment.getPath());
sb.append(", matrix parameters ");
sb.append(pathSegment.getMatrixParameters());
sb.append("<br/>");
}
return sb.toString();
}
Response
path: categories, matrix parameters [name=foo]
path: objects, matrix parameters [name=green]
path: attributes, matrix parameters [name=size]
--Too important to be relegated to comment section.--
I'm not sure what the big deal is with matrix URLs. According to the w3c design article that TBL wrote, it was just a design idea and explicitly states that it's not a feature of the web. Things like relative URLs aren't implemented when using it. If you want to use it, that's fine; there's just no standard way to use it because it's not a standard.
— Steve Pomeroy.
So short answer is, if you need RS for business purpose, you are better off using request parameter.
I am having trouble on finding authoritative information about the behavior with HTTP GET query string duplicate fields, like
http://example.com/page?field=foo&field=bar
and in particular if the order is kept or not. Most web-oriented languages produce an array containing both foo and bar associated to a key "field", but I would like to know if authoritative statement exist (e.g. on a RFC) about this point. RFC 3986 has a section 3.4. Query, which refers to key=value pairs, but nothing is said on how to interpret order and duplicate fields and so on. This makes sense, since it's backend dependent, and not in the scope of that RFC...
Although a de-facto standard exists, I'd like to see an authoritative source for it, just out of curiosity.
There is no spec on this. You may do what you like.
Typical approaches include: first-given, last-given, array-of-all, string-join-with-comma-of-all.
Suppose the raw request is:
GET /blog/posts?tag=ruby&tag=rails HTTP/1.1
Host: example.com
Then there are various options for what request.query['tag'] should yield, depending on the language or the framework:
request.query['tag'] => 'ruby'
request.query['tag'] => 'rails'
request.query['tag'] => ['ruby', 'rails']
request.query['tag'] => 'ruby,rails'
The situation seems to have changed since this question was asked and the accepted answer was written 12 years ago. I believe we now have an authoritative source: The WHATWG URL Standard describes the process of extracting and parsing a query string in detail in section 6.2 (https://url.spec.whatwg.org/#interface-urlsearchparams) and section 5.1 on x-www-form-urlencoded parsing (https://url.spec.whatwg.org/#urlencoded-parsing). The parsing output is "an initially empty list of name-value tuples where both name and value hold a string", where a list is defined as a finite ordered sequence, and the key-value pairs are added to this list in the order they appear in the URL. At first there is no mention of repeated keys, but some methods on the URLSearchParams class in section 6.2 (https://url.spec.whatwg.org/#interface-urlsearchparams) set clear expectations on ordering: "The getAll(name) method steps are to return the values of all name-value pairs whose name is name... in list order"; The sort() method specifies that "The relative order between name-value pairs with equal names must be preserved." (Emphasis mine). Examining the Github issue referenced in the commit where the sort method was added, we see that the original proposal was to sort on values where keys were identical, but this was changed: "The reason for the default sort not affecting the value order is that ordering of the values can be significant. We should not assume that it's ok to move the order of the values around." (https://github.com/whatwg/url/issues/26#issuecomment-271600764)
I can confirm that for PHP (at least in version 4.4.4 and newer) it works like this:
GET /blog/posts?tag=ruby&tag=rails HTTP/1.1
Host: example.com
results in:
request.query['tag'] => 'rails'
But
GET /blog/posts?tag[]=ruby&tag[]=rails HTTP/1.1
Host: example.com
results in:
request.query['tag'] => ['ruby', 'rails']
This behavior is the same for GET and POST data.
yfeldblum's answer is perfect.
Just a note about a fifth behavior I noticed recently: on Windows Phone, opening an application with an uri with a duplicate query key will result in NavigationFailed with:
System.ArgumentException: An item with the same key had already been added.
The culprit is System.Windows.Navigation.UriParsingHelper.InternalUriParseQueryStringToDictionary(Uri uri, Boolean decodeResults).
So the system won't even let you handle it the way you want, it will forbid it. You are left with the only solution to choose your own format (CSV, JSON, XML, ...) and uri-escape-it.
Most (all?) of the frameworks offer no guarantees, so assume they will be returned in random order.
Always take the safest approach.
For example, java HttpServlet interface:
ServletRequest.html#getParameterValues
Even the getParameterMap method leaves out any mention about parameter order (the order of a java.util.Map iterator cannot be relied on either.)
Typically, duplicate parameter values like
http://example.com/page?field=foo&field=bar
result in a single queryString parameter that is an array:
field[0]=='foo'
field[1]=='bar'
I've seen this behavior in ASP, ASP.NET and PHP4.
The ?array[]=value1&array[]=value2 approach is certainly a very popular one.
supported by most Javascript frameworks
supported by Java Spring
supported by PHP