RESTful data structure patterns - http

I tried Googling and searching everywhere, but couldn't find a definitive authority on this topic. While being true to REST principles, how should I design the HTTP interface for:
An ordered list (get, add, insert into position, reorder, remove)
A set (get, add, remove)
A hash-table (get, add, remove)
NOTE: These data structures are to contain references to existing resources with known ids

That's how I would do it for an ordered list and hash table. I guess the methods would be the same for a set and a list:
Ordered list
Get item 123:
GET /list/123
Append an item to the list:
POST /list/
Insert new item into position 5:
POST /list/?position=5
Move item 123 to position 3:
PUT /list/123?position=3
Delete item 123:
DELETE /list/123
Delete item at position 3:
DELETE /list/?position=3
Of course, your API should update the indexes of all the elements when doing insertion and deletion.
Hash table
Get item "somekey":
GET /hashtable/somekey
Add item "somekey":
POST /hashtable/somekey
Remove item "somekey":
DELETE /hashtable/somekey

#dadads
You can not define such interface directly.
An ordered list (get, add, insert into position, reorder, remove)
By excluding "insert into position" and "reorder" you can perfectly implement "get", "add" and "remove" for example:
You define your resource /service/users
You can use POST /service/users to add new user to the "users" collection
You can GET /service/users to retrieve users
You can GET /service/users/user-id to retrieve particular user
You can DELETE /service/users/user-id from users collection
This is a very rough example, though it outlines some ideas. In order to achieve "reorder" and "insert into position" you need to implement your own action semantics which you can include in your resource representation and let client know HOW to perform these operations. As a reference you can see this JSON PATCH specification proposal: https://www.rfc-editor.org/rfc/rfc6902 which tries to describe such operations.
It is not necessary to use already existing media format, you can define your own under your own namespace for example: application/vnd.your-company.format-name+json which describes these capabilities and also advertises this information to clients.

You should decouple the transport mechanism from the underlying application. I would consider designing the application correctly, then figure out how to access it via HTTP. This way you could easily add or change the transport mechanisms (SOAP, SCA, etc) without affecting the underlying application.
Once you have the application correctly designed, consider accessing it from the HTTP requests via something like an Adapter or Visitor pattern.

This is my idea for reordering.
There is a HTTP method called PATCH that is used to update fragments of a resource. Give your resource a new property called index, then make a call with PATCH method
PATCH /collection
[
{
"id: "original index 0"
"index": 1
}
{
"id: "original index 1"
"index": 0
}
]
Then your server back-end needs to figure out how to do this atomically. But interface-wise, I think this is the best way to stay true to RESTful.
Alternatively, there is a better solution, but it may not apply to everyone's case. Since ordering always depends some sort of criteria, it can even be as simple as insertion order. Let your collection url support an orderBy query string, and let this orderBy dictate on how the result gets ordered. Then during your reordering call from client, just update the resource's property used for the ordering criteria.

I came to this question mostly looking for a RESTful way to reorder. I don't really like any of the answers, so here is what I think is most RESTful.
For reorder you could make the order a resource:
/list/order
Then you can do normal operations on it (for these examples assume a list with 5 items currently in it):
"items":" [
{
"id": "A",
"name": "Monkey"
},
{
"id": "B",
"name": "Cow"
},
{
"id": "C",
"name": "Horse"
},
{
"id": "D",
"name": "Turkey"
},
{
"id": "E",
"name": "Tasmanian Devil"
},
]
Note that "order" is not included in the resource response. It's not needed - the order is implicitly specified by the response order of the items.
GET /list/order
returns a list of item ids in their correct order
['A','B','C','D','E']
POST /list/order
with payload ['D','B','C','A','E']
GET /list/order
returns a list of item ids in their correct order
['D','B','C','A','E']
Also obviously you would return the items in the list in the correct order when you do a GET on /list.
GET /list
returns a list of items in their correct order
"items":" [
{
"id": "D",
"name": "Turkey"
},
{
"id": "B",
"name": "Cow"
},
{
"id": "C",
"name": "Horse"
},
{
"id": "A",
"name": "Monkey"
},
{
"id": "E",
"name": "Tasmanian Devil"
},
]

Related

Composite index for optional field in Cosmos

I have a collection in Cosmos DB which contains documents of different types (and schemas):
{
"partKey": "...",
"type": "type1",
"data": {
"field1": 123,
"field2": "sdfsdf"
}
}
{
"partKey": "...",
"type": "type2",
"data": {
"field3": ["123", "456", "789"]
}
}
I'm trying to create a composite index [/type, /data/field3/[]/?], but faced an issue:
The indexing path '\\/data\\/field3\\/[]\\/?' could not be accepted, failed near position '15'. Please ensure that the path is a valid path. Common errors include invalid characters or absence of quotes around labels
We don't support wildcards for Composite Indexes in Cosmos DB. Here is a composite index sample as reference.
We will update our docs to be more clear in this. I looked over these and we don't currently document this today.
Thanks.
In composite indexes, you just need to specify the paths that you want to index, rather than the values, so for your example:
"compositeIndexes":[
[
{
"path":"/type",
"order":"ascending"
},
{
"path":"/data/field3",
"order":"descending"
}
]
]
Just specify the order type you need for your queries (I've just used these ones as an example).
For different documents that have different properties underneath your data property, I believe you will have to add each composite index for each use case that you need since composite indexes don't support wildcards, so you would need to add:
/data/field1 /data/field2 etc etc
Hope this helps.

Cosmos DB queries - using ORDER BY when a property does not exist in all documents

We are experiencing an issue in when writing queries for Cosmos Document DB and we want to create a new document property and use it in an ORDER BY clause
If, for example, we had a set of documents like:
{
"Name": "Geoff",
"Company": "Acme"
},
{
"Name": "Bob",
"Company": "Bob Inc"
}
...and we write a query like SELECT * FROM c ORDER BY c.Name this works fine and returns both documents
However, if we were to add a new document with an additional property:
{
"Name": "Geoff",
"Company": "Acme"
},
{
"Name": "Bob",
"Company": "Bob Inc"
},
{
"Name": "Sarah",
"Company": "My Company Ltd",
"Title": "President"
}
...and we write a query like SELECT * FROM c ORDER BY c.Title it will only return the document for Sarah and excludes the 2 without a Title property.
This means that the ORDER BY clause is behaving like a filter rather than just a sort, which seems unexpected.
It seems that all document schemas are likely to add properties over time. Unless we go back and add these properties to all existing document records in the container then we can never use them in an ORDER BY clause without excluding records.
Does anyone have a solution to allow the ORDER BY to only effect the Sort order of the result set?
Currently, ORDER BY works off of indexed properties, and missing values are not included in the result of a query using ORDER BY.
As a workaround, you could do two queries and combine the results:
The current query you're doing, with ORDER BY, returning all documents containing the Title property, ordered
A second query, returning all documents that don't have Title defined.
The second query would look something like:
SELECT * FROM c
WHERE NOT IS_DEFINED(c.Title)
Also note that, according to this note within the EF Core repo issue list, behavior is a bit different when using compound indexes (where documents with missing properties are returned).

Can I iterate through an object type document field when querying a collection in cloud firestore?

I have a collection "foo". Each document in the collection has a property/field "bar" that is an array of objects like so:
foo = [{
bar: [{
id: "random_string",
"status": "string"
}, {
id: "random_string",
"status": "string2"
}]
}, {
bar: [{
id: "random_string",
"status": "string"
}, {
id: "random_string",
"status": "string2"
}]
}]
What I want to achieve is I want to be able to query the db so that I can get two different collections, one with all the documents in the "foo" collection that have "string" as a value of at least one of the objects in the "bar" array, and another collection for all the documents that have "string2" as a value of at least one of the objects in the "bar" array.
Is that even possible? I'm struggling quite a lot to this one, so any help would be greatly appreciated. I'm also happy to change the db schema if needed, totally open to suggestions!
You can't do this with a single query because Firestore currently doesn't support logical OR conditions. In other words, you can't have a query that gives you all the documents where any one of a set of conditions is true.
Also, you need to be able to call out a particular field in a document in order to perform a query against it. Without a specific field to use, the query can't use an index to speed things up, and the query would never scale at the magnitude offered by Firestore.

Hot to improve a query based on a nested object

I'm using AWS DynamoDB to store data in JSON format. The Partition key is "device" and sort key is "timestamp". I can query the table for a specific device in a range of dates. I can then filter the content by the specific endpoint (in the nested "reports" object) the application is interested in.
{
"device": "AAA111",
"attr1": "bbb",
"reports": [
{
"endpoint": 1,
"value": "23"
},
{
"endpoint": 3,
"value": "26"
},
{
"endpoint": 4,
"value": "20"
}
],
.........
............
...........
"timestamp": "2017-11-30T03:50:30z"
}
The problem I have is if for example, I want to retrieve the latest value of an specific "endpoint". So, I can retrieve the latest record for a "device" based on the latest "timestamp", but it doesn't guarantee this record will contain value for this particular endpoint (not all records contains all endpoints). To solve this I have to basically scan the latest records (in descending order) and return the first object where the endpoint is found. Also, I don't know how many records I have to retrieve to find one...
I'm wondering if there is a better way of doing this... I tried with secondary indexes but this would require to duplicate the data, creating an object for each endpoint value (duplicating the common data). I would like to avoid this...
I would appreciate any hints on how to solve this issue.
Thanks
Gus

Freebase MQL query to get all info about a specific date

I'm wondering if it is possible to get all info about a specific date from Freebase.
I can easily retrieve info about a date giving a specific topic, for example, to grab all persons of interest who were born on a specific date:
[{
"type":"/people/person",
"limit":1000,
"sort":"name",
"name":null,
"guid":null,
"timestamp":null,
"/people/person/date_of_birth":"1955-02-24"
}]
Is it possible to grab all types? I'm after things like people born on that date (which I have), major events (start of a war, assassination of a person of interest, etc), and so on.
Essentially I want to match all fields that are dates and return the full information about that entry, regardless of type.
Reflection is what you need here:
[{
"/type/reflect/any_value": [{
"type": "/type/datetime",
"value": "1955-02-24",
"link": {
"source": {
"id": null
},
"master_property": null
}
}]
}]
A couple of notes on that: the MQL manual I've linked to is somewhat bitrotted in its details but is still the best documentation that exists on MQL. Secondly, there's what I'm pretty sure is in MQL bug if you use "*": null or more specifically "target_value": null in the link clause above which makes it ignore the outer value you specified... so don't do that :-)

Resources