I'm using AWS DynamoDB to store data in JSON format. The Partition key is "device" and sort key is "timestamp". I can query the table for a specific device in a range of dates. I can then filter the content by the specific endpoint (in the nested "reports" object) the application is interested in.
{
"device": "AAA111",
"attr1": "bbb",
"reports": [
{
"endpoint": 1,
"value": "23"
},
{
"endpoint": 3,
"value": "26"
},
{
"endpoint": 4,
"value": "20"
}
],
.........
............
...........
"timestamp": "2017-11-30T03:50:30z"
}
The problem I have is if for example, I want to retrieve the latest value of an specific "endpoint". So, I can retrieve the latest record for a "device" based on the latest "timestamp", but it doesn't guarantee this record will contain value for this particular endpoint (not all records contains all endpoints). To solve this I have to basically scan the latest records (in descending order) and return the first object where the endpoint is found. Also, I don't know how many records I have to retrieve to find one...
I'm wondering if there is a better way of doing this... I tried with secondary indexes but this would require to duplicate the data, creating an object for each endpoint value (duplicating the common data). I would like to avoid this...
I would appreciate any hints on how to solve this issue.
Thanks
Gus
looking for an answer to this question if possible, not looking for a refactoring advice or redesign, i just need to understand what else I am missing here :
I have an existing item in dynamodb:
{
"CartId": 321,
"UserId": usr555,
"CartItems": [
{
"ProductId":59999,
"Quantity": 1
},
{
"ProductId": 58888,
"Quantity": 2
}
]
}
in my code I want to insert another nested attribute into the array CartItems in the item above. i can't find a way of doing this without retrieving the entire item and then inserting, which could amount to KBs in size.
all I want to do is insert a single array element in an existing nested attribute without having to retrieve the item.
the language I am using is nodejs and the function is DynamoDB.put.
UpdateExpression attribute supports SET action and SET action supports list_append function. Since list_append function operands must be list, enclose the new value within a list as below.
response = dynamodb_client.update_item(TableName='your_table',
Key={'CartId':'321'},
UpdateExpression='SET CartItems = list_append(CartItems, :val1)',
ExpressionAttributeValues = {':val1':[{'ProductId':12345, 'Quantity': 100}]})
Hope this helps
I have a Players collection and a Games collection. I want to construct a data structure that looks as follows:
{
"_id": "1234",
"accounts": {
"battlenet": "blah#1234"
},
"games": {
"overwatch": {
"class": "Hanzo",
"timePlayed": ISODate
},
"world-of-warcraft": {
"class": "Shaman",
"timePlayed": ISODate
}
}
}
games is an object, where every key refers to a specific document in the Games collection's slug attribute. Every value is a sub-schema definition with autoValues.
I can't find any good way to create validation in such a way that it updates an autoform correctly without weird coersion of data. Is there any way to accomplish this validation with simple schema?
I have a little problem, maybe can you help me. I'm trying to get the "currency features" from freebase. So I tried to do : "/base/schemastaging/person_extra/net_worth": null but, I can't get the value written on freebase (for example, with Madonna, it's 650,000,000). Do you know, why it's not working ?
First of all, as the property path suggests, /base/schemastaging/person_extra/net_worth is just being staged right now so the final property ID will be something else (follow the mailing list to discuss new schema). You should NOT be using this property for anything other than experimentation.
The reason why you don't see the data that you want withe the following query is because this property is a CVT.
{
"id": "/en/madonna",
"type": "/base/schemastaging/person_extra",
"net_worth": null
}
CVT values are complex objects that need to be expanded to access the values that you want. In this case, net_worth is a CVT so that we can record a person's net worth at different points in time.
If you expand your query to include the relevant properties from /measurement_unit/dated_money_value you'll see the data that you're after.
{
"id": "/en/madonna",
"type": "/base/schemastaging/person_extra",
"net_worth": {
"amount": null,
"currency": null,
"valid_date": null
}
}
One other issue, that isn't obvious from this example, is that since there can be multiple dated money values, you'll need to make your query more precise so as to get only the latest value. You can do that like this;
{
"id": "/en/madonna",
"type": "/base/schemastaging/person_extra",
"net_worth": {
"amount": null,
"currency": null,
"valid_date": null,
"sort": "-valid_date",
"limit": 1,
"optional": true
}
}
Update: Made net worth an optional property value.
I tried Googling and searching everywhere, but couldn't find a definitive authority on this topic. While being true to REST principles, how should I design the HTTP interface for:
An ordered list (get, add, insert into position, reorder, remove)
A set (get, add, remove)
A hash-table (get, add, remove)
NOTE: These data structures are to contain references to existing resources with known ids
That's how I would do it for an ordered list and hash table. I guess the methods would be the same for a set and a list:
Ordered list
Get item 123:
GET /list/123
Append an item to the list:
POST /list/
Insert new item into position 5:
POST /list/?position=5
Move item 123 to position 3:
PUT /list/123?position=3
Delete item 123:
DELETE /list/123
Delete item at position 3:
DELETE /list/?position=3
Of course, your API should update the indexes of all the elements when doing insertion and deletion.
Hash table
Get item "somekey":
GET /hashtable/somekey
Add item "somekey":
POST /hashtable/somekey
Remove item "somekey":
DELETE /hashtable/somekey
#dadads
You can not define such interface directly.
An ordered list (get, add, insert into position, reorder, remove)
By excluding "insert into position" and "reorder" you can perfectly implement "get", "add" and "remove" for example:
You define your resource /service/users
You can use POST /service/users to add new user to the "users" collection
You can GET /service/users to retrieve users
You can GET /service/users/user-id to retrieve particular user
You can DELETE /service/users/user-id from users collection
This is a very rough example, though it outlines some ideas. In order to achieve "reorder" and "insert into position" you need to implement your own action semantics which you can include in your resource representation and let client know HOW to perform these operations. As a reference you can see this JSON PATCH specification proposal: https://www.rfc-editor.org/rfc/rfc6902 which tries to describe such operations.
It is not necessary to use already existing media format, you can define your own under your own namespace for example: application/vnd.your-company.format-name+json which describes these capabilities and also advertises this information to clients.
You should decouple the transport mechanism from the underlying application. I would consider designing the application correctly, then figure out how to access it via HTTP. This way you could easily add or change the transport mechanisms (SOAP, SCA, etc) without affecting the underlying application.
Once you have the application correctly designed, consider accessing it from the HTTP requests via something like an Adapter or Visitor pattern.
This is my idea for reordering.
There is a HTTP method called PATCH that is used to update fragments of a resource. Give your resource a new property called index, then make a call with PATCH method
PATCH /collection
[
{
"id: "original index 0"
"index": 1
}
{
"id: "original index 1"
"index": 0
}
]
Then your server back-end needs to figure out how to do this atomically. But interface-wise, I think this is the best way to stay true to RESTful.
Alternatively, there is a better solution, but it may not apply to everyone's case. Since ordering always depends some sort of criteria, it can even be as simple as insertion order. Let your collection url support an orderBy query string, and let this orderBy dictate on how the result gets ordered. Then during your reordering call from client, just update the resource's property used for the ordering criteria.
I came to this question mostly looking for a RESTful way to reorder. I don't really like any of the answers, so here is what I think is most RESTful.
For reorder you could make the order a resource:
/list/order
Then you can do normal operations on it (for these examples assume a list with 5 items currently in it):
"items":" [
{
"id": "A",
"name": "Monkey"
},
{
"id": "B",
"name": "Cow"
},
{
"id": "C",
"name": "Horse"
},
{
"id": "D",
"name": "Turkey"
},
{
"id": "E",
"name": "Tasmanian Devil"
},
]
Note that "order" is not included in the resource response. It's not needed - the order is implicitly specified by the response order of the items.
GET /list/order
returns a list of item ids in their correct order
['A','B','C','D','E']
POST /list/order
with payload ['D','B','C','A','E']
GET /list/order
returns a list of item ids in their correct order
['D','B','C','A','E']
Also obviously you would return the items in the list in the correct order when you do a GET on /list.
GET /list
returns a list of items in their correct order
"items":" [
{
"id": "D",
"name": "Turkey"
},
{
"id": "B",
"name": "Cow"
},
{
"id": "C",
"name": "Horse"
},
{
"id": "A",
"name": "Monkey"
},
{
"id": "E",
"name": "Tasmanian Devil"
},
]