Consider the following document item / syntax in a DynamoDB table:
{
"id": "0f00b15e-83ee-4340-99ea-6cb890830d96",
"name": "region-1",
"controllers": [
{
"id": "93014cf0-bb05-4fbb-9466-d56ff51b1d22",
"routes": [
{
"direction": "N",
"cars": 0,
"sensors": [
{
"id": "e82c45a3-d356-41e4-977e-f7ec947aad46",
"light": true,
},
{
"id": "78a6883e-1ced-4727-9c94-2154e0eb6139",
}
]
}
]
}
]
}
My goal is to update a single attribute in this JSON representation, in this case cars.
My approach
I know all the sensors IDs. So, the easiest way to reach that attribute is to find, in the array, the route which has a sensor with any of the ids. Having found that sensor, Dynamo should know which object in the routes array he has to update. However, I cannot run this code without my condition being rejected.
In this case, update attribute cars, where the route has a sensor with id e82c45a3-d356-41e4-977e-f7ec947aad46 or 78a6883e-1ced-4727-9c94-2154e0eb6139.
var params = {
TableName: table,
Key:{
"id": "0f00b15e-83ee-4340-99ea-6cb890830d96",
"name": "region-1"
},
UpdateExpression: "set controllers.intersections.routes.cars = :c",
ConditionExpression: ""controllers.intersections.routes.sensors.id = :s",
ExpressionAttributeValues:{
":c": 1,
":s": "e82c45a3-d356-41e4-977e-f7ec947aad46"
},
ReturnValues:"UPDATED_NEW"
};
docClient.update(params, ...);
How can I achieve this?
Unfortunately, you can't achieve this in DynamoDB without knowing the array index. You have very complex nested structure. The DynamoDB API doesn't have a feature to handle this scenario.
I think you need the array index for controllers, routes and sensors to get the update to work.
Your approach may work in other databases like MongoDB. However, it wouldn't work on DynamoDB. Generally, it is not recommended to have this complex structure in DynamoDB especially if your use case has update scenario.
TableName : 'tablename',
Key : { id: id},
ReturnValues : 'ALL_NEW',
UpdateExpression : 'set someitem['+`index`+'].somevalue = :reply_content',
ExpressionAttributeValues : { ':reply_content' : updateddata }
For updating nested array element need to fing out array index . Then you can update nested array element in dynamo db.
Related
I am trying to guarantee uniqueness in my DynamoDB table, across the partition key and other attributes (but not the sort key). Something is wrong with my ConditionExpression, because it is evaluating to true and the same values are getting inserted, leading to data duplication.
Here is my table design:
email: partition key (String)
id: sort key (Number)
firstName (String)
lastName (String)
Note: The id (sort key) holds randomly generated unique number. I know... this looks like a bad design, but that is the use case I have to support.
Here is the NodeJS code with PutItem:
const dynamodb = new AWS.DynamoDB({apiVersion: '2012-08-10'})
const params = {
TableName: <table-name>,
Item: {
"email": { "S": "<email>" },
"id": { "N": "<someUniqueRandomNumber>" },
"firstName": { "S": "<firstName>" },
"lastName": { "S": "<lastName>" }
},
ConditionExpression: "attribute_not_exists(email) AND attribute_not_exists(firstName) AND attribute_not_exists(lastName)"
}
dynamodb.putItem(params, function(err, data) {
if (err) {
console.error("Put failed")
}
else {
console.log("Put succeeded")
}
})
The documentation https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Expressions.OperatorsAndFunctions.html says the following:
attribute_not_exists (path)
True if the attribute specified by path does not exist in the item.
Example: Check whether an item has a Manufacturer attribute.
attribute_not_exists (Manufacturer)
it specifically says "item" not "items" or "any item", so I think it really means that it checks only the item being overwritten. As you have a random sort key, it will always create a new item and the condition will be always true.
Any implementation which would check against a column which is not an index and would test all the records would cause a scan of all items and that is something what would not perform very well.
Here is an interesting article which covers how to deal with unique attributes in dynamodb https://advancedweb.hu/how-to-properly-implement-unique-constraints-in-dynamodb/ - the single table design together with transactions would be a possible solution for you if you can allow the additional partition keys in your table. Any other solution may be challenging under your current schema. DynamoDB has its own way of doing things and it may be frustrating to try to push to do things which it is not designed for.
I am trying to get specific object value from items array. I am updating array by pushing new items into items array and then I want to get last updated item from an array. I have _id of the item that is last updated and using this query but it's returning complete record with all items.
let repData = Items.findOne({'_id': rid, 'items._id': newRepId },{ "items.$": 1 });
{
"errorCode": false,
"data": {
"_id": "NhNpaN8EHn6uJXfg5",
"cText": "Let me know you views",
"aId": "YxEjFqsE3czZZJvgP",
"cId": "EwDS5iYSEuGThHE38",
"datePosted": "2019-07-11T07:13:59.177Z",
"items": [
{
"reText": "pppppp sssssssss",
"reOwn": "FK7KQ7eMs7QvX5uHh",
"datePosted": "2019-07-11T07:19:17.722Z",
"_id": "b87e532807ce37ff83d37a09"
},
{
"reText": "bbbbbb vvvvvvvv",
"reOwn": "FK7KQ7eMs7QvX5uHh",
"datePosted": "2019-07-11T07:22:36.089Z",
"_id": "28d2bf66a517bfcfaa0fabca"
},
{
"reText": "mmmmmnnnnnn bbbbbbvvvvv",
"reOwn": "FK7KQ7eMs7QvX5uHh",
"datePosted": "2019-07-11T07:23:20.587Z",
"_id": "85f52e3e3a8ae6d18e98dbb2"
},
}
}
The Mongo search will always get you the complete record, you can restrict the elements that you fetch, but you can't filter the items array, you will have to do that afterwards, something like this:
const lastItem = repData.items
.sort((a, b) => new Date(a.datePosted)-new Date(b.datePosted))
.pop()
console.log(lastItem)
Basically it sorts the array by date (perhaps not necessary), and then pops the last element into lastItem
Referring to the default sample schema mentioned in https://hasura.io/hub/project/hasura/hello-world/data-apis i.e. to the following two tables:
1) author: id,name
2) article: id, title, content, rating, author_id
where article:author_id has an array relationship to author:id.
How do I make a query to select authors who have written at least one article? Basically, something like select author where len(author.articles) > 0
TL;DR:
There's no length function that you can use in the Hasura data API syntax right now. Workaround 1) filter on a property that is guaranteed to be true for every row. Like id > 0. 2) Build a view and expose APIs on your view.
Option 1:
Use an 'always true' attribute as a filter.
{
"type": "select",
"args": {
"table": "author",
"columns": [
"*"
],
"where": {
"articles": {
"id": {
"$gt": "0"
}
}
}
}
}
This reads as: select all authors where ANY article has id > 0
This works because id is an auto-incrementing int.
Option 2:
Create a view and then expose data APIs on them.
Head to the Run SQL window in the API console and run a migration:
CREATE VIEW author_article_count as (
SELECT au.*, ar.no_articles
FROM
author au,
(SELECT author_id, COUNT(*) no_articles FROM article GROUP BY author_id) ar
WHERE
au.id = ar.author_id)
Make sure you mark this as a migration (a checkbox below the RunSQL window) so that this gets added to your migrations folder.
Now add data APIs to the view, by hitting "Track table" on the API console's schema page.
Now you can make select queries using no_articles as the length attribute:
{
"type": "select",
"args": {
"table": "author_article_count",
"columns": [
"*"
],
"where": {
"no_articles": {
"$gt": "0"
}
}
}
}
Let's say I have a DocumentDB collection populated with documents that have this shape:
[{ "Name": "KT", "Dob": "5/25/1990", "Children": [], "IsMale": false },
{ "Name": "Chris", "Dob": "10/1/1980", "Children": [], "IsMale": true }]
Now let's say I don't the structure of the documents above.
Is there a query I can write that will return me a distinct list of those property names ("Name", "Dob", "Children", "IsMale")?
In other words, is there a way for be to sniff out the schema of those documents?
This might be a duplicate of this question. In any case, the answers there might give you some ideas.
tl;dr; The only way to do it is to read all of the docs. You can pull them all back to your machine or you can read them inside of a stored procedure and only send the calculated schema back to your machine.
You need a dynamic ORM or ODM for Azure DocumentDB like Slazure to do something like this. Example follows:
using SysSurge.Slazure.AzureDocumentDB.Linq;
using SysSurge.Slazure.Core;
using SysSurge.Slazure.Core.Linq.QueryParser;
public void EnumProperties()
{
// Get a reference to the collection
dynamic storage = new QueryableStorage<DynDocument>("URL=https://contoso.documents.azure.com:443/;DBID=DDBExample;TOKEN=VZ+qKPAkl9TtX==");
QueryableCollection<DynDocument> collection = storage.TestCustomers;
// Build collection query
var queryResult = collection.Where("SignedUpForNewsletter = true and Age < 22");
foreach (DynDocument document in queryResult)
{
foreach (KeyValuePair<string, IDynProperty> keyValuePair in document)
{
Console.WriteLine(keyValuePair.Key);
}
}
}
I'm successfully able to GET data from
GET /mycollection?ql=select data.visitor.badges where data.visitor._id = 'f33498'
Which returns
{
"action": "get",
"application": "313hhlkhj77080",
"params": {
"ql": [
"select data.visitor.badges where data.visitor._id = 'f33498'"
]
},
"path": "/mycollection",
"uri": "http://xxxx/appservices/xxxxxx/mycollection",
"list": [
[
[
"New Visitor",
"Cart Abandoner"
]
],
[
[
"New Visitor",
"Repeat Visitors",
"Cart Abandoner"
]
],
[
[
"New Visitor",
"Repeat Visitors",
"Browse Abandoner"
]
]
],
"timestamp": 1407968065207,
"duration": 35,
"organization": "visitor-baas",
"applicationName": "sandbox",
"count": 3
}
However, I cannot figure out how to modify the following query to allow me to narrow the result set to only those containing a "Cart Abandoner" value in the data.user.badges array.
Is this possible? I've tried:
GET /mycollection?ql=select data.visitor.badges where data.visitor.badges = 'Cart Abandoner'
This appears to return data.visitor.badges arrays where 'Cart Abandoner' is the last position of the array.
GET /mycollection?ql=select data.visitor.badges where data.visitor.badges contains 'Cart Abandoner'
This appears to return nothing.
What am I missing?
Unfortunately there's currently no way to query arrays. Your best option is to store it as an object instead.
Couple things: I query elements of arrays all the time, but the ql query string is a little temperamental. element = 'string' should return the entire JSON payload if the 'string' is contained anywhere in the array 'element' so the fact you're getting mixed results may be due to the complexity of your nested arrays.
That said, The ql query string allows you to restrict the resources that get returned (like your first example where id = 'xxx'). There isn't any way to return anything other than the entire JSON payload from that resource (such as truncating your array based on the query restriction).
So, if what you're trying to do is pull just the times that your customer returned, I would suggest creating a separate resource called something like "visitorbadges" and connect it to the user record. So instead of querying with the id and trying t query the array you'd have something like:
https://api.usergird.org/{yourorg}/{yourapp}/users/{userid}/vistorbadges
If you use the BaaS userid rather than your own you can go to /users/uuid or, you could also store the userid with the label 'name' ({"name" : "f33498"}) which will let you go to /users/f33498/visitorbadges
See the Apige docs for how to connect resources:
http://apigee.com/docs/app-services/content/connecting-users-other-data