I would like to 'upsert' a document in DynamoDB. That is, I would like to specify a key, and a set of field/value pairs. If no document exists with that key, I want one created with that key and the key/value pairs I specified. If a document exists with that key, I want the fields I specified to be set to the values specified (if those fields did not exist before, then they should be added). Any other, unspecified fields on the existing document should be left alone.
It seems I can do this pretty well with the UpdateItem call, when the field/value pairs I am setting are all top-level fields. If I have nested structures, UpdateItem will work to set the nested fields, as long as the structure exists. In other words, if my existing document has "foo": {}, then I can set "foo.bar": 42 successfully.
However, I don't seem to be able to set "foo.bar": 42 if there is no foo object already (like in the case where there is no document with the specified field at all, and my 'upsert' is behaving as an 'insert'.
I found a discussion on the AWS forums from a few years ago which seems to imply that what I want to do cannot be done, but I'm hoping this has changed recently, or maybe someone knows of a way to do it?
UpdateItem behaves like an "upsert" operation: The item is updated if it exists in the table, but if not, a new item is added (inserted).
http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/SQLtoNoSQL.UpdateData.html
That ("foo.bar": 42) can be achieved using the below query:
table.update_item(Key = {'Id' : id},
UpdateExpression = 'SET foo = :value1',
ExpressionAttributeValues = {':value1': {'bar' : 42}}
)
Hope this helps :)
I found this UpdateItem limitation (top level vs nested attributes) frustrating as well. Eventually I came across this answer and was able to work around the problem: https://stackoverflow.com/a/43136029/431296
It requires two UpdateItem calls (possibly more depending on level of nesting?). I only needed a single level, so this is how I did it:
Update the item using an attribute_exists condition to create the top level attribute as an empty map if it doesn't already exist. This will work if the entire item is missing or if it exists and has other pre-existing attributes you don't want to lose.
Then do the 2nd level update item to update the nested value. As long as the parent exists (ex: an empty map in my case) it works great.
I got the impression you weren't using python, but here's the python code to accomplish the upsert of a nested attribute in an item like this:
{
"partition_key": "key",
"top_level_attribute": {
"nested_attribute": "value"
}
}
python boto3 code:
def upsert_nested_item(self, partition_key, top_level_attribute_name, nested_attribute_name, nested_item_value):
try:
self.table.update_item(
Key={'partition_key': partition_key},
ExpressionAttributeNames={f'#{top_level_attribute_name}': top_level_attribute_name},
ExpressionAttributeValues={':empty': {}},
ConditionExpression=f'attribute_not_exists(#{top_level_attribute_name})',
UpdateExpression=f'SET #{top_level_attribute_name} = :empty',
)
except self.DYNAMODB.meta.client.exceptions.ConditionalCheckFailedException:
pass
self.table.update_item(
Key={'partition_key': partition_key},
ExpressionAttributeNames={
f'#{top_level_attribute_name}': top_level_attribute_name,
f'#{nested_attribute_name}': nested_attribute_name
},
ExpressionAttributeValues={f':{top_level_attribute_name}': nested_item_value},
UpdateExpression=f'SET #{top_level_attribute_name}.#{nested_attribute_name} = :{top_level_attribute_name}',
)
Related
(1) In my Canvas App I create a collection like this:
Collect(colShoppingBasket; {Category: varCategoryTitle ; Supplier: lblSupplier.Text ; ProductNumber: lblProductNumber.Text });;
It works - I get a collection. And whenever I push my "Add to shopping basket" button, an item are added to my collection.
(2) Now I want to sort the collection and then use the sorted output for other things.
This function sorts it by supplier. No problems here:
Sort(colShoppingBasket; Supplier)
(3) Then I want to display the SORTED version of the collection in various scenarios. And this is where the issue is. Because all I can do is manipulate a DISPLAY of the collection "colShoppingBasket" - (unsorted).
(4) What would be really nice would be the option to create and store a manipulated copy of the original collection. And the display that whereever I needed. Sort of:
Collect(colShoppingBasketSORTED; { Sort(colShoppingBasket; supplier) });; <--- p.s. I know this is not a working function
You could use the following:
ClearCollect(colShoppingBasketSorted, Sort(colShoppingBasket, Supplier))
Note that it is without the { }
This will Clear and Collect the whole colShoppingBasket sorted.
If you want to store the table in a single row in a collection, you can use
ClearCollect(colShoppingBasketSortedAlternative, {SingleRow: Sort(colShoppingBasket, Supplier)})
I wouldn't recommend this though because if you want to use the Sorted values you'd have to do something like this:
First(colShoppingBasketSortedAlternative).SingleRow -> this returns the first records of the colShoppingBasketSortedAlternative then gets the content of the SingleRow column, which in this case is a Collection
Note: You will need to replace the , with ; to work on your case
I have such structure in my database:
User 1
+- items
+- 1
| |- question: some question
| |- answer: some answer
|
+- 2
|- question: another question
I want to add answer to second item, but I don't have the id.
I'm trying to get items and it returns
[null, {answer: some answer, question: some question}, {question: another question}]
But I can't get needed item without answer field and I can't get their ids.
I tried to use equalTo, but it didn't help.
// attempt 1
Query child = _databaseReference.child(item._user).child(db_items).equalTo(null, key: db_answer);
child.reference().once().then((DataSnapshot snapshot) {
print('snapshot: ${snapshot.value}');
});
// attempt 2
Query child = _databaseReference.child(item._user).child(db_items).equalTo(null, key: db_answer);
child.reference().child(db_question).once().then((DataSnapshot snapshot) {
print('snapshot: ${snapshot.value}');
});
First attempt returns the same output as I wrote above, second returns null
So, does anybody know how can I add answer field to second question?
P.S. I use Flutter, but I don't think it means a lot for this issue.
So basically you are trying to add 'answer' to your second item.
I would have done it in this way:
First is to know the key beforehand, so that I can insert a child node in the second item. That way i can write something like:
myDBReference.child("items").child("key_known_beforehand").child("answer").setValue("this is the ansewr");
Or in case I didnt know the key, which seems like the problem here:
I would use Firebase Query. Something like..
myDBReference.child("items").orderByValue().startAt(starting_point_of_child_item).endAt(ending_point_of_child_item).child
This query would givve me the requred node and i would set value there.
"
Using startAt(), endAt(), and equalTo() allows you to choose arbitrary starting and ending points for your queries
To filter data, you can combine any of the limit or range methods with an order-by method when constructing a query.
Unlike the order-by methods, you can combine multiple limit or range functions. For example, you can combine the startAt() and endAt() methods to limit the results to a specified range of values.
"
There is a many to many relationship between two records namely countries and clients. When I fetch some records from the clients ( an array of clients ) and I try to assign them problematically to a country( record ) like this record[clientsRelationName] = clients I get the following bazaar error, TypeError: Cannot read property "data" from undefined. I know for sure that the variable clientsRelationName is actually a string that corresponds to the name of the relation which is simply just called clients. And it has nothing to do with a variable called data. In fact data does't exist. And I know for sure that record is a defined variable.
Any idea why this is happening? Is it a bug?
I have seen this issue where using Object.keys() on a server-side record yields [key, data, state] instead of the expected fields for that record. So if your programmatic assignment involves iterating on the properties of that record object, you may hit this data property.
Unfortunately that's all I know so far. Maybe the App Maker Team can provide further insight.
As you pointed out in your question, clientsRelationName is a string corresponding to the name of the relation. Your actual relation is just Clients, therefore either of the following should work:
record[Clients] = clients;
or
record.Clients = clients;
I would actually suggest using record.YourRelation because when you use the dot after your record the intellisense will automatically bring up all options for field names or relation end names that are available with that record.
After a lot of trail and error, I finally found a way to make it work using a rather very simple solution and its the only way I could make it work. Basically to a void getting this strange error when when modifying an association on a record TypeError: Cannot read property "data" from undefined , I did the following:
Loop through the record relation(array) and and popup every record in side it. Then loop through the other records that you want to assign the record relation to ( modify the association ) pushing every element to the record relation.
var length = record[relationNameAsVariable].length;
for(var i=0; i<length; i++){
record[relationNameAsVariable].pop();
}
now record[relationNameAsVariable] is empty so do the following:
for(var i=0; i < clientsArray.length; i++ ){
record[relationNameAsVariable].push(clientsArray[i]);
}
It could be a bug or something else that I'm doing wrong when trying to replace the whole association. I'm not sure. But this works like a champ.
I'm trying to generate unique id for label & input pairs.
After googling I now know that, unlike with handlebars, there is no array #index syntax extension in spacebars yet (also anybody knows why Blaze development has been inactive since the version 0.1 for past 5 months?).
So I ended up using the JS Array .map() solution inspired by this blog post and other posts. However, this solution returns label & input pairs of objects which DOM appears to be rendering the same on 'pagination' through Session.
Live example: http://meteorpad.com/pad/NXLtGXXD4yhYr9LHC
When clicking on first set of "Non-Indexed IDs" checkboxes, then next/previous, DOM will display new set of checkboxes correctly.
However clicking on the second set of "Indexed IDs" checkboxes below, then next/previous, DOM seems to retain the same checkboxes because one selected from the previous page remains checked on the next page.
What am I doing wrong or missing?
I also put the code on github for quick testing & refinement:
The solution, which I've found by looking at the ObserveSequence source, appears to be to give your generated objects a unique field called _id (generated like {{questionId}}:{{questionIndex}}:{{choiceIndex}}). See this meteorpad: http://meteorpad.com/pad/2EaLh8ZJncnqyejSr
I don't know enough about Meteor internals to say why, but this comment seems relevant:
// 'lastSeqArray' contains the previous value of the sequence
// we're observing. It is an array of objects with '_id' and
// 'item' fields. 'item' is the element in the array, or the
// document in the cursor.
//
// '_id' is whichever of the following is relevant, unless it has
// already appeared -- in which case it's randomly generated.
//
// * if 'item' is an object:
// * an '_id' field, if present
// * otherwise, the index in the array
//
// * if 'item' is a number or string, use that value
//
// XXX this can be generalized by allowing {{#each}} to accept a
// general 'key' argument which could be a function, a dotted
// field name, or the special #index value.
When the _id is absent, it uses the index in the array, so I guess ObserveSequence assumes it's the same object with changed fields, rather than a different object, so it re-uses the old elements rather than destroying them and recreating them. I suppose the name _id is chosen so that it works well with arrays generated by .fetch() on a Minimongo cursor.
I don't know if this is documented behaviour, or if it might change in the future.
I have a lot of objects with unique IDs. Every object can have several labels associated to it, like this:
123: ['a', 'hello']
456: ['dsajdaskldjs']
789: (no labels associated yet)
I'm not planning to store all objects in DynamoDB, only these sets of labels. So it would make sense to add labels like that:
find a record with (id = needed_id)
if there is one, and it has a set named label_set, add a label to this set
if there is no record with such id, or the existing record doesn't have an attribute named label_set, create a record and an attribute, and initialize the attribute with a set consisting of the label
if I used sets of numbers, I could use just ADD operation of UPDATE command. This command does exactly what I described. However, this does not work with sets of strings:
If no item matches the specified primary key:
ADD— Creates an item with supplied primary key and number (or set of numbers) for the attribute value. Not valid for a string type.
so I have to use a PUT operation with Expected set to {"label_set":{"Exists":false}}, followed (in case it fails) by an ADD operation. These are two operations, and it kinda sucks (since you pay per operation, the costs of this will be 2 times more than they could be).
This limitations seems really weird to me. Why are something what works with numbers sets would not work with string sets? Maybe I'm doing something wrong.
Using many records like (123, 'a'), (123, 'hello') instead of one record per object with a set is not a solutions: I want to get all the values from the set at once, without any scans.
I use string sets from the Java SDK the way you describe all the time and it works for me. Perhaps it has changed? I basically follow the pattern in this doc:
http://docs.amazonwebservices.com/amazondynamodb/latest/developerguide/API_UpdateItem.html
ADD— Only use the add action for numbers or if the target attribute is
a set (including string sets). ADD does not work if the target
attribute is a single string value or a scalar binary value. The
specified value is added to a numeric value (incrementing or
decrementing the existing numeric value) or added as an additional
value in a string set. If a set of values is specified, the values are
added to the existing set. For example if the original set is [1,2]
and supplied value is [3], then after the add operation the set is
[1,2,3], not [4,5]. An error occurs if an Add action is specified for
a set attribute and the attribute type specified does not match the
existing set type.
If you use ADD for an attribute that does not exist, the attribute and
its values are added to the item.
When your set is empty, it means the attribute isn't present. You can still ADD to it. In fact, a pattern that I've found useful is to simply ADD without even checking for the item. If it doesn't exist, it will create a new item using the specified key and create the attribute set with the value(s) I am adding. If the item exists but the attribute doesn't, it creates the attribute set and adds the value(s). If they both exist, it just adds the value(s).
The only piece that caught me up at first was that the value I had to add was a SS (String set) even if it was only one string value. From DynamoDB's perspective, you are always merging sets, even if the existing set is an empty set (missing) or the new set only contains one value.
IMO, from the way you've described your intent, you would be better off not specifying an existing condition at all. You are having to do two steps because you are enforcing two different situations but you are trying to perform the same action in both. So might as well just blindly add the label and let DynamoDB handle the rest.
Maybe you could: (pseudo code)
try:
add_with_update_item(hash_key=42, "label")
except:
element = new Element(hash_key=42, labels=["label"])
element.save()
With this graceful recovery approach, you need 1 call in the general case, 2 otherwise.
You are unable to use sets to do what you want because Dynamo Db doesn't support empty sets. I would suggest just using a string with a custom schema and building the set from that yourself.
To avoid two operations, you can add a "ConditionExpression" to your item.
For example, add this field/value to your item:
"ConditionExpression": "attribute_not_exists(RecordID) and attribute_not_exists(label_set)"
Source documentation.
Edit: I found a really good guide about how to use the conditional statements