I am running into an issue with the JavaScript v3 client for DynamoDB. According to the AWS documentation, I should be able to create an UpdateExpression which uses an ADD keyword to add an element to a list. When attempting this, it throws an error not recognizing the update expression.
If I run the same command using SET instead of ADD it works just fine, however, SET as noted in the documentation replaces the current value, and does not add to the list.
The Error:
"errorMessage": "Invalid UpdateExpression: Syntax error; token: \"=\", near: \"activeUsers = :au\"",
The Code:
const joinParams = {
TableName: tableName,
Key: {
id: myTable.Items[0].id
},
UpdateExpression: "SET activeUsers = :ac",
ExpressionAttributeValues: {
":ac": [user]
}
};
await docClient.update(joinParams).promise();
Take a look at the answer to a similar question here.
I think your syntax after the ADD is slightly off.
UpdateExpression: "ADD activeUsers :ac", <!-- remove the "="
Related
I'm using DynamoDB for my new Serverless Restful API with nodejs.
The Restful API supports query for resources with the limit and lastKey query parameters for key pagination.
Assume there's a table like below:
PK
SK
School
firstSchool
School
secondSchool
School
thirdSchool
PK is partition key, and SK is sort key.
I use SK for key pagination.
If I call the api with http://somewhere/api/school?limit=1&lastKey=secondSchool, ExclusiveStartKey in query will be {"PK" : "School", "SK" : "secondSchool"}, and the returned item will be {"PK" : "School", "SK" : "thirdSchool"}.
It works well in that case, but the problem is the same result is created with the url like http://somewhere/api/school?limit=1&lastKey=seco.
In this case, ExclusiveStartKey in query will be {"PK" : "School", "SK" : "seco"}
It seems DynamoDB doesn't use exact match for a sk value in ExclusiveStartKey.
Is there any way to force DynamoDB to use exact match for ExclusiveStartKey?
I attach my test code below:
const { DynamoDBClient } = require("#aws-sdk/client-dynamodb");
const { DynamoDBDocument } = require("#aws-sdk/lib-dynamodb");
const ddbClient = new DynamoDBClient({
region: AWS_REGION,
endpoint: AWS_DYNAMODB_END_POINT,
credentials: {
accessKeyId: AWS_ACCESSKEY_ID,
secretAccessKey: AWS_SECRET_ACCESS_KEY,
},
});
const ddbDocClient = DynamoDBDocument.from(ddbClient);
(async () => {
try {
const data = await ddbDocClient.query({
TableName: "Table Name",
KeyConditionExpression: "#pk = :pk",
ExpressionAttributeNames: {
"#pk": "PK",
},
ExpressionAttributeValues: {
":pk": "Test",
},
Limit: 1,
ExclusiveStartKey: { PK: "Test", SK: "Seco" },
});
console.log(data);
} catch (err) {
console.log("Error", err);
}
})();
The ExclusiveKeyStart is used mainly for paging large Scan or Query requests - i.e., retrieving the next page of results after the previous page ended with a LastEvaluatedKey, and you are supposed to give exactly that key (not some subset of it...) as the ExclusiveKeyStart of the next request.
You are trying to do something different, and to achieve you can't use ExclusiveKeyStart, but you can use something else:
The Query request has a KeyConditionExpression. You can specify sk > :value as a key condition expression (don't pass ExclusiveKeyStart), and you'll get this all the sort keys higher than that :value like your string "seco". Please note, however, that because your sort key is truncated, this result may actually include one or more extra results before the first key you want (e.g., the keys "seco" and "secoaaaa" come before "secondSchool") so you may need to drop them yourself from the results.
The KeyConditionExpression is implemented efficiently - DynamoDB knows how to skip directly to that sort key in the partition, and doesn't charge you for reading the entire partition, so in this respect it is just as good as ExclusiveKeyStart.
I'm using the Dynamodb resource in Retool, which is successful for GETs/Scans/Puts/Queries, but I can't seem to get an UpdateItem statement to work.
I'm trying to update an item to add a key for a list of maps if it doesn't exist and append an item if the key already does exist.
Configuration
Update Expression
SET images = list_append(:val, if_not_exists(images, :emptylist))
ExpressionAttributeValues
In Retool, my ExpressionAttributeValues are
":val": [{"location": "{{s3Uploader1.s3FolderName}}/{{s3Uploader1.s3FileName}}"}], ":emptylist":[], which pulls the s3 folder and file names from an s3Uploader and renders to ":val": [{"location": "redactedpath/redacted/redactedfilename"}], ":emptylist":[]
I originally tried the format of calling out the data types, e.g. "M", "L", etc, but I got exactly the same error.
":val":
{
"L":
[
{
"M":
{
"location":
{
"S": "{{s3Uploader1.s3FolderName}}/{{s3Uploader1.s3FileName}}"
}
}
}
]
},
":emptylist":
{
"L":[]
}
Result/Error
When I run the query, I get the following error:
statusCode:422
error:"Unprocessable Entity"
message:"ExpressionAttributeValues contains invalid key: Syntax error; key: "44""
data:null
estimatedResponseSizeBytes:147
resourceTimeTakenMs:363
isPreview:false
resourceType:"dynamodb"
lastReceivedFromResourceAt:1644774304601
source:"resource"
From my understanding, that error message usually specifies the actual key that caused the problem, but from what I can tell, my ExpressionAttributeValues does not contain the string 44. I'm wondering if this is something coming from Retool or if it's perhaps a location instead of the actual key.
I've dug through what feels like the depths of StackOverflow to try different things, but now I feel like I'm stuck.
Additional Information
My original ExpressionAttributeValues was based on Is it possible to combine if_not_exists and list_append in update_item
Similar question, but no answer and different key: ValidationException: ExpressionAttributeValues contains invalid key
Is there anything in the ExpressionAttributeValues that looks like it could cause that error?
I am trying to append to a string set (array of strings) column, which may or may not already exist, in a DynamoDB table. I referred to SO questions like this and this when writing my UpdateExpression.
My code looks like this.
const AWS = require('aws-sdk')
const dynamo = new AWS.DynamoDB.DocumentClient()
const updateParams = {
// The table definitely exists.
TableName: process.env.DYNAMO_TABLE_NAME,
Key: {
email: user.email
},
// The column may or may not exist, which is why I am combining list_append with if_not_exists.
UpdateExpression: 'SET #column = list_append(if_not_exists(#column, :empty_list), :vals)',
ExpressionAttributeNames: {
'#column': 'items'
},
ExpressionAttributeValues: {
':vals': ['test', 'test2'],
':empty_list': []
},
ReturnValues: 'UPDATED_NEW'
}
dynamo.update(updateParams).promise().catch((error) => {
console.log(`Error: ${error}`)
})
However, I am getting this error: ValidationException: An operand in the update expression has an incorrect data type. What am I doing incorrectly here?
[Update]
Thanks to Nadav Har'El's answer, I was able to make it work by amending the params to use the ADD operation instead of SET.
const updateParams = {
TableName: process.env.DYNAMO_TABLE_NAME,
Key: {
email: user.email
},
UpdateExpression: 'ADD items :vals',
ExpressionAttributeValues: {
':vals': dynamo.createSet(['test', 'test2'])
}
}
A list and a string set are not the same type - a string set can only hold strings while a list may hold any types (including nested lists and objects), element types don't need to be the same, and a list can hold also duplicate items. So if your original item is indeed as you said a string set, not a list, this explains why this operation cannot work.
To add items to a string set, use the ADD operation, not the SET operation. The parameter you will give to add should be a set (not a list, I don't know the magic js syntax to specify this, check your docs) with a bunch of elements. If the attribute already exists these elements will be added to it (dropping duplicates), and if the attribute doesn't already exit, it will be set to the set of these elements. See the documentation here: https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_UpdateItem.html#DDB-UpdateItem-request-UpdateExpression
I'm trying to update a user objects attribute which may not exist.
The attribute is called claimed which itself will have a property for each currency type #c1 (USD, euro, ect). A user can have multiple currencies but starts with none so claimed may or may not exist on a user object.
My initial draft was:
let params = {
TableName: 'myproject-user',
Key: {"id":req.user.sub},
UpdateExpression: 'set claimed.#c1 = :o',
ExpressionAttributeValues:{
":o": req.body.currency
},
ExpressionAttributeNames:{ "#c1": req.body.currency.type },
ReturnValues:"UPDATED_NEW"
};
This returns the error:
"Error: ValidationException: The document path provided in the update expression is invalid for update"
I've tried some variations using if_not_exists but I can't seem to get it working. How can I modify the params to have the desired effect?
you can use ConditionExpression: "attribute_not_exists(<YOUR_ATTRIBUTE>)"
for example:
let params = {
TableName: 'myproject-user',
Key: {"id":req.user.sub},
UpdateExpression: 'set claimed.#c1 = :o',
ExpressionAttributeValues:{
":o": req.body.currency
},
ExpressionAttributeNames:{ "#c1": req.body.currency.type },
ConditionExpression: "attribute_not_exists(claimed.#c1)",
ReturnValues:"UPDATED_NEW"
};
I am trying to do an upsert on the server side with a custom field as the unique identifier instead of a mongo id (data is being pulled from a 3rd party api).
A simplified version of what I am trying to achieve:
var myItem = {
myUniqueID : 'abc123',
name: 'foo'
};
MyCollection.upsert(
{
"myUniqueID ": myItem.myUniqueID
},
{
"$set": myItem
}
);
I receive the following error:
Error: Meteor does not currently support objects other than ObjectID as ids
It appears that this was caused by me adding:
MyCollection._ensureIndex({myUniqueID : 1}, {unique: 1});
right after declaring the Mongo Collection... even when I commented this line out the damage had been done.. I had to rename the collection (essentially create a new collection) to get past it.