Dynamoose model update with hash key - amazon-dynamodb

I'm trying to execute an update against a dynamoose model. Here's the docs on calling model.update
Model.update(key[, updateObj[, settings]],[ callback])
key can be a string representing the hashKey or an object containing the hashKey & rangeKey.
My schema has both a hash key (partition key) and range key (sort key) like this:
// create model
let model = dynamoose.model(
"SampleStatus",
{
id: {
type: String,
hashKey: true,
},
date: {
type: Date,
rangeKey: true,
},
status: String,
});
I've created an object like this (with a fixed timestamp for demoing)
let timestamp = 1606781220842; // Date.Now()
model.create({
id: "1",
date: new Date(timestamp),
status: "pending",
});
I'd like to be able to update the status property by referencing just the id property like this:
model.update({id: "1"}, {status: "completed"})
// err: The provided key element does not match the schema
model.update("1", {status: "completed"})
// err: Argument of type 'string' is not assignable to parameter of type 'ObjectType'
But both result in the shown errors:
I can pass in the full composite key if I know the timestamp, so the following will work:
let timestamp = 1606781220842; // Date.Now()
model.update({ id: "1", date: timestamp }, { status: "completed" });
However, that requires me holding onto the timestamp and persisting alongside the id.
The ID field, in my case, should, by itself, be unique, so I don't need both to create a key, but wanted to add the date as a range key so it was sortable. Should I just update my schema so there's only a single hash key? I was thinking the docs that said a "`key can be a string representing the hashkey" would let me just pass in the ID, but that throws an error on compile (in typescript).
Any suggestions?

The solution here is to remove the rangeKey from the date property.
This is because in DynamoDB every document/item must have a unique “key”. This can either be the hashKey or hashKey + rangeKey.
Since you mention that your id property is unique, you probably want to use just the hashKey as the key, which should fix the issue.
In your example there could have been many documents with that id, so DynamoDB wouldn’t know which to update.
Don’t forget that this causes changes to your table so you might have to delete and recreate the table. But that should fix the problem you are running into.

Logically there is nothing stopping you than inserting more than 1 entry into the same partition (in your case the unique id). You could insert more than one item with the same id, if it had a different date.
Therefore if you want to get an item by only its partition key, which is really a unique ID, you need to use a query to retrieve the item (as opposed to a GET), but the return signature will be a collection of items. As you know you only have one item in the partition, you can take the first item, and specify a limit of 1 to save RCU.

// create model
let model = dynamoose.model(
"SampleStatus",
{
id: {
type: String,
hashKey: true,
"index": {
"name": "index_name",
"rangeKey": "date",
}
},
date: {
type: Date
},
status: String,
});
You have to tell the schema that hashKey and range are one partition key.
Ref: https://dynamoosejs.com/guide/Schema#index-boolean--object--array

Related

ConditionExpression for PutItem not evaluating to false

I am trying to guarantee uniqueness in my DynamoDB table, across the partition key and other attributes (but not the sort key). Something is wrong with my ConditionExpression, because it is evaluating to true and the same values are getting inserted, leading to data duplication.
Here is my table design:
email: partition key (String)
id: sort key (Number)
firstName (String)
lastName (String)
Note: The id (sort key) holds randomly generated unique number. I know... this looks like a bad design, but that is the use case I have to support.
Here is the NodeJS code with PutItem:
const dynamodb = new AWS.DynamoDB({apiVersion: '2012-08-10'})
const params = {
TableName: <table-name>,
Item: {
"email": { "S": "<email>" },
"id": { "N": "<someUniqueRandomNumber>" },
"firstName": { "S": "<firstName>" },
"lastName": { "S": "<lastName>" }
},
ConditionExpression: "attribute_not_exists(email) AND attribute_not_exists(firstName) AND attribute_not_exists(lastName)"
}
dynamodb.putItem(params, function(err, data) {
if (err) {
console.error("Put failed")
}
else {
console.log("Put succeeded")
}
})
The documentation https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Expressions.OperatorsAndFunctions.html says the following:
attribute_not_exists (path)
True if the attribute specified by path does not exist in the item.
Example: Check whether an item has a Manufacturer attribute.
attribute_not_exists (Manufacturer)
it specifically says "item" not "items" or "any item", so I think it really means that it checks only the item being overwritten. As you have a random sort key, it will always create a new item and the condition will be always true.
Any implementation which would check against a column which is not an index and would test all the records would cause a scan of all items and that is something what would not perform very well.
Here is an interesting article which covers how to deal with unique attributes in dynamodb https://advancedweb.hu/how-to-properly-implement-unique-constraints-in-dynamodb/ - the single table design together with transactions would be a possible solution for you if you can allow the additional partition keys in your table. Any other solution may be challenging under your current schema. DynamoDB has its own way of doing things and it may be frustrating to try to push to do things which it is not designed for.

DyamoDB put item with ConditionExpression instead of key

Since email is not the primary key, I need to check the uniqueness of a record based on the email field. It does not work. The user gets saved. Does DynamoDB not allow conditionExpression on another field instead of a key?
const params = {
TableName: process.env.tableName,
Item: user.toItem(),
ConditionExpression: "#email <> :email",
ExpressionAttributeNames: {
"#email": "email",
},
ExpressionAttributeValues: {
":email": body.email,
},
};
await docClient.put(params).promise();
The condition is valid, but what conditional puts prevent is overwriting records with the same primary key:
The PutItem operation overwrites an item with the same key (if it exists). If you want to avoid this, use a condition expression. This allows the write to proceed only if the item in question does not already have the same key.
To prevent duplicate emails, make it part of your table's primary key or manually check for uniquenes before writing to DynamoDB.

DynamoDB - Get Item by Global Secondary Index

I have an existing table which has 10 fields. Fields are like this:
AuthID, UserID, Age, Job, .etc
The table stores data of my users. "AuthID" is primary key and "UserID" is a Global Secondary Index.
When I get item by AuthID, everything is fine. But I can't get item by UserID. I tried GetItem, Query and Scan methods but I failed in all three method.
I need to be able to get data with these 3 methods I wrote below :
1 - Get user data by AuthID (It's already works fine)
2 - Get user data by UserID
3 - Get user data by AuthID and UserID both
AuthID and UserID is unique. Can someone point me right way as to what to do?
I have searched a lot in the documentation and found that if you need to get a single item even then you cant use the get or getItem method when using a global secondary index. One can use the query method. a sample of query method with global secondary index is
let params = {
TableName: "Users",
IndexName: "your-index",
ExpressionAttributeValues: {
":v1": "myid"
},
KeyConditionExpression: "my_partition_key_in_gsi = :v1",
};
dynamodb.query(params, function(err, data) {
if (err) console.log(err, err.stack); // an error occurred
else console.log(data); // successful response
});

dynamo db FilterExpression, find in json object using value as key

It is possible to somehow filter results by key name that stored in the same object?
I have JSON object "keys", in property "default" stored key of the object that I need. Is it somehow possible to filter like that keys[keys.default].type = some_type?
var params = {
TableName: 'TABLE_NAME',
IndexName: 'TABLE_INDEX', // optional (if querying an index)
KeyConditionExpression: 'myId = :value',
FilterExpression: '#kmap[#kmap.#def].#tp = :keyval',
ExpressionAttributeNames: {names with special characters
'#kmap': 'keys',
'#tp': 'type',
'#def': 'default'
},
ExpressionAttributeValues: { // a map of substitutions for all attribute values
':value': '1',
':keyval': 'some_type'
},
Limit: 10, // optional (limit the number of items to evaluate)
ProjectionExpression: "displayName, #kmap",
ReturnConsumedCapacity: 'TOTLAL', // optional (NONE | TOTAL | INDEXES)
};
docClient.query(params, function(err, data) {
if (err) ppJson(err); // an error occurred
else ppJson(data); // successful response
});
I'm pretty sure the answer is no.
This keys[keys.default] is not even valid json, as far as I can tell.
Of course, you can do this in two steps:
First, query to get the default key
Then query to get the value
Don't forget, filters are obly applied to the result set - it still requires a libear traversal as specified by your Query or Scan operation.
So you can probably more easily run your query on the client.
And lastly, if this is a typical query ypu need to perform, as an optimization, you can lift the default key and value to be top level attributes on the item. Thrn you can actually create a GSI on that attribure and can actually do efficient lookups.

DynamoDB Descending Order fetch records

i have 100 records in collection,
collection name:'users'
{
"name":'senthilkumar',
"email":'senthily88#gmail.com', //HashKey
"age":21,
"created":1465733486137, //RangeKey-timestamp
}
i need to fetch records the following sql query wise
select * from users order by created desc limit 10
How i can get above query format records from DynamoDB
Dynamodb sorts the results by the range key attribute. You can set the ScanIndexForward boolean parameter to true for ascending or false for descending.
resource: http://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Query.html
Use the KeyConditionExpression parameter to provide a specific value
for the partition key. The Query operation will return all of the
items from the table or index with that partition key value. You can
optionally narrow the scope of the Query operation by specifying a
sort key value and a comparison operator in KeyConditionExpression.
You can use the ScanIndexForward parameter to get results in forward
or reverse order, by sort key.
To Save Json Data to DynamoDB us put()
var Newparams = {
TableName: this.SuffleTableName,
Item: {
"userId": /* YOUR PRIMARY KEY */,
"addedAt": /* YOUR SORT KEY */,
"status": /* Additional Datas */,
}
}
Fetch Data From DynamoDB using Query()
QueryParam = {
TableName: 'YOUR TABLE NAME HERE',
IndexName: 'YOUR INDEX NAME HERE', //IF YOUR CREATED NEW INDEX
KeyConditionExpression: "UserId = :UserId ", //YOUR PRIMARY KEY
ExpressionAttributeValues: {
":UserId": UserId,
},
ScanIndexForward: false, //DESC ORDER, Set 'true' if u want asc order
ExclusiveStartKey: LastEvalVal, //Pagination - LastEvaluatedKeyPair
Limit: 10 //DataPerReq
}
If you want to return all rows in your table, you cannot use the query API, because that API requires you to provide a partition key value to filter your results by (i.e. assuming that your partition key is name you would only be able to use the query API to bring back the subset of results that have name = a given value, i.e. name= senthilkumar
If you want to return all rows in your table, you must use the Scan API: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/SQLtoNoSQL.ReadData.Scan.html
Note that all results will be provided in ascending order by the value of the Range Key. You cannot reverse sort the contents with the Scan API. You would need to reverse your resultset in the application tier using whatever language you're writing your code in to turn the results upside down.
Scan does not scale well and it is not possible to use Scan to create a paginated, reverse sorted solution if your table contains items with unique partition keys.
If this is your situation, and if you want to return paginated + reverse sorted sets back from DynamoDB, you will need to re-consider the design of your table and which columns are the partition key/range key/index so that you can use the Query API.

Resources