How to update only part of the JSON? - amazon-dynamodb

I have a DynamoDB table:
+--------------------------------------------------+
| Customer ID (Primary Key)|Gamestats (JSON entry) |
+--------------------------------------------------+
JSON:
{
"Gamestats": [
{
"ID": "QuickShootingMode",
"status": 1
},
{
"ID": "FastReloadMode", // Just want to update this and not update the entire JSON
"status": 0
}
],
"CustomerID": "xyz"
}
I want to update only parts of the JSON. What is the best way to do it? Eg, update the QuickShootingMode to be false.
One way is to make a call and fetch the JSON and then Iterate the JSON and update the value and then put the new JSON back in dynamo DB. It means it would make 2 calls
A) to get the data and
B) to put the data in DB.
Is there a better way by which I could directly update the data and avoid making these extra network calls? I could convert each key of the JSON to be a column in dynamo BD, but if the number of keys grows then I’ll end up having lots of column (which might be a bad design), hence I think having the JSON saved in one column Game stats would make more sense.
Map<String, AttributeValue> key = new HashMap<>();
AmazonDynamoDB dynamoDB = dynamoDBClient.getDynamoDB();
key.put(USER_ID_KEY, new AttributeValue().withS("xyz"));
key.put("Gamedata", new AttributeValue().withS("some JSON"));
PutItemRequest request = new PutItemRequest()
.withTableName(table)
.withItem(key);
PutItemResult result = dynamoDB.putItem(request);
Is there a better way to achieve what I want?

It looks like from your question you are storing stringified JSON. If so an update won't help you, but as far as I can tell there is no value in storing stringified JSON instead of using dynamodb maps and lists.
You can use an update to set a nested attribute in a map or a list. Using a map instead of a list for the gamestats attribute is better because then you don't have to worry about the order of the attributes.
Javascript example with Gamestats being a map.
dynamodb.update({
TableName: table,
Key: key,
UpdateExpression: 'SET #gs.#qs.#status = :newStatus',
ExpressionAttributeNames: {'#gs': 'Gamestats', '#qs': 'QuickShootingMode', '#status': 'status' },
ExpressionAttributeValues: { ':newStatus': false }
}, callback)

Related

ConditionExpression for PutItem not evaluating to false

I am trying to guarantee uniqueness in my DynamoDB table, across the partition key and other attributes (but not the sort key). Something is wrong with my ConditionExpression, because it is evaluating to true and the same values are getting inserted, leading to data duplication.
Here is my table design:
email: partition key (String)
id: sort key (Number)
firstName (String)
lastName (String)
Note: The id (sort key) holds randomly generated unique number. I know... this looks like a bad design, but that is the use case I have to support.
Here is the NodeJS code with PutItem:
const dynamodb = new AWS.DynamoDB({apiVersion: '2012-08-10'})
const params = {
TableName: <table-name>,
Item: {
"email": { "S": "<email>" },
"id": { "N": "<someUniqueRandomNumber>" },
"firstName": { "S": "<firstName>" },
"lastName": { "S": "<lastName>" }
},
ConditionExpression: "attribute_not_exists(email) AND attribute_not_exists(firstName) AND attribute_not_exists(lastName)"
}
dynamodb.putItem(params, function(err, data) {
if (err) {
console.error("Put failed")
}
else {
console.log("Put succeeded")
}
})
The documentation https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Expressions.OperatorsAndFunctions.html says the following:
attribute_not_exists (path)
True if the attribute specified by path does not exist in the item.
Example: Check whether an item has a Manufacturer attribute.
attribute_not_exists (Manufacturer)
it specifically says "item" not "items" or "any item", so I think it really means that it checks only the item being overwritten. As you have a random sort key, it will always create a new item and the condition will be always true.
Any implementation which would check against a column which is not an index and would test all the records would cause a scan of all items and that is something what would not perform very well.
Here is an interesting article which covers how to deal with unique attributes in dynamodb https://advancedweb.hu/how-to-properly-implement-unique-constraints-in-dynamodb/ - the single table design together with transactions would be a possible solution for you if you can allow the additional partition keys in your table. Any other solution may be challenging under your current schema. DynamoDB has its own way of doing things and it may be frustrating to try to push to do things which it is not designed for.

Azure Cosmos Db document partition key having duplicate, but find duplicate document with combination of other columns

I have below document JSON (pasted partial JSON, actual JSON will be complex and embedded). The JSON has Code as ParitionKey, I am trying to build No SQL database documents by migrating my sql tables, and I will have Code, Type making Unique row, as you can see below Code = 4 is duplicated with different Type and id I just generated GUID (not sure on id field so generated GUID and assigned to it).
we only have two values for Type filed, it's either RI or NRI for entire data, and Code is duplicated like below sample data Code:4, but combination of Type & Code fields make it unique.
Example JSON:
{
"id" : "88725628-2a9a-4fc7-90ed-29c5ffbd45fa"
"Code": "4",
"Type": "RI",
"Description": "MAC/CHEESE ",
},
{
"id" : "88725628-9a3b-4fc7-90ed-29c5ffbd34sk"
"Code": "8",
"Type": "RI",
"Description": "Cereals",
},
{
"id" : "88725628-6d9f-4fc7-90ed-29c4ffbd87de"
"Code": "4",
"Type": "NRI",
"Description": "Christmas Deal",
}
In NoSQL cosmos document db, I couldn't use two columns as partition key, so I have only code as Partition key, but when I am trying to insert into Cosmos Db how do I check if not exists then only insert or else I would end up creating duplicate documents:
CreateItemAsync --> I need a way to check if the document already exists if not then create
I have below code to check and if not found create Item
try
{
// Read the item to see if it exists.
ItemResponse<Item> itemResponse = await this.container.ReadItemAsync<Item>(itm.Id, new PartitionKey(itm.Code));
}
catch (CosmosException ex) when (ex.StatusCode == HttpStatusCode.NotFound)
{
// Create an item in the container representing the Andersen family. Note we provide the value of the partition key for this item, which is "Andersen"
ItemResponse<Item> itemResponse = await this.container.CreateItemAsync<Item>(itm, new PartitionKey(itm.Code));
}
But from above code in ReadItemAsync parameters, how do I know id parameter as it is a GUID randomly generated on every insert, is there a better way to utilize id property before insert into Cosmos DB, so it can be utilized while ReadItemAsync ?
second parameter is paritionKey, If I give code as partition key, it wouldn't work as expected as Code can be duplicated with different "Type" values and it's valid, but Code & Type together makes it unique and we shouldn't allow another document to be inserted if code and type are same.
How do I do it in Cosmos db insert ? I have below questions:
id field --> can I generate GUID and save document or id filed has any purpose which can be utilized during reads ?
Is it ok to pick a partition key which can potentially have duplicates like Code field.
How do I check document exists before insert with above qualifiers as Code filed can be duplicated but only With Type it makes it unique ?
Any suggestions ?
If code and type make a unique row then you should use the value of type for id as well rather than generating a GUID because in Cosmos DB the combination of your partition key and id must be unique.
Then when you do an insert, if the data is already there it will throw an exception which you can catch. For reads, if you know the value for code and type, you can use these to perform a point read to get a single row of data, rather than using a query. This is the most efficient way to fetch data in Cosmos DB.
It is fine to have duplicates for partition key values. You only need to make sure that you have less than 20GB of data for each partition key value.

FIrebase setData kef-value sequence not preserved

I am trying to add data in firebase as the key-value sequence I have provided in setData.
create({Function onAdded, Function onDuplicate}) async {
bool temp = await lrCheck();
if (temp) {
await fireStore.collection("Admin").document(TextFieldData.lrNO).setData(
{
"lrNO": TextFieldData.lrNO,
"departureDate": TextFieldData.departureDate,
"partyName": TextFieldData.partyName,
"vehicleNo": TextFieldData.vehicleNo,
"origin": TextFieldData.origin,
"destination": TextFieldData.destination,
"quantity": TextFieldData.quantity,
"invoiceNo": TextFieldData.invoiceValue,
"invoiceValue": TextFieldData.invoiceValue,
"arrivalDate": TextFieldData.arrivalDate,
"materialType": TextFieldData.materialType,
"phoneNo": TextFieldData.phoneNo,
},
).whenComplete(onAdded());
But the data stored in firebase is in alphabatical orer of keys plzz help.
ex:
arrivalDate: value
destination: value
and so on ....
Fierstore does not set an order for the fields in a document. If you need an order, you will have to apply that in your app code. If you have an array of ordered data to store, consider putting that data in a single list type field instead.
The data in the firebase document are stored in key-value pairs so, There should be no problem even if the data is received in unordered fashion as long as you are mapping the data correctly in your model.

dynamo db FilterExpression, find in json object using value as key

It is possible to somehow filter results by key name that stored in the same object?
I have JSON object "keys", in property "default" stored key of the object that I need. Is it somehow possible to filter like that keys[keys.default].type = some_type?
var params = {
TableName: 'TABLE_NAME',
IndexName: 'TABLE_INDEX', // optional (if querying an index)
KeyConditionExpression: 'myId = :value',
FilterExpression: '#kmap[#kmap.#def].#tp = :keyval',
ExpressionAttributeNames: {names with special characters
'#kmap': 'keys',
'#tp': 'type',
'#def': 'default'
},
ExpressionAttributeValues: { // a map of substitutions for all attribute values
':value': '1',
':keyval': 'some_type'
},
Limit: 10, // optional (limit the number of items to evaluate)
ProjectionExpression: "displayName, #kmap",
ReturnConsumedCapacity: 'TOTLAL', // optional (NONE | TOTAL | INDEXES)
};
docClient.query(params, function(err, data) {
if (err) ppJson(err); // an error occurred
else ppJson(data); // successful response
});
I'm pretty sure the answer is no.
This keys[keys.default] is not even valid json, as far as I can tell.
Of course, you can do this in two steps:
First, query to get the default key
Then query to get the value
Don't forget, filters are obly applied to the result set - it still requires a libear traversal as specified by your Query or Scan operation.
So you can probably more easily run your query on the client.
And lastly, if this is a typical query ypu need to perform, as an optimization, you can lift the default key and value to be top level attributes on the item. Thrn you can actually create a GSI on that attribure and can actually do efficient lookups.

make book.randomID key in amazon dynamodb table

for some reason I want to use book.randomID as key in amazon DynamoDB table using java code. when i tried id added a new field in the item named "book.randomID"
List<KeySchemaElement> keySchema = new ArrayList<KeySchemaElement>();
keySchema.add(new KeySchemaElement().withAttributeName("conceptDetailInfo.conceptId").withKeyType(KeyType.HASH)); // Partition
and here is the json structure
{
"_id":"123",
"book":{
"chapters":{
"chapterList":[
{
"_id":"11310674",
"preferred":true,
"name":"1993"
}
],
"count":1
},
"randomID":"1234"
}
}
so is it possible to use such element as key. if yes how can we use it as key
When creating DynamoDB tables AWS limits it to the types String, Binary and Number. Your attribute book.random seems to be a String.
As long as it's not one of the other data types like List, Map or Set you should be fine.
Just going to AWS console and trying it out worked for me:

Resources