when I scan the aws dynamodb with the attribute name as "Asset_Name" string equals "MakingofaMusicVideo_HD", I am not getting any hits. But the payload contains the field. What's the best way to search for quick search.
Are there any other tools or projects which has a ux that enables users to search quickly?
"Asset_Name": {"S": "MakingofaMusicVideo_HD"}
Below is the payload in dynamodb
"payload": {
"M": {
"CableLabArray": {
"M": {
"ADI": {
"M": {
"Metadata": {
"M": {
"__custom:APP_DATA:2": {
"M": {
"_attributes": {
"M": {
"App": {
"S": "MOD"
},
"Value": {
"S": "CableLabsVOD1.1"
},
"Name": {
"S": "Metadata_Spec_Version"
}
}
}
}
},
"AMS": {
"M": {
"_attributes": {
"M": {
"Asset_ID": {
"S": "XXXXXXXXXXXXXXXXX"
},
"Version_Minor": {
"S": "1"
},
"Description": {
"S": "Making of a Music Video"
},
"Provider_ID": {
"S": "hs90s.ca"
},
"Creation_Date": {
"S": "2020-04-05"
},
"verb": {
"S": "EMPTY"
},
"Product": {
"S": "MOD"
},
"Version_Major": {
"S": "1"
},
"Asset_Name": {
"S": "MakingofaMusicVideo_HD"
},
"Asset_Class": {
"S": "package"
},
"Provider": {
"S": "HS90s"
}
}
}
}
}```
While DynamoDb supports storing complex attributes (e.g. maps, lists, sets, any combination of these), it does not offer search capabilities across complex attributes. You either need to store it as a top level attribute, or work it into the primary key.
Related
I'm trying to filter out a query based on a nested object (no array). I'm currently using AppSync and DynamoDB and the expression with expression values are executed correctly. But the filtering doesn't seem to work.
This is the sample data I'm trying to get (Filter by indicator.id):
Here's my query:
{
"version": "2017-02-28",
"operation": "Query",
"query": {
"expression": "pk = :pk and begins_with(sk, :sk)",
"expressionValues": {
":pk": { "S": "tenant:5fc30406-346c-42e2-8083-fda33ab6000a" },
":sk": {
"S": "school-year:2019-2020:grades:bVgA9abd:subject:m_kpc1Ae6:indicator:"
}
}
},
"filter": {
"expression": " contains(#indicatorId, :sk1) or contains(#indicatorId, :sk2) or contains(#indicatorId, :sk3)",
"expressionNames": { "#indicatorId": "indicator" },
"expressionValues": {
":sk1": {
"M": { "id": { "S": "07c658dd-999f-4e6f-95b8-c6bae422760a" } }
},
":sk2": {
"M": { "id": { "S": "0cf9f670-e284-4a93-b297-5e4a40c50228" } }
},
":sk3": { "M": { "id": { "S": "cd7902be-6512-4b47-b29d-40aff30c73e6" } } }
}
}
}
I've also tried:
{
"version": "2017-02-28",
"operation": "Query",
"query": {
"expression": "pk = :pk and begins_with(sk, :sk)",
"expressionValues": {
":pk": { "S": "tenant:5fc30406-346c-42e2-8083-fda33ab6000a" },
":sk": {
"S": "school-year:2019-2020:grades:bVgA9abd:subject:m_kpc1Ae6:indicator:"
}
}
},
"filter": {
"expression": " contains(#indicatorId, :sk1) or contains(#indicatorId, :sk2) or contains(#indicatorId, :sk3)",
"expressionNames": { "#indicatorId": "indicator.id" },
"expressionValues": {
":sk1": { "S": "07c658dd-999f-4e6f-95b8-c6bae422760a" },
":sk2": { "S": "0cf9f670-e284-4a93-b297-5e4a40c50228" },
":sk3": { "S": "cd7902be-6512-4b47-b29d-40aff30c73e6" }
}
}
}
I've also tried searching around StackOverflow, and Amazon forums and haven't found it directly to my problem:
How to filter by elements in an array (or nested object) in DynamoDB
Nested Query in DynamoDB returns nothing
Referring to this answer.enter link description here
According to DDB Nested Attributes doc, the filter expression should look like the following format:
"filter" : {
"expression" : "#path.#filter = :${fp}", ## filter path parent.target = :target
"expressionNames": {
"#path" : "${path}",
"#filter" : "${fp}"
},
"expressionValues" : {
":${fp}" : $util.dynamodb.toDynamoDBJson(${$target[$fp].eq}) ## :target : value to filter for
}
}
I hope I can express myself a way you can all understand.
Here's the thing, I'm developing a back-end for an app in nodeJS. There's a table named 'chat' in dynamoDB with the ID of the chat's creator and a list of messages IDs, also there's a table named 'messages' with the author of each message and the text.
When the front-end asks the server for all the chats it has to send a list of chats objects with this sctructure:
[
{
"ID_CHAT": {
"S": "asd123asd"
},
"author": {
"name": "Name",
"lastname": "Lastname"
},
"date": {
"S": "19/11/2017 16:00"
},
"USER_DESTINATION": {
"S": "222asd222"
},
"messages": [
{
"Item": {
"author": {
"name": "Name",
"lastname": "Lastname"
},
"text": {
"S": "Hi!"
},
"MESSAGE_ID": {
"S": "456asd456"
},
"date": {
"S": "19/11/2017 16:05"
}
}
},
{
"Item": {
"author": {
"name": "Name",
"lastname": "Lastname"
},
"text": {
"S": "zup?"
},
"MESSAGE_ID": {
"S": "789asd789"
},
"date": {
"S": "19/11/2017 16:06"
}
}
}
]
}
{
"ID_CHAT": {
"S": "123asd123asd"
},
"author": {
"S": "123asd123"
}, and so on...
]
But in order to send this I need to get the chat's authors data from dynamodb (asynchronous function) and the list of messages data again from dynamodb.
I tried combining map() to get the chats authors and Promise.all() for the messages but it won't work cause the second iteration of the map() function runs after the Promise.then().
I'll appretiate your help!
I am using the AWS service proxy to expose DynamoDB as API. For PutItem it works fine, but I don't have idea how to update a array of object.
Below "paymentDetail" is the list of Map, I can post to DynamoDB, but I failed for update the JSON payload. Not sure DynamoDB support this or not, I always got this message:
{"__type": "com.amazon.coral.service#SerializationException"}
Can anybody share your ideas how to use mapping template to update JSON payload? Thanks in advance.
Here is my code, please advise and help.Please kindly let me know in case that the information is not sufficient.
--------- Request Method for UpdateItem ------------
{
"TableName": "claim",
"Key": {
"claimNumber": {
"S": "$input.params("claimNumber")"
}
},
"UpdateExpression": "set claimStatus = :claimStatus, statusUpdateDate = :statusUpdateDate, netClaimAmount = :netClaimAmount, paymentDetail = :paymentDetail, updatedAt = :updatedAt",
"ConditionExpression": "claimNumber = :claimNumber",
"ExpressionAttributeValues": {
":claimNumber": {"S": "$input.params("claimNumber")"},
":claimStatus": {"S": "$input.path("$.claimStatus")"},
":statusUpdateDate": {"S": "$input.path("$.statusUpdateDate")"},
":netClaimAmount": {"N": "$input.path("$.netClaimAmount")"},
":paymentDetail": {"L": "$input.path("$.paymentDetail")"},
":updatedAt": {"S": "$input.path("$.updatedAt")"}
},
"ReturnValues": "UPDATED_NEW"
}
----------test data for UpdateItem---------
{
"claimNumber": "000-00-000959",
"updatedAt":"2017-09-10",
"claimStatus": "close",
"statusUpdateDate": "2017-09-13",
"netClaimAmount": 60000000,
"paymentDetail": [
{
"coverage": "AAA",
"claimPaymentDate": "2017-09-03",
"claimAmount": "10000000",
"costCategory": "1",
"costType": "A"
},
{
"coverage": "BBB",
"claimPaymentDate": "2017-09-03",
"claimAmount": "20000000",
"costCategory": "2",
"costType": "B"
},
{
"coverage": "CCC",
"claimPaymentDate": "2017-09-03",
"claimAmount": "30000000",
"costCategory": "3",
"costType": "C"
}
]
}
---PutItem for your information---
#set($inputRoot = $input.path('$'))
{
"TableName": "claim",
"Item": {
"claimNumber": {
"S": "$input.path("$.claimNumber")"
},
"policyNumber": {
"S": "$input.path("$.policyNumber")"
},
"productName": {
"S": "$input.path("$.productName")"
},
"lossDate": {
"S": "$input.path("$.lossDate")"
},
"lossTime": {
"S": "$input.path("$.lossTime")"
},
"reportedDate": {
"S": "$input.path("$.reportedDate")"
},
"lossCause": {
"S": "$input.path("$.lossCause")"
},
"description": {
"S": "$input.path("$.description")"
},
"prefectureCode": {
"S": "$input.path("$.prefectureCode")"
},
"city": {
"S": "$input.path("$.city")"
},
"address": {
"S": "$input.path("$.address")"
},
"reportedByType": {
"S": "$input.path("$.reportedByType")"
},
"createdAt": {
"S": "$input.path("$.createdAt")"
},
"updatedAt": {
"S": "$input.path("$.updatedAt")"
},
"claimImageId": {
"S": "$input.path("$.claimImageId")"
},
"contact": {
"M": {
"lastName": {
"S": "$input.path("$.contact.lastName")"
},
"firstName": {
"S": "$input.path("$.contact.firstName")"
},
"postCode": {
"S": "$input.path("$.contact.postCode")"
},
"prefectureCode": {
"S": "$input.path("$.contact.prefectureCode")"
},
"city": {
"S": "$input.path("$.contact.city")"
},
"address": {
"S": "$input.path("$.contact.address")"
},
"homePhone": {
"S": "$input.path("$.contact.homePhone")"
},
"email": {
"S": "$input.path("$.contact.email")"
}
}
},
"claimStatus": {
"S": "$input.path("$.claimStatus")"
},
"statusUpdateDate": {
"S": "$input.path("$.statusUpdateDate")"
},
"netClaimAmount": {
"N": "$input.path("$.netClaimAmount")"
},
"paymentDetail": {
"L":[
#foreach($elem in $inputRoot.paymentDetail){
"M": {
"coverage": {
"S": "$elem.coverage"
},
"claimPaymentDate": {
"S": "$elem.claimPaymentDate"
},
"claimAmount": {
"S": "$elem.claimAmount"
},
"costCategory": {
"S": "$elem.costCategory"
},
"costType": {
"S": "$elem.costType"
}
}
}#if($foreach.hasNext),#end
#end
]
}
}
}
Your body mapping template seems to have issues.
Please see below the corrected one.
#set($inputRoot = $input.path('$'))
{
"TableName": "claim",
"Key": {
"claimNumber": {
"S": "$input.path("$.claimNumber")"
}
},
"UpdateExpression": "set claimStatus = :claimStatus, statusUpdateDate = :statusUpdateDate, netClaimAmount = :netClaimAmount, paymentDetail = :paymentDetail, updatedAt = :updatedAt",
"ConditionExpression": "claimNumber = :claimNumber",
"ExpressionAttributeValues": {
":claimNumber": {"S": "$input.path("$.claimNumber")"},
":claimStatus": {"S": "$input.path("$.claimStatus")"},
":statusUpdateDate": {"S": "$input.path("$.statusUpdateDate")"},
":netClaimAmount": {"N": "$input.path("$.netClaimAmount")"},
":updatedAt": {"S": "$input.path("$.updatedAt")"},
":paymentDetail": {
"L":[
#foreach($elem in $inputRoot.paymentDetail){
"M": {
"coverage": {
"S": "$elem.coverage"
},
"claimPaymentDate": {
"S": "$elem.claimPaymentDate"
},
"claimAmount": {
"S": "$elem.claimAmount"
},
"costCategory": {
"S": "$elem.costCategory"
},
"costType": {
"S": "$elem.costType"
}
}
}#if($foreach.hasNext),#end
#end
]
}
},
"ReturnValues": "UPDATED_NEW"
}
I have the following record in DynamoDB:
{
"BusinessNo": {
"N": "12345"
},
"Metadata": {
"M": {
"MimeType": {
"S": "audio/wav"
},
"FileName": {
"S": "00032329.wav"
},
"CustomC": {
"S": "baz"
},
"CustomA": {
"S": "foo"
},
"CustomB": {
"S": "bar"
},
"Size": {
"S": "3992020323"
}
}
},
"Id": {
"S": "f0de8af3-a7f5-4d9b-ad5d-b2f150abd15e"
},
"Revision": {
"N": "2"
}
}
But when I submit the following using the update method of DynamoDB.DocumentClient from the nodejs AWS SDK (I have also tried add instead of set):
{
"TableName": "Storage_FileMetadata",
"Key": {
"Id": "f0de8af3-a7f5-4d9b-ad5d-b2f150abd15e",
"BusinessNo": "12345"
},
"ExpressionAttributeNames": {
"#m": "Metadata",
"#k": "CustomD",
"#r": "Revision"
},
"ExpressionAttributeValues": {
":r": 4,
":v": "doo-wop"
},
"UpdateExpression": "set #m.#k = :v",
"ConditionExpression": "#r < :r"
}
I get the following exception:
{
"message": "Type mismatch for attribute to update",
"code": "ValidationException",
"time": "2016-11-11T18:55:01.543Z",
"requestId": "b9d78c87-1c4d-400a-8968-d761b657cd53",
"statusCode": 400,
"retryable": false,
"retryDelay": 0
}
I think I'm missing something about adding/updating nested attributes but after reading the docs I can't figure out what.
Seems that you need to send the value "BusinessNo": "12345" as a number
"Key": {
"Id": "f0de8af3-a7f5-4d9b-ad5d-b2f150abd15e",
"BusinessNo": 12345
}
So I'm trying to set up a basic POST to a API Post Method I made that's connected to a simple dynamodb. I have the following Body Mapping Template below:
{
"TableName": "bars",
"Item": {
"barid": {
"S": "$input.path('$.barid')"
},
"phone": {
"S": "$input.path('$.phone')"
},
"location": {
"S": "$input.path('$.location')"
},
"happyhour": {
"L": [
{
"M": {
"Time": {
"S": "$input.path('$.Time')"
},
"Deal": {
"S": "$input.path('$.Deal')"
},
"Day": {
"S": "$input.path('$.Day')"
}
}
},
{
"M": {
"Time": {
"S": "$input.path('$.Time')"
},
"Deal": {
"S": "$input.path('$.Deal')"
},
"Day": {
"S": "$input.path('$.Day')"
}
}
},
{
"M": {
"Time": {
"S": "$input.path('$.Time')"
},
"Deal": {
"S": "$input.path('$.Deal')"
},
"Day": {
"S": "$input.path('$.Day')"
}
}
},
{
"M": {
"Time": {
"S": "$input.path('$.Time')"
},
"Deal": {
"S": "$input.path('$.Deal')"
},
"Day": {
"S": "$input.path('$.Day')"
}
}
},
{
"M": {
"Time": {
"S": "$input.path('$.Time')"
},
"Deal": {
"S": "$input.path('$.Deal')"
},
"Day": {
"S": "$input.path('$.Day')"
}
}
},
{
"M": {
"Time": {
"S": "$input.path('$.Time')"
},
"Deal": {
"S": "$input.path('$.Deal')"
},
"Day": {
"S": "$input.path('$.Day')"
}
}
},
{
"M": {
"Time": {
"S": "$input.path('$.Time')"
},
"Deal": {
"S": "$input.path('$.Deal')"
},
"Day": {
"S": "$input.path('$.Day')"
}
}
}
]
},
"name": {
"S": "$input.path('$.name')"
}
}
}
Here is what I'm testing the post statement with:
{
"barid": {
"S": "005"
},
"happyhour": {
"L": [{
"M": {
"Time": {
"S": "11AM-9AM"
},
"Deal": {
"S": "$3 Mimosas; $3 Bloody Marys"
},
"Day": {
"S": "Sunday"
}
}
}, {
"M": {
"Time": {
"S": "4PM - 9PM"
},
"Deal": {
"S": "$4 Margaritas, Corona, Corona Light; $3 Bud Light Lime"
},
"Day": {
"S": "Monday"
}
}
}, {
"M": {
"Time": {
"S": "4PM-9PM"
},
"Deal": {
"S": "1/2 Price Burgers; $2 Bud and Bud Light Drafts"
},
"Day": {
"S": "Tuesday"
}
}
}, {
"M": {
"Time": {
"S": "4PM-9PM"
},
"Deal": {
"S": "$2 Drafts; $3 Food Menu"
},
"Day": {
"S": "Wednesday"
}
}
}, {
"M": {
"Time": {
"S": "4PM-9PM"
},
"Deal": {
"S": "$2 Beers, House Liquor, Flavored Vodka; 1/2 Price Wings; $5 Pizzas or Pasta Bowl"
},
"Day": {
"S": "Thursday"
}
}
}, {
"M": {
"Time": {
"S": "4PM-9PM"
},
"Deal": {
"S": "$5 Red Bull House Liquor Cocktails; $5 Wings, Nachos, Sliders, Flatbreads; $5 Finlandia Cocktails and Martinis; $15 Sam Adams Light Buckets"
},
"Day": {
"S": "Friday"
}
}
}, {
"M": {
"Time": {
"S": "None"
},
"Deal": {
"S": "None"
},
"Day": {
"S": "Sunday"
}
}
}]
},
"phone": {
"S": "(703) 527-1600"
},
"location": {
"S": "3100 Clarendon Blvd, Arlington, VA 22201"
},
"name": {
"S": "Mister Days"
}
}
Whenever I run this it seems that it populates the barid, phone and location field but leaves the happyhour array blank. Below is what I got in the logs after running the test.
"TableName": "bars",
"Item": {
"barid": {
"S": "{S=005}"
},
"phone": {
"S": "{S=(703) 527-1576}"
},
"location": {
"S": "{S=2500 Hess Road}"
},
"happyhour": {
"L": [
{
"M": {
"Time": {
"S": ""
},
"Deal": {
"S": ""
},
"Day": {
"S": ""
}
}
},
{
"M": {
"Time": {
"S": ""
},
"Deal": {
"S": ""
},
"Day": {
"S": ""
}
}
},
{
"M": {
"Time": {
"S": ""
},
"Deal": {
"S": ""
},
"Day": {
"S": ""
}
[TRUNCATED]
Thu Jun 30 15:36:27 UTC 2016 : Endpoint response body before transformations: {"__type":"com.amazon.coral.service#UnknownOperationException"}
Not sure what I'm doing wrong but anything would help. Thanks!
It looks like happyhour contains an object with a single attribute named L which contains an array of object. You'll probably want to loop over the array so that your mapping template will still work regardless of the length of the array. You also need to reference items in the input by their full path, not just their name.
It should look something like this...
#set($inputRoot = $input.path('$'))
{
"TableName": "bars",
"Item": {
"barid": {
"S": "$inputRoot.barid.S"
},
"name": {
"S": "$inputRoot.name.S"
},
"phone": {
"S": "$inputRoot.phone.S"
},
"location": {
"S": "$inputRoot.location.S"
},
"happyhour": {
"L": [
#foreach($elem in $inputRoot.happyhour.L)
{
"M": {
"Time": {
"S": "$elem.M.Time.S"
},
"Deal": {
"S": "$elem.M.Deal.S"
},
"Day": {
"S": "$elem.M.Day.S"
}
}
}#if($foreach.hasNext),#end
#end
]
}
}
Note that the mapping template order and the output order do not have to match the input order. Also, you don't have to keep the exact same structure. You could, for example, remove all of the extra type information and output cleaner json like this...
#set($inputRoot = $input.path('$'))
{
"TableName": "bars",
"Item": {
"barid": "$inputRoot.barid.S",
"name": "$inputRoot.name.S",
"phone": "$inputRoot.phone.S",
"location": "$inputRoot.location.S",
"happyhour": [
#foreach($elem in $inputRoot.happyhour.L)
{
"Time": "$elem.M.Time.S",
"Deal": "$elem.M.Deal.S",
"Day": "$elem.M.Day.S"
}#if($foreach.hasNext),#end
#end
]
}
This might not be right approach if you really want to do the mapping in the API, but you can leave the problem up to aws-sdk.
You can create a simple lambda function between your API and dynamoDB. Using there some of the methods provided by Aws:
http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/DynamoDB/DocumentClient.html
I tackled a similar issue with this function:
var AWS = require("aws-sdk");
var docClient = new AWS.DynamoDB.DocumentClient();
var tableName = "tableName";
var saveData = function (data) {
var params = {
TableName: tableName,
Item: data
};
docClient.put(params, function (err, data) {
if (err) {
console.error("Unable to add item. Error JSON:", JSON.stringify(err, null, 2));
} else {
console.log("Added item:", JSON.stringify(data, null, 2));
}
});
};
exports.handler = function (event) {
saveData(event);
};