I'm having some trouble storing arrays in DynamoDB on PutItem with AppSync, and it's turning me crazy :P
The problem is that the array I pass from GraphQl disappears when I run $util.dynamodb.toMapValuesJson. What am I doing wrong?
I'm really stuck on this, and would very much appreciate any help I can get!
This is the template:
#set($failing_list = [ "foo", 123, { "bar" : "baz" } ])
{
"version" : "2017-02-28",
"operation" : "PutItem",
"key": {
"userId": { "S": "$ctx.args.input.email" },
"createdAt": { "N": "$util.time.nowEpochMilliSeconds()" }
},
"data" : {
"test": $util.dynamodb.toMapValuesJson({
"working_list": [ "foo", 123, { "bar" : "baz" } ],
"failing_list": $failing_list
})
}
}
This is the result:
{
version: '2017-02-28',
operation: 'PutItem',
key: {
userId: { S: 'xxxxxxxxxxxx' },
createdAt: { N: '1531521284789' }
},
data: {
test: {
working_list: {
L: [ { S: 'foo' }, { N: '123' }, { M: { bar: { S: 'baz' } } } ]
},
failing_list: {
M: { L: { L: [ { M: { map: { M: {} } } }, { M: { map: { M: {} } } }, { M: { map: { M: {} } } } ] } }
}
}
}
}
Related
{
"id":{"N": "1"},
"attributes": {
"L": [
{
"M": {
"name": { "S": "AA" }
}
},
{
"M": {
"name": { "S": "BB" }
}
}
]
}
},
{
"id":{"N": "1"},
"attributes": {
"L": [
{
"M": {
"name": { "S": "BB" }
}
}
]
}
}
With the above data in the same partition( id as the partition key), How can I find records where any of attributes has name = 'BB' ?
I can filter by Nth item, for example.
KeyConditionExpression: 'id = :myId',
ExpressionAttributeValues: {'myId':{N:'1'},'myValue':{S:'BB'}},
ExpressionAttributeNames:{'#name': 'name'},
FilterExpression: 'attributes[0].#name=:myValue'
This would return only 2nd item. But is there a way to filter by ANY item in the array? It should return both records.
Tried set FilterExpression to attributes[*].#name=:myValue or attributes.#name=:myValue, neither works.
this is my code chunk
db.collection.aggregate([
{
"$unwind": "$items"
},
{
"$group": {
"_id": "$_id"
}
},
{
"$addFields": {
"total": {
"$multiply": [
"$items.quantity",
"$items.price"
]
}
}
},
{
"$limit": 10
}
])
This is what it outputs
This is a sample document
The problem you're facing is due to your $group stage. it works similar to SQL's groupby if you're more familiar.
What this means is that the quantity and price field are lost during that stage. you'll have to modify it a little to retain them.
I'm not sure exactly what you're trying to achieve as you did not specify but here is a toy example of what I imagine you're looking for:
db.collection.aggregate([
{
"$unwind": "$items"
},
{
"$group": {
"_id": "$items.name",
quantity: {
$sum: "$items.quantity"
},
price: {
$first: "$items.price"
}
}
},
{
"$addFields": {
"total": {
"$multiply": [
"$quantity",
"$price"
]
}
}
},
{
$sort: {
total: -1
}
},
{
"$limit": 10
}
])
Mongo Playground
So I've been playing around with the firebase firestore feature for my REST API backend for a Godot project I'm working on. I'm using HTTP requests to post and get data from the database. So far I've only been using simple types of data, associating one user's ID to his or her username as a string.
In my code, I've defined a function that is responsible for doing the actual saving of a new file through an HTTP request:
func save_document(path: String, fields: Dictionary, http: HTTPRequest) -> void:
var document = { "fields": fields }
var body = to_json(document)
var url = FIRESTORE_URL + path
var res = http.request(url, _get_request_headers(), false, HTTPClient.METHOD_POST, body)
if res == OK:
print("successfully created document")
else:
print("failed creating document")
Then I call the function:
var payload_body_1 = {
"name": {
"stringValue": username.text
}
}
save_document("users?documentId=%s" % firebase.user_info.id, payload_body_1, http)
All of the above works 100% fine. But when I try to post more complicated data, such as a dictionary of dictionaries of dictionaries, I run into some problems.
When I run the following code:
var res = {
"north": {"spades": [], "hearts": [], "diamonds": [], "clubs": []},
"south": {"spades": [], "hearts": [], "diamonds": [], "clubs": []},
"east": {"spades": [], "hearts": [], "diamonds": [], "clubs": []},
"west": {"spades": [], "hearts": [], "diamonds": [], "clubs": []},
}
save_document("online?documentId=test", res, http)
It doesn't work. I imagine it has something to do with the "stringValue" key I used in the case where it worked, but I can't seem to figure out what to do when the value isn't a string value. Does anyone know how to solve this?
Ok so I actually solved it but I thought I might as well post the answer here if someone else potentially has the same problem in the future.
So basically, every field needs a data type, for instance, "stringValue". for Dictionaries, you need "mapValue", and for lists, you need "arrayValue". In the above Example, you would need to use some code looking like this:
var res = {
"north": { "mapValue": { "fields": {
"spades": { "arrayValue": { "values": [] } },
"hearts": { "arrayValue": { "values": [] } },
"diamonds": { "arrayValue": { "values": [] } },
"clubs": { "arrayValue": { "values": [] } }
} } },
"south": { "mapValue": { "fields": {
"spades": { "arrayValue": { "values": [] } },
"hearts": { "arrayValue": { "values": [] } },
"diamonds": { "arrayValue": { "values": [] } },
"clubs": { "arrayValue": { "values": [] } }
} } },
"east": { "mapValue": { "fields": {
"spades": { "arrayValue": { "values": [] } },
"hearts": { "arrayValue": { "values": [] } },
"diamonds": { "arrayValue": { "values": [] } },
"clubs": { "arrayValue": { "values": [] } }
} } },
"west": { "mapValue": { "fields": {
"spades": { "arrayValue": { "values": [] } },
"hearts": { "arrayValue": { "values": [] } },
"diamonds": { "arrayValue": { "values": [] } },
"clubs": { "arrayValue": { "values": [] } }
} } }
}
the mapValues need the "fields" key which is a dictionary of the keys in the dictionary associated with "mapValue". The arrayValues need something similar, but "values" instead of "fields". each element of the array in values should be a dictionary, and if you want the array for example to contain strings, each element should look like:
{ "stringValue": "text" }
I'm trying to filter out a query based on a nested object (no array). I'm currently using AppSync and DynamoDB and the expression with expression values are executed correctly. But the filtering doesn't seem to work.
This is the sample data I'm trying to get (Filter by indicator.id):
Here's my query:
{
"version": "2017-02-28",
"operation": "Query",
"query": {
"expression": "pk = :pk and begins_with(sk, :sk)",
"expressionValues": {
":pk": { "S": "tenant:5fc30406-346c-42e2-8083-fda33ab6000a" },
":sk": {
"S": "school-year:2019-2020:grades:bVgA9abd:subject:m_kpc1Ae6:indicator:"
}
}
},
"filter": {
"expression": " contains(#indicatorId, :sk1) or contains(#indicatorId, :sk2) or contains(#indicatorId, :sk3)",
"expressionNames": { "#indicatorId": "indicator" },
"expressionValues": {
":sk1": {
"M": { "id": { "S": "07c658dd-999f-4e6f-95b8-c6bae422760a" } }
},
":sk2": {
"M": { "id": { "S": "0cf9f670-e284-4a93-b297-5e4a40c50228" } }
},
":sk3": { "M": { "id": { "S": "cd7902be-6512-4b47-b29d-40aff30c73e6" } } }
}
}
}
I've also tried:
{
"version": "2017-02-28",
"operation": "Query",
"query": {
"expression": "pk = :pk and begins_with(sk, :sk)",
"expressionValues": {
":pk": { "S": "tenant:5fc30406-346c-42e2-8083-fda33ab6000a" },
":sk": {
"S": "school-year:2019-2020:grades:bVgA9abd:subject:m_kpc1Ae6:indicator:"
}
}
},
"filter": {
"expression": " contains(#indicatorId, :sk1) or contains(#indicatorId, :sk2) or contains(#indicatorId, :sk3)",
"expressionNames": { "#indicatorId": "indicator.id" },
"expressionValues": {
":sk1": { "S": "07c658dd-999f-4e6f-95b8-c6bae422760a" },
":sk2": { "S": "0cf9f670-e284-4a93-b297-5e4a40c50228" },
":sk3": { "S": "cd7902be-6512-4b47-b29d-40aff30c73e6" }
}
}
}
I've also tried searching around StackOverflow, and Amazon forums and haven't found it directly to my problem:
How to filter by elements in an array (or nested object) in DynamoDB
Nested Query in DynamoDB returns nothing
Referring to this answer.enter link description here
According to DDB Nested Attributes doc, the filter expression should look like the following format:
"filter" : {
"expression" : "#path.#filter = :${fp}", ## filter path parent.target = :target
"expressionNames": {
"#path" : "${path}",
"#filter" : "${fp}"
},
"expressionValues" : {
":${fp}" : $util.dynamodb.toDynamoDBJson(${$target[$fp].eq}) ## :target : value to filter for
}
}
I'm very newbie to ElasticSearch. This is how my data looks like.how can I map and query on the following data.
[parameters] => Array
(
[0] => Array
(
[param_id] => "Browser"
[param_values] => Array
(
[0] => "Firefox"
)
)
[1] => Array
(
[param_id] => "BrowserVersion"
[param_values] => Array
(
[0] => "39"
)
)
[2] => Array
(
[param_id] => "OS"
[param_values] => Array
(
[0] => "Windows"
)
)
[3] => Array
(
[param_id] => "Softwares"
[param_values] => Array
(
[0] => "Java"
[1] => "Oracle"
[2] => "PHP"
)
)
)
I have to get the results of the following query.
The Users which are using "OS" as "Windows" and "Softwares" as "Java". How to map these data and query?
Below is the Mapping JSON:
{
"response": {
"properties": {
"profile_id" : {"type" : "long"},
"timestamp" : {"type" : "string"},
"parameters" : {
"type": "nested",
"properties":{
"param_id":{"type": "string"},
"param_values": {
"type": "string"
}
}
}
}
}
}
Below is my Query JSON:
{ "query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"bool": {
"must": [
{
"nested": {
"path": "parameters",
"query": {
"bool": {
"must": [
{
"term": {
"parameters.param_id": "Softwares"
}
},
{
"term": {
"parameters.param_values": "Java"
}
}
]
}
}
}
}
]
}
}
} }}
So let's start off by creating the index using
curl -XPUT localhost:9200/responses -d '{
"mappings": {
"response": {
"properties": {
"profile_id": {
"type": "long"
},
"timestamp": {
"type": "string"
},
"parameters": {
"type": "nested",
"properties": {
"param_id": {
"type": "string"
},
"param_values": {
"type": "string"
}
}
}
}
}
}
}'
And then we index your sample document:
curl -XPUT localhost:9200/responses/response/1 -d '{
"profile_id": 1,
"timestamp": "2015-01-01T00:00:00",
"parameters": [
{
"param_id": "Browser",
"param_values": [
"Firefox"
]
},
{
"param_id": "BrowserVersion",
"param_values": [
"39"
]
},
{
"param_id": "OS",
"param_values": [
"Windows"
]
},
{
"param_id": "Softwares",
"param_values": [
"Java",
"Oracle",
"PHP"
]
}
]
}'
You've started off very well and your query is almost correct. If you want to query for users who are using "Windows" as "OS" and "Java" as "Softwares", you already have the constraint on the latter, now you just need one more contraint for the former (i.e. OS=Windows)
{
"size": 30,
"query": {
"filtered": {
"filter": {
"bool": {
"must": [
{
"nested": {
"path": "parameters",
"query": {
"bool": {
"must": [
{
"term": {
"parameters.param_id": "os"
}
},
{
"term": {
"parameters.param_values": "windows"
}
}
]
}
}
}
},
{
"nested": {
"path": "parameters",
"query": {
"bool": {
"must": [
{
"term": {
"parameters.param_id": "softwares"
}
},
{
"term": {
"parameters.param_values": "java"
}
}
]
}
}
}
}
]
}
}
}
}
}