Does Amazon DynamoDB scan operation allow you to query on nested attributes of type Array or Object? For example,
{
Id: 206,
Title: "20-Bicycle 206",
Description: "206 description",
RelatedItems: [
341,
472,
649
],
Pictures: {
FrontView: "123",
RearView: "456",
SideView: "789"
}
}
Can I query on RelatedItems[2] or Pictures.RearView attributes?
Yes, you can use a Filter Expression, which is just like Condition Expression. The section that talks about the functions that you can use in these types of expressions mentions the following:
"For a nested attribute, you must provide its full path; for more information, see Document Paths."
The Document Paths reference has examples on how to reference nested attributes in DynamoDB data types like List (what you are calling an array) and Map (what you are calling an object). Check out that reference for examples on how to do so:
MyList[0]
AnotherList[12]
ThisList[5][11]
MyMap.nestedField
MyMap.nestedField.deeplyNestedField
Please note that in DyanomoDB query and scan are quite different (scan is a much costlier operation). So while you can filter on both as pointed out by #coffeeplease; you can only query/index on:
The key schema for the index. Every attribute in the index key schema must be a top-level attribute of type String, Number, or Binary. Other data types, including documents and sets, are not allowed (ref).
Yes, you can by passing list or value.
data = table.scan(FilterExpression=Attr('RelatedItems').contains([1, 2, 3]) & Attr('Pictures.RearView').eq('1'))
Yes, you can query on nested attributes of type array or object using scan or query .
Reference for Python boto3:
https://boto3.amazonaws.com/v1/documentation/api/latest/guide/dynamodb.html#querying-and-scanning
Example: Suppose you want to find out records for which the RearView" > 500 and second item of RelatedItems" > 200, you can do the following:
data = table.scan(
FilterExpression=Attr('RelatedItems[1]').gt('200') & Attr('Pictures.RearView').gt('500'))
Related
Lets say I have this schema:
source_id -> String, HashKey
created_at -> String, RangeKey
required_capabilities -> StringSet
required_capabilities is a Set of Strings that we need to provide in the query in order to be able to retrieve a particular element.
For example:
If I have this three elements:
{
"source_id": "1",
"created_at": "2021-01-18T10:53:25Z",
"required_capabilities": ["Cap1", "Cap2", "Cap3"]
},
{
"source_id": "1",
"created_at": "2021-01-18T10:59:31Z",
"required_capabilities": ["Cap1", "Cap3"]
},
{
"source_id": "1",
"created_at": "2021-01-18T11:05:15Z"
}
I want to create a query, filtering for example source_id = "1" and providing a FilterExpression with the required_capabilities = ["Cap1", "Cap3", "Cap4"].
And I would expect as a result:
{
"source_id": "1",
"created_at": "2021-01-18T10:59:31Z",
"required_capabilities": ["Cap1", "Cap3"] // Since I've provided "Cap1", "Cap3" and "Cap4"
},
{
"source_id": "1",
"created_at": "2021-01-18T11:05:15Z" // Since it doesn't require any capability.
}
I've tried the IN operator as follows, since the stored StringSet should be IN (or Contained by) the given SS, but it didn't work.
aws dynamodb query --table-name TableName --key-condition-expression "source_id = :id" --filter-expression "required_capabilities IN (:rq)" --expression-attribute-values '{":id": {"S": "1"}, ":rq": { "SS": ["Cap1", "Cap3", "Cap4"] }}'
It works only when I provide the exact same StringSet, but If I provide a set that contains the saved one and also have more values, it doesn't return anything.
it seems your issue is around the use of the IN keyword, which does not work with sets. From the docs on conditionals
IN : Checks for matching elements in a list.
AttributeValueList can contain one or more AttributeValue elements of type String, Number, or Binary. These attributes are compared against an existing attribute of an item. If any elements of the input are equal to the item attribute, the expression evaluates to true.
I believe you want the CONTAINS keyword:
CONTAINS : Checks for a subsequence, or value in a set.
AttributeValueList can contain only one AttributeValue element of type String, Number, or Binary (not a set type). If the target attribute of the comparison is of type String, then the operator checks for a substring match. If the target attribute of the comparison is of type Binary, then the operator looks for a subsequence of the target that matches the input. If the target attribute of the comparison is a set ("SS", "NS", or "BS"), then the operator evaluates to true if it finds an exact match with any member of the set. CONTAINS is supported for lists: When evaluating "a CONTAINS b", "a" can be a list; however, "b" cannot be a set, a map, or a list.
Actually, I found out that dynamodb doesn't support the use case I needed, so I found a workaround.
Basically instead of modelling the required_capabilities as a StringSet, I've created a field called required_capability, containing a single required capability (which is ok so far for me) and using the IN operator to check.
If in the future I need to check for more than one capability, I just need to add new fields required_capability_2 and required_capability_3.
It's clearly not ideal, but I guess it's good enough, considering I won't have a lot of required capabilities in a single record, it's usually one, maybe two.
I have the following JSON and am trying to write a JSON Path expression which will return me the isbn number when I have a id of either '123456789' or '987654321'. I tried the following but this did not work. Can anybody tell me what I am doing wrong please. Thanks in advance
JSON Path Expression
$.books[?(#.ids== '123456789' )].isbnNumber
JSON
{
"books": [{
"title": "10",
"isbnNumber": "621197725636",
"ids": [
"123456789",
"987654321"
]
}]
}
The (more traditional) JSONPath implementations that stick close(r) to Goessner's reference specification do not offer handy functions like in which are available in extended implementations like JayWay's JSONPath.
Using Gatling's JSONPath, one thing we could do if the positions of the Ids in question are fixed is accessing their respective indices directly to make the comparison:
$.books[?(#.ids[0] == "123456789" || #.ids[1] == "987654321")].isbnNumber
This will give you the desired result of your example; however, some books only have one of the two indices, or they Id to compare to shows up on a different position it won't work.
With JSONPath, how can you extract a single value from a list of known keys?
For example, I want to write one JSON path expression that can extract Sean from all three of these JSON documents:
{ "firstName": "Sean" }
{ "first_name": "Sean" }
{ "first_name": "Sean", "firstName": "Sean" }
This example is a little contrived, but I have an actual use case that requires this behavior.
The best I can come up with is the expression $.firstName,first_name which will work for #1 and #2 but returns an array for #3 — and I just want the first value that matches.
Basically, I’m looking for a JSONPath extract expression that simulates this JavaScript code:
json.firstName || json.first_name
I believe you want something like below :)
You can get json path using the index .Whn I'm using rest-assured I always use something similar to below code to extract values from my json response .
Response response=given().contentType(ContentType.JSON).get("http://localhost:3000/posts");
JsonPath jsonPathEvaluator = response.jsonPath();
String fn1 = jsonPathEvaluator.get("firstName[0]");
String fn_1=jsonPathEvaluator.get("first_name[0]");
String fn2=jsonPathEvaluator.get("firstName[1]");
You can pass all pair to dict and then extract your values or if you need only values you can use set structure to store keys and separate list to values.
I'm trying to use the update_item functionality for DynamoDB in boto3.
I'm struggling right now to update lists for items. I would like to create a new list if the list does not exist yet and otherwise append to the existing list.
Using an UpdateExpression of the form SET my_list = list_append(my_list, :my_value) returns an error "The provided expression refers to an attribute that does not exist in the item" if the list does not exist yet.
Any idea how I would have to modify my UpdateExpression?
You can use list_append(if_not_exists()) construction.
UpdateExpression:
'SET my_list2 = list_append(if_not_exists(my_list2, :empty_list), :my_value)'
ExpressionAttributeValues:
{ ":my_value":{"L": [{"S":"test"}]}, ":empty_list":{"L":[]} }
Update: as mentioned in the comments, boto3 now raises an error for the expression above and a version without explicit types works: { ":my_value": ["test"], ":empty_list":[] }.
An alternative to Boris solution could be to use set instead of list datatype and use the ADD keyword, it does exactly what you want.
With Add, the update expression becomes: ADD setName :s
And the expression attribute values can be like: {":s": {"SS":["First", "Second"]}}
http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Expressions.UpdateExpressions.html#Expressions.UpdateExpressions.ADD
Is there a way to access dictionary keys in Swift 2 using an index?
var dict = ["item1":1, "item2":2, "item3":3]
dict.keys[0]
results in the error:
15:29: note: overloads for 'subscript' exist with these partially
matching parameter lists: (Base.Index), (Range),
(Self.Index)
print("key=" + dict.keys[i])
I saw an examples from August (Swift: dictionary access via index) doing this:
dict.keys.array[0]
At least in Swift 2, there isn't an array object on dictionary keys.
In Swift 2 the equivalent of dict.keys.array would be Array(dict.keys):
let dict = ["item1":1, "item2":2, "item3":3]
let firstKey = Array(dict.keys)[0] // "item3"
Note: of course, as dictionaries are unordered collections, "first key" of the resulting array may not have a predictable value.
do not rely on order of items in a dictonary, using array directly would be better in your case, you can also handle key/value in array with making array units objects.