Firebase orderByPriority used with equalTo strange behavior - firebase

I try to use the second argument of equalTo(value, key) when using orderByPriority(), as documented here.
The problem is that results are inconsistent when using that second key argument.
Example dataset
items
key1
key2
key3
key4
All items have the same priority: 10 (this is just for the example, in my app there are other items with other priorities)
When issuing the following Firebase query to get the first two items with that priority:
dbRef.child('items').orderByPriority().equalTo(10).limitToFirst(2)
I get the following - expected - results :
{ "key1": ..., "key2": ... }
Then I try to get the results after the key2 item, as explained in the docs:
dbRef.child('items').orderByPriority().equalTo(10, 'key2').limitToFirst(2)
Result is pretty strange, with always only one item, the one with the provided key:
{ "key2": ... }
What i was expecting is two results starting at or after the key provided, so
{ "key2": ..., "key3": ... }
Or
{ "key3": ..., "key4": ... }
Question
How should I use the equalTo() filter with its second argument?
It seem this question was already asked, but it did not get any answers...

The equalTo() filter will only get items exactly matching the key specified. If you'd like to start at 'key2' and get multiple after that, use the startAt() filter. You cannot use multiple orderBy statements in the same query, though, so you may need to reformat your code. I would recommend that you save items by desired priority. Then you can query that group of items using the startAt() filter. Heres a dataset:
items10
key1
key2
items9
key1
key2
In the startAt() filter, you specify the starting key, and then it will query for any key with the same or greater value. Here's a link to the docs for this functionality: startAt().
Using this method, limitToFirst(2) should give you key2 and key3. Here's a code example, using priority 10.
dbRef.child('items10').orderByKey().startAt('key2').limitToFirst(2)
Note: you may also need to just grab an element, without knowing its priority. For this, create another key called items that just contains all keys (you would not use this for queries).

Related

How to filter elements by node key in JSONPath?

What I want
Apply a JSONPath to given json response, to match specific elements by comparing their children's node keys with a value.
Input
{
"data": {
"ticket": {
"1": "foo",
"2": "bar",
"3": "baz"
}
}
}
Output (expected)
"3": "baz"
Case description
I want to apply a JSONPath expression, to filter ticket elements with ticket key greater than "2", so in this case it should match only the 3rd "baz" ticket.
ticket keys are only integer numbers in my data
Code area
This matches all node keys aka ticket keys
$.data.ticket.*~
This is a basic example of filtering
$..book[?(#.price<10)] // -> filter all books cheaper than 10
I am trying somehow to combine them in order to achieve the desired result
Where I test it
https://jsonpath.com/
References
https://goessner.net/articles/JsonPath/
It is possible with jsonpath-plus. The site https://jsonpath.com/ uses jsonpath-plus library internally.
It has some convenient additions or elaborations not provided in the original spec of jsonpath.
Use the #property to compare the value of the key.
$.data.ticket[?(#property > 2)]

DynamoDb check that a SS attribute in contained in a given SS

Lets say I have this schema:
source_id -> String, HashKey
created_at -> String, RangeKey
required_capabilities -> StringSet
required_capabilities is a Set of Strings that we need to provide in the query in order to be able to retrieve a particular element.
For example:
If I have this three elements:
{
"source_id": "1",
"created_at": "2021-01-18T10:53:25Z",
"required_capabilities": ["Cap1", "Cap2", "Cap3"]
},
{
"source_id": "1",
"created_at": "2021-01-18T10:59:31Z",
"required_capabilities": ["Cap1", "Cap3"]
},
{
"source_id": "1",
"created_at": "2021-01-18T11:05:15Z"
}
I want to create a query, filtering for example source_id = "1" and providing a FilterExpression with the required_capabilities = ["Cap1", "Cap3", "Cap4"].
And I would expect as a result:
{
"source_id": "1",
"created_at": "2021-01-18T10:59:31Z",
"required_capabilities": ["Cap1", "Cap3"] // Since I've provided "Cap1", "Cap3" and "Cap4"
},
{
"source_id": "1",
"created_at": "2021-01-18T11:05:15Z" // Since it doesn't require any capability.
}
I've tried the IN operator as follows, since the stored StringSet should be IN (or Contained by) the given SS, but it didn't work.
aws dynamodb query --table-name TableName --key-condition-expression "source_id = :id" --filter-expression "required_capabilities IN (:rq)" --expression-attribute-values '{":id": {"S": "1"}, ":rq": { "SS": ["Cap1", "Cap3", "Cap4"] }}'
It works only when I provide the exact same StringSet, but If I provide a set that contains the saved one and also have more values, it doesn't return anything.
it seems your issue is around the use of the IN keyword, which does not work with sets. From the docs on conditionals
IN : Checks for matching elements in a list.
AttributeValueList can contain one or more AttributeValue elements of type String, Number, or Binary. These attributes are compared against an existing attribute of an item. If any elements of the input are equal to the item attribute, the expression evaluates to true.
I believe you want the CONTAINS keyword:
CONTAINS : Checks for a subsequence, or value in a set.
AttributeValueList can contain only one AttributeValue element of type String, Number, or Binary (not a set type). If the target attribute of the comparison is of type String, then the operator checks for a substring match. If the target attribute of the comparison is of type Binary, then the operator looks for a subsequence of the target that matches the input. If the target attribute of the comparison is a set ("SS", "NS", or "BS"), then the operator evaluates to true if it finds an exact match with any member of the set. CONTAINS is supported for lists: When evaluating "a CONTAINS b", "a" can be a list; however, "b" cannot be a set, a map, or a list.
Actually, I found out that dynamodb doesn't support the use case I needed, so I found a workaround.
Basically instead of modelling the required_capabilities as a StringSet, I've created a field called required_capability, containing a single required capability (which is ok so far for me) and using the IN operator to check.
If in the future I need to check for more than one capability, I just need to add new fields required_capability_2 and required_capability_3.
It's clearly not ideal, but I guess it's good enough, considering I won't have a lot of required capabilities in a single record, it's usually one, maybe two.

Is it possible to combine if_not_exists and list_append in update_item

I'm trying to use the update_item functionality for DynamoDB in boto3.
I'm struggling right now to update lists for items. I would like to create a new list if the list does not exist yet and otherwise append to the existing list.
Using an UpdateExpression of the form SET my_list = list_append(my_list, :my_value) returns an error "The provided expression refers to an attribute that does not exist in the item" if the list does not exist yet.
Any idea how I would have to modify my UpdateExpression?
You can use list_append(if_not_exists()) construction.
UpdateExpression:
'SET my_list2 = list_append(if_not_exists(my_list2, :empty_list), :my_value)'
ExpressionAttributeValues:
{ ":my_value":{"L": [{"S":"test"}]}, ":empty_list":{"L":[]} }
Update: as mentioned in the comments, boto3 now raises an error for the expression above and a version without explicit types works: { ":my_value": ["test"], ":empty_list":[] }.
An alternative to Boris solution could be to use set instead of list datatype and use the ADD keyword, it does exactly what you want.
With Add, the update expression becomes: ADD setName :s
And the expression attribute values can be like: {":s": {"SS":["First", "Second"]}}
http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Expressions.UpdateExpressions.html#Expressions.UpdateExpressions.ADD

DynamoDB nested attribute querying support

Does Amazon DynamoDB scan operation allow you to query on nested attributes of type Array or Object? For example,
{
Id: 206,
Title: "20-Bicycle 206",
Description: "206 description",
RelatedItems: [
341,
472,
649
],
Pictures: {
FrontView: "123",
RearView: "456",
SideView: "789"
}
}
Can I query on RelatedItems[2] or Pictures.RearView attributes?
Yes, you can use a Filter Expression, which is just like Condition Expression. The section that talks about the functions that you can use in these types of expressions mentions the following:
"For a nested attribute, you must provide its full path; for more information, see Document Paths."
The Document Paths reference has examples on how to reference nested attributes in DynamoDB data types like List (what you are calling an array) and Map (what you are calling an object). Check out that reference for examples on how to do so:
MyList[0]
AnotherList[12]
ThisList[5][11]
MyMap.nestedField
MyMap.nestedField.deeplyNestedField
Please note that in DyanomoDB query and scan are quite different (scan is a much costlier operation). So while you can filter on both as pointed out by #coffeeplease; you can only query/index on:
The key schema for the index. Every attribute in the index key schema must be a top-level attribute of type String, Number, or Binary. Other data types, including documents and sets, are not allowed (ref).
Yes, you can by passing list or value.
data = table.scan(FilterExpression=Attr('RelatedItems').contains([1, 2, 3]) & Attr('Pictures.RearView').eq('1'))
Yes, you can query on nested attributes of type array or object using scan or query .
Reference for Python boto3:
https://boto3.amazonaws.com/v1/documentation/api/latest/guide/dynamodb.html#querying-and-scanning
Example: Suppose you want to find out records for which the RearView" > 500 and second item of RelatedItems" > 200, you can do the following:
data = table.scan(
FilterExpression=Attr('RelatedItems[1]').gt('200') & Attr('Pictures.RearView').gt('500'))

SELECT COUNT(*) doesn't work in QML

I'm trying to get the number of records with QML LocalStorage, which uses sqlite. Let's take this snippet in account:
function f() {
var db = LocalStorage.openDatabaseSync(...)
db.transaction (
function(tx) {
var b = tx.executeSql("SELECT * FROM t")
console.log(b.rows.length)
var c = tx.executeSql("SELECT COUNT(*) FROM t")
console.log(JSON.stringify(c))
}
)
}
The output is:
qml: 3
qml: {"rowsAffected":0,"insertId":"","rows":{}}
What am I doing wrong that the SELECT COUNT(*) doesn't output anything?
EDIT: rows only seems empty in the second command. Calling
console.log(JSON.stringify(c.rows.item(0)))
gives
qml: {"COUNT(*)":3}
Two questions now:
Why is rows shown as empty
How can I access the property inside c.rows.item(0)
In order to visit the items, you have to use:
b.rows.item(i)
Where i is the index of the item you want to get (in your first example, i belongs to [0, 1, 2] for you have 3 items, in the second one it is 0 and you can query it as c.rows.item(0)).
The rows field appears empty and it is a valid result, for the items are not part of the rows field itself (indeed you have to use a method to get them, as far as I know that method could also be a memento that completely enclose the response data) and the item method is probably defined as not enumerable (I cannot verify it, I'm on the beach and it's quite difficult to explore the Qt code now :-)). You can safely rely on the length parameter to know if there are returned values, thus you can iterate over them to print them out. I did something like that in a project of mine and it works fine.
The properties inside item(0) have the same names given for the query. I suggest to rewrite that query as:
select count(*) as cnt from t
Then, you can get the count as:
c.rows.item(0).cnt

Resources