If I insert data into node/a/0 in firebase.
the result will treat a is an array a[0].
The same way, if I set my data in node/a/1 the first array will become null
"a" : [ null, {
"-J-22j_Mb59l0Ws0H9xc" : {
"name" : "chok wee ching"
}
} ]
But it will be fine if node/a/2
"a" : {
"2" : {
"-J-25xjEXUqcpsC-5LOE" : {
"name" : "chok wee ching"
}
}
}
my guess, firebase automatically treat single 0 and 1 as an array.
How can I prevent this?
Firebase has no native support for arrays. If you store an array, it really gets stored as an "object" with integers as the key names.
However, to help people that are storing arrays in Firebase, when you call .val() or use the REST api to read data from Firebase, if the data looks like an array, Firebase will render it as an array.
In particular, if all of the keys are integers, and more than half of the keys between 0 and the maximum key in the object have non-empty values, then Firebase will render it as an array. It's this latter part of the heuristic that you are running into. Your second example only sets a value for 2, not 0 and 1, so less than half of the keys have values, and therefore Firebase renders it as an object.
You can't currently change or prevent this behavior (though it's a common source of confusion so we'll likely make some tweaks here in the future). However it's usually very easy to deal with the behavior once you understand it. If you need further help, perhaps you can expand your question to explain what you need to accomplish and how this behavior is preventing it.
I've encountered the same problem, but actually wanted to have a numeric key array in Swift ([Int:AnyObject]). I've written this function to make sure to always have an array (without null values):
func forceArray(from: Any?) -> [Int:AnyObject] {
var returnArray = [Int:AnyObject]()
if let array = from as? [String:AnyObject] {
for (key, value) in array {
if let key = Int(key) {
returnArray[key] = value
}
}
return returnArray
}
if let array = from as? [AnyObject] {
for (key, value) in array.enumerated() {
if !(value is NSNull) {
returnArray[key] = value
}
}
return returnArray
}
return returnArray
}
Result:
["0":1, "1":2] becomes: [0:1, 1:2]
{"0":1, "6":2} becomes: [0:1, 6:2]
["0":1, "1": null, "2":2] becomes: [0:1, 2:2]
Hope this is helpful for someone!
Yes, storing 0 and 1 as the key will have an issue as Firebase will think it is an array.
My simple workaround is using:
String(format:"%03d", intValue)
So that the resulting key will be "000" and "001", and they can be converted back to Int with ease.
TL;DR;
Workaround for REST API: add a meaningless filter like this one orderBy="$.key"&startAt="0" which actually filters-out all items with negative key): https://workaround-arrays-bagohack.firebaseio.com/matchesHeuristic.json?orderBy=%22$key%22&startAt=%220%22&print=pretty
Explanation
This is a known and unfortunate (IMHO) behaviour of Firebase, documented deep down in their support knowledge base. Quote from Best Practices: Arrays in Firebase:
Firebase has no native support for arrays. If you store an array, it
really gets stored as an "object" with integers as the key names.
// we send this ['hello', 'world']
// Firebase stores this {0:'hello', 1: 'world'}
However, to help people that are storing arrays
in Firebase, when you call .val() or use the REST api to read data, if
the data looks like an array, Firebase will render it as an array.
In particular, *if all of the keys are integers, and more than half of
the keys between 0 and the maximum key in the object have non-empty
values, then Firebase will render it as an array. This latter part is
important to keep in mind.
You can't currently change or prevent this behavior. Hopefully
understanding it will make it easier to see what one can and can't do
when storing array-like data.
So I've set up a small repro for you. Original data:
{
"matchesHeuristic": {
"1": {
"id": "foo",
"value": "bar"
},
"2": {
"id": "w",
"value": "tf"
}
},
"notMatchesHeuristic": {
"1": {
"id": "foo",
"value": "bar"
},
"365": {
"id": "w",
"value": "tf"
}
}
}
As returned by Firebase REST API: https://workaround-arrays-bagohack.firebaseio.com/.json?print=pretty
{
"matchesHeuristic" : [ null, {
"id" : "foo",
"value" : "bar"
}, {
"id" : "w",
"value" : "tf"
} ],
"notMatchesHeuristic" : {
"1" : {
"id" : "foo",
"value" : "bar"
},
"365" : {
"id" : "w",
"value" : "tf"
}
}
}
as you can see matchesHeuristic object is transformed into an array with a null value at index 0 (because it matches the heuristic defined in Firebase docs) whereas notMatchesHeuristic is left intact. This is especially "nice" if you have dynamic data like we do - so we don't know untill runtime if it will match heauristic or not.
Workaround (REST API)
However this portion of the docs doesn't seem to hold:
You can't currently change or prevent this behavior. Hopefully
understanding it will make it easier to see what one can and can't do
when storing array-like data.
You can actually workaround this by requesting items searched by key, so
this https://workaround-arrays-bagohack.firebaseio.com/matchesHeuristic.json?print=pretty is broken
this is intact https://workaround-arrays-bagohack.firebaseio.com/matchesHeuristic.json?orderBy=%22$key%22&startAt=%220%22&print=pretty (add a meaningless filter like this one orderBy="$.key"&startAt="0" which actually filters-out all items with negative key):
{"1":{"id":"foo","value":"bar"},"2":{"id":"w","value":"tf"}}
NB: interestingly, seems Firebase support guys don't know about thi workaround (at least they didn't suggest it when we asked them about this behavior).
Related
I am working on a 'typeahead’ type function which will check my Database with the current typed text to provide search suggestions of users using Felgo.
Here is the link for Felgos Firebase documentation
As to not search every entry I am looking to use the startAt and limitTo for a lower data use.
However when applying the startAt my searches only return undefined, I have tried testing this by changing my startAt from a variable to explicit data but this still only returns undefined.
My function is below:
function searchUsers(searchString) {
db.getValue("public/nameList/", {
orderByChild: true,
startAt: searchString, //searchString is a variable with my .currentText to search.
limitToFirst: 10,
}, function(success, key, value) {
if(success) {
searchArr = []
searchArr = value
console.debug("Read user value for key", key, "from DB:", value)
}
})
}
I have also tried by passing my var searchString through JSON.stringify(searchString) and also return undefined!
Removing the startAt: query entirely returns the entire result of nameList as expected, but no matter how I try to implement my startAt it always returns undefined.
A sample of my nameList JSON is:
nameList: {
"EddieLaw245" : 530343772383,
"EddieLawrence91" : 530343772385,
"EdwardL91" : 530343772386,
"EdwardLaw" : 530343772384,
"Edwardlawrence91" : 530343772380,
"JoBrownLondon" : 530343772381,
"KatiePrescottHair" : 543592635596,
"Tracey-Sweeting" : 530343772382
}
So with the above example, When I type E it should remove the last 3 entries, and so on.
The problem is that you're specifying orderByChild: true. If we look at the documentation of that:
orderByChild: If present, the queried object will have its properties ordered by values at sub-paths defined by the value of this property. Ordering by child properties makes the filter properties startAt, endAt and equalTo filter by the child property values
It may not be immediately clear from this, but orderByChild allows you to order the results on a property value under each of those nodes. So your code tries to order the child nodes on the value of a property true, which isn't possible (and should actually generate a compile-time error in the library) as the nodes under nameList don't have any child properties of their own. They merely have a key and a value.
What you're looking for is orderByKeys, which orders the child nodes on their keys. So:
db.getValue("public/nameList/", {
orderByKeys: true,
startAt: searchString,
limitToFirst: 10,
}
You'll typically also want to specify an endAt value, to ensure your type-ahead only shows values that start with the search string. If you only allow ASCII values in the keys, the simplest way to do this is:
startAt: searchString,
endAt: searchString + "~",
The ~ here is no magic operator, but merely the last ASCII characters. If you want to allow a broader character set, you'll need to use the last character in that character set - for example \uF7FF is the last code point for Unicode.
Update from OP
Though I'm certian Franks correct with typical Firebase usage; I suspect due to the Felgo plugin I am using the full solution has a slight adjustment;
db.getValue("public/nameList/", {
"orderByKey": true,
"startAt": searchString,
"endAt": searchString+"~",
"limitToFirst": 10,
}, function(success, key, value) {....}
})
Notes on the above - my filters/queries are surrounded by quotation marks "startAt", also instead of orderByKeys, I have used orderByKey
Edit for clarity: There are no error messages, it simply returns an empty list if the input string is from the context.arguments, suggesting that it simply isn't getting the input variable out on the query tester (setting it up incorrectly brings up that famous typing error of course). I've also made this into a pipeline with the exact same result. Looking around, people suggest making an intermediate object, but surely I'm just getting my input variables out wrong somehow.
I'm working on a project in AWS Appsync using DynamoDB and I've run into a problem with the context.arguments input.
Basically the code all works if I hardcode the string for the book id into the query (full context to follow), but if I use the context.arguments, it simply refuses to work properly, returning an empty array for the "spines".
I have the following types in my schema:
type Book {
id: ID!
title: String
spines: [Spine]
}
type Spine {
id: ID!
name: String
bookId: ID!
}
I use the following query:
type Query {
getBook(id: ID!): Book
query getBook($bookId: ID!){
getBook(id: $bookId){
title
id
spines {
name
bookId
}
}
}
With the following input (assume this is a relevant guid):
{
"bookId": "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa"
}
And this resolver for the spines object:
{
"version" : "2017-02-28",
"operation" : "Query",
"index" : "bookId-index",
"query" : {
"expression": "#bookId = :bookId",
"expressionNames" : {
"#bookId" : "bookId"
},
"expressionValues" : {
":bookId" : { "S" : "${context.arguments.id}" }
}
}
}
}
I made sure my data set contained false positives too (spines for other books) so that I know when my query brings back the correct data.
This works if I hardcode a guid as string instead of using context.arguments, and gets exactly what I'm looking for for each book guid.
For example, replacing the expression values with this works perfectly:
"expressionValues" : {
":bookId" : { "S" : "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa" }
}
Why does "${context.arguments.id}" not get the input variable here the same way as it seems to in other queries?
Thanks to #IonutTrestian for pointing me in the right direction.
$ctx.args was empty, but I decided to go up the chain to see what was in the entire context, so $util.error($util.toJson($ctx)).
The json object I found included a little object called "Source", which contained the query return for the Book object.
Long story short, $ctx.source.id when applied to my query worked a charm.
I also know a bit more about debugging DynamoDB resolvers in case I encounter problems like this in future. Thank you so much!
I am trying to write a pact consumer test to match the following response.
[
{
"accountId" : 1,
"permissions" : [
{
"schedule" : {
"01/01/2018" : false,
"01/01/1900" : true
},
"permissionId" : 3
}
]
}
]
Each schedule object is composed of an unknown number of keys which match a simple regular expression. But I don't see a way to match a key using a regular expression while having the value map to a simple boolean.
For instance, I see the following method in the API.
public LambdaDslObject eachKeyLike(
String exampleKey,
Consumer<LambdaDslObject> nestedObject)
But that is going to expect a new object as the value, instead of a primitive type.
"schedule" : {
"01/01/2018" : { ... }, // not what I want to match
"01/01/1900" : false // what I want to match
}
Is there a way to specify an imprecise key mapped to a primitive value in pact-jvm?
Sorry, this feature doesn't exist yet, but it's been discussed for the next version of the pact specification. You can add your thoughts on this issue: https://github.com/pact-foundation/pact-specification/issues/47
My question is : how to append a value given by a user to an entity. The user provided value is dynamic.
The Watson response overwrites the toppings variable with the value given by the user, as you can see in the attached image.
{
"output": {
"text": "I got an order to add one or more toppings.
Adding <?context.toppings.append('toppings')?>.
Toppings to provide: <?entities['toppings']?.toString()?>"
},
"context": {
"toppings": "<? entities['toppings']?.toString()?>"
}
}
You can append to an array with the .append() function.
In your example, the expression "toppings": "<? entities['toppings']?.toString()?>" will overwrite the toppings variable every time this node is processed with the actual recognized entities #toppings. First the the $toppings variable needs to be defined as an array, e.g.:
"context" : {
"toppings" : []
}
Then in context part of a dialog node you can write:
"context" : {
"toppings" : "<?$toppings.append(entities['toppings'].toJsonArray())?>"
}
More info in our doc: Watson Conversation Doc
EDIT: Thinking about this, it is probably not a good idea to have the same name for the entity and for the variable you store it in. :-)
On the firebase structure data section, it shows how to structure data with a many-many user-group situation. But, why they have used "referece":true on both the side instead of using a simple array od ids.
Like, it can be used like both the ways:
A user having array of groups
"groups" : [ "groupId1", "groupId2", ... ]
A user having
"groups": {
"groupId1" : true,
"groupId2" : true,
..
}
They have done it a second way. What is the reason for that?
Something was told at the Google I/O 2016 for that in some video. But, I'm unable to recall.
Example from structure your data:
// An index to track Ada's memberships
{
"users": {
"alovelace": {
"name": "Ada Lovelace",
// Index Ada's groups in her profile
"groups": {
// the value here doesn't matter, just that the key exists
"techpioneers": true,
"womentechmakers": true
}
},
...
},
"groups": {
"techpioneers": {
"name": "Historical Tech Pioneers",
"members": {
"alovelace": true,
"ghopper": true,
"eclarke": true
}
},
...
}
}
Firebase recommends against using arrays in its database for most cases. Instead of repeating the reasons here, I'll refer you to this classic blog post on arrays in Firebase.
Let's look at one simple reason you can easily see from your example. Since Firebase arrays in JavaScript are just associative objects with sequential, integer keys, your first sample is stored as:
"groups" : {
0: "groupId1",
1: "groupId2"
]
To detect whether this user is in groupId2, you have to scan all the values in the array. When there's only two values, that may not be too bad. But it quickly gets slower as you have more values. You also won't be able to query or secure this data, since neither Firebase Queries nor its security rules support a contains() operator.
Now look at the alternative data structure:
"groups": {
"groupId1" : true,
"groupId2" : true
}
In this structure you can see whether the user is in groupId2 by checking precisely one location: /groups/groupId2. It that key exists, the user is a member of groupId2. The actual value doesn't really matter in this case, we just use true as a marker value (since Firebase will delete a path if there's no value).
This will also work better with queries and security rules, because you now "just" needs an exists() operator.
For some great insights into this type of modeling, I highly recommend that article on NoSQL data modeling.