I have data like this:
scores: {
uid1: {
score: 15,
displayName: "Uciska"
},
uid2: {
score: 3,
displayName: "Bob"
},
uid3: {
etc...
}
}
I want to rank them by score and keep only the top 100.
I did that by following the doc. But it does not work. It always returns the same order even if the score changes.
const query = firebase.database().ref('scores')
.orderByChild('score')
.limitToLast(100)
query.on('child_added', snapshot => {
const score = snapshot.val().score
console.log(score)
})
I added that too in the rules to optimize but I'm not sure it's correct:
"scores": {
".indexOn": ["score"]
}
What is the right way to go?
Your code is correct and should show the desired result.
You may encounter difficulties to see the result due to the 'child_added' event since "the listener is passed a snapshot containing the new child's data", as detailed here in the doc.
You could use the once() method, as follows, which will show the result a bit more clearly since it will display the entire set of scores.
const query = firebase.database().ref('scores')
.orderByChild('score')
.limitToLast(100)
query.once('value', function (snapshot) {
snapshot.forEach(function (childSnapshot) {
var childKey = childSnapshot.key;
var childData = childSnapshot.val();
console.log(childData);
// ...
});
});
Also, your rule can be written as ".indexOn": "score" since there is only one parameter.
Related
I have a collection users like this:
[{
'uid' : '1',
'favourites' : [
{ // fav1 },
{ // fav2 },
{ // fav3 },
etc
]
},
{
'uid' : '2',
'favourites' : [
{ // fav1 },
{ // fav2 },
{ // fav3 },
etc
]
},
etc
]
In some situations I have to update the favourites collection with a new "fav" and I can do that in this way:
final doc = FirebaseFirestore.instance.collection('users').doc(userId);
doc.update({ 'favourites': FieldValue.arrayUnion([fav.toJson()]) });
however the item might be not there so I have to use doc.set to create a new item. As I am new with Firebase, what is a "best practice" for a problem like this (if the element is not there create it first, otherwise update it)?
You can specify a merge option to set, which does precisely what you want:
doc.set({ 'favourites': FieldValue.arrayUnion([fav.toJson()]) }, SetOptions(merge : true))
You can use a function that can check if there is a doc or not with that specific info. And you can create a if-else statement depends on if there is a doc named like that or not.
An example function for checking the doc:
Future<bool> checkIfDocExists(String stuffID) async {
try {
/// Check If Document Exists
// Get reference to Firestore collection
var collectionRef = FirebaseFirestore.instance
.collection('favorites');
var doc = await collectionRef.doc(userId).get();
return doc.exists;
} catch (e) {
throw e;
}
}
This code has the server insert some documents in a collection for the client to find it later.
I need to return the array for a given task
But the page is saying
No data received
Why is that and how to fix it? Thanks
//Both
FooterButtons2 = new Mongo.Collection('footerButtons2');
//Server
Meteor.publish('footerButtons2', function(){
return FooterButtons2.find();
});
FooterButtons2.insert(
{ "task1": ["submit"]},
{ "task2": ["cancel","continue"]}
);
//client
Meteor.subscribe('footerButtons2');
var res = FooterButtons2.findOne("task1");
When you search like this:
var res = FooterButtons2.findOne("task1");
you are searching an object that has the "_id" key equal to "task1", this is not correct.
You want the object that has the key "task1" in it. The correct way would be:
var res = FooterButtons2.findOne({
task1: { $exists: true }
});
But ideally, you should be doing searches based on values and not keys. Something like this:
FooterButtons2.insert({
task: "task1",
buttons: ["submit"]
}, {
task: "task2",
buttons: ["cancel", "continue"]
});
var res = FooterButtons2.findOne({
task: "task1"
});
I'm building an app where I need to process 5k+ tasks in small batches. For that I have a queue of tasks that is stored in a Firebase. I'd like to be able to pull certain amount of tasks with empty status, update their status and write back.
Currently I don't see how I can pull data where a certain field is empty. Is it possible? If not, what would be the alternative solution?
UPDATED 02/12. Here is the data structure that I have:
{
"-KAMnc89C5Yi_ef18ewc" : {
"0": {
"url": "https://excample.com/url",
"status": "done"
},
"1": {
"url": "https://excample.com/url1"
},
"2": {
"url": "https://excample.com/ur2"
},
"3": {
"url": "https://excample.com/ur3"
}
}
And this is the query I'm using:
queueRef.orderByChild('status').equalTo(null).limitToFirst(1).once('value', function(snapshot) {
console.log(snapshot.val());
});
queueRef points to "-KAMnc89C5Yi_ef18ewc" from the data above.
I expect to get one object - "1", but instead I'm getting all of them. Is there something I'm missing?
Firebase doesn't allow you to store a property without a value. That simply means that the property doesn't exist.
Luckily this doesn't really matter too much, because this seems to work. Given this data structure:
{
"-KADbswYg3FiQF78mmUf": {
"name": "task1",
"status": "done"
},
"-KADbugr7QzTx0s93Fs0": {
"name": "task2"
},
"-KADbvKvBgiAXxnQvoBp": {
"name": "task3"
}
}
This works:
ref.orderByChild('status').equalTo(null).once('value', function(snapshot) {
console.log(snapshot.val());
})
This prints task2 and task3.
Use the DataSnapshot.exists()
This will returns true if this snapshot contains any data. If not it will return false. According to the documentation here. It is slightly more efficient than using snapshot.val() !== null.
With a data structure like this:
{
"girlfriend": {
"first": "Pamala",
"last": "Anderson"
}
}
And a firebase call like this:
var ref = new Firebase("https://myURL/girlfriend/Pamala");
ref.once("value", function(snapshot) {
var a = snapshot.exists();
// a === true
var b = snapshot.child("girlfriend").exists();
// b === true
var c = snapshot.child("girlfriend/first").exists();
// c === true
var d = snapshot.child("girlfriend/middle").exists();
// d === false (because there is no "name/middle" girlfriend in the data snapshot)
});
How do you "upsert" a property to a DynamoDB row. E.g. SET address.state = "MA" for some item, when address does not yet exist?
I feel like I'm having a chicken-and-egg problem because DynamoDB doesn't let you define a sloppy schema in advance.
If address DID already exist on that item, of type M (for Map), the internet tells me I could issue an UpdateExpression like:
SET #address.#state = :value
with #address, #state, and :value appropriately mapped to address, state, and MA, respectively.
But if the address property does not already exist, this gives an error:
'''
ValidationException: The document path provided in the update expression is invalid for update
'''
So.. it appears I either need to:
Figure out a way to "upsert" address.state (e.g., SET address = {}; SET address.state = 'MA' in a single command)
or
Issue three (!!!) roundtrips in which I try it, SET address = {}; on failure, and then try it again.
If the latter.... how do I set a blank map?!?
Ugh.. I like Dynamo, but unless I'm missing something obvious this is a bit crazy..
You can do it with two round trips, the first conditionally sets an empty map for address if it doesn't already exist, and the second sets the state:
db.update({
UpdateExpression: 'SET #a = :value',
ConditionExpression: 'attribute_not_exists(#a)',
ExpressionAttributeValues: {
":value": {},
},
ExpressionAttributeNames: {
'#a': 'address'
}
}, ...);
Then:
db.update({
UpdateExpression: 'SET #a.#b = :v',
ExpressionAttributeNames: {
'#a': 'address',
'#b': 'state'
},
ExpressionAttributeValues: {
':v': 'whatever'
}
}, ...);
You cannot set nested attributes if the parent document does not exist. Since address does not exist you cannot set the attribute province inside it. You can achieve your goal if you set address to an empty map when you create the item. Then, you can use the following parameters to condition an update on an attribute address.province not existing yet.
var params = {
TableName: 'Image',
Key: {
Id: 'dynamodb.png'
},
UpdateExpression: 'SET address.province = :ma',
ConditionExpression: 'attribute_not_exists(address.province)',
ExpressionAttributeValues: {
':ma': 'MA'
},
ReturnValues: 'ALL_NEW'
};
docClient.update(params, function(err, data) {
if (err) ppJson(err); // an error occurred
else ppJson(data); // successful response
});
By the way, I had to replace state with province as state is a reserved word.
Another totally different method is to simply create the address node when creating the parent document in the first place. For example assuming you have a hash key of id, you might do:
db.put({
Item: {
id: 42,
address: {}
}
}, ...);
This will allow you to simply set the address.state value as the address map already exists:
db.update({
UpdateExpression: 'SET #a.#b = :v',
AttributeExpressionNames: {
'#a': 'address',
'#b': 'state'
},
AttributeExpressionValues: {
':v': 'whatever'
}
}, ...);
Some kotlin code to do this recursively regardless how deep it goes. It sets existence of parent paths as condition and if condition check fails, recursively creates those paths first. It has to be in the library's package so it can access those package private fields/classes.
package com.amazonaws.services.dynamodbv2.xspec
import com.amazonaws.services.dynamodbv2.document.Table
import com.amazonaws.services.dynamodbv2.model.ConditionalCheckFailedException
import com.amazonaws.services.dynamodbv2.xspec.ExpressionSpecBuilder.attribute_exists
fun Table.updateItemByPaths(hashKeyName: String, hashKeyValue: Any, updateActions: List<UpdateAction>) {
val parentPaths = updateActions.map { it.pathOperand.path.parent() }
.filter { it.isNotEmpty() }
.toSet() // to remove duplicates
try {
val builder = ExpressionSpecBuilder()
updateActions.forEach { builder.addUpdate(it) }
if (parentPaths.isNotEmpty()) {
var condition: Condition = ComparatorCondition("=", LiteralOperand(true), LiteralOperand(true))
parentPaths.forEach { condition = condition.and(attribute_exists<Any>(it)) }
builder.withCondition(condition)
}
this.updateItem(hashKeyName, hashKeyValue, builder.buildForUpdate())
} catch (e: ConditionalCheckFailedException) {
this.updateItemByPaths(hashKeyName, hashKeyValue, parentPaths.map { M(it).set(mapOf<String, Any>()) })
this.updateItemByPaths(hashKeyName, hashKeyValue, updateActions)
}
}
private fun String.parent() = this.substringBeforeLast('.', "")
Here is a helper function I wrote in Typescript that works for this a single level of nesting using a recursive method.
I refer to the top-level attribute as a column.
//usage
await setKeyInColumn('customerA', 'address', 'state', "MA")
// Updates a map value to hold a new key value pair. It will create a top-level address if it doesn't exist.
static async setKeyInColumn(primaryKeyValue: string, colName: string, key: string, value: any, _doNotCreateColumn?:boolean) {
const obj = {};
obj[key] = value; // creates a nested value like {address:value}
// Some conditions depending on whether the column already exists or not
const ConditionExpression = _doNotCreateColumn ? undefined:`attribute_not_exists(${colName})`
const AttributeValue = _doNotCreateColumn? value : obj;
const UpdateExpression = _doNotCreateColumn? `SET ${colName}.${key} = :keyval `: `SET ${colName} = :keyval ` ;
try{
const updateParams = {
TableName: TABLE_NAME,
Key: {key:primaryKeyValue},
UpdateExpression,
ExpressionAttributeValues: {
":keyval": AttributeValue
},
ConditionExpression,
ReturnValues: "ALL_NEW",
}
const resp = await docClient.update(updateParams).promise()
if (resp && resp[colName]) {
return resp[colName];
}
}catch(ex){
//if the column already exists, then rerun and do not create it
if(ex.code === 'ConditionalCheckFailedException'){
return this.setKeyInColumn(primaryKeyValue,colName,key, value, true)
}
console.log("Failed to Update Column in DynamoDB")
console.log(ex);
return undefined
}
}
I've got quite similar situation. I can think of only a one way to do this in 1 query/atomically.
Extract map values to top level attributes.
Example
Given I have this post item in DynamoDB:
{
"PK": "123",
"SK": "post",
"title": "Hello World!"
}
And I want to later add an analytics entry to same partition:
{
"PK": "123",
"SK": "analytics#december",
"views": {
// <day of month>: <views>
"1": "12",
"2": "457463",
// etc
}
}
Like in your case, it's not possible to increment/decrement views days counters in single query if analytics item nor views map might not exist (could be later feature or don't want to put empty items).
Proposed solution:
{
"PK": "123",
"SK": "analytics#december",
// <day of month>: <views>
"1": "12", // or "day1" if "1" seems too generic
"2": "457463",
// etc
}
}
Then you could do something like this (increment +1 example):
{
UpdateExpression: "SET #day = if_not_exists(#day, 0) + 1",
AttributeExpressionNames: {
'#day': "1"
}
}
if day attribute value doesn't exist, set default value to 0
if item in database doesn't exist, update API adds a new one
I have a pretty basic data structure
events
topics
I would like to be able to easily show (query)
what topics are owned by an event
what events cover a topics
what are the most popular topics this month
I am pretty comfortable with my events structure like
/events/880088/topics.json *
["Firebase", "Cloud"]
but I struggle with how to structure the /topics nodes. I partially get the idea of going with something like
/topics/Firebase
{"12345":true,"88088":true}
and then if when I update an events's topic collection I would have to iterate over all the /topics/ nodes and update /topics/{{topic}}/{{eventid}} to {true | null}. Which seems rather ham fisted.
ALSO, then I am still at a loss of how to query say, what are the topics covered by events this month.
Example JSBin from comments below http://jsbin.com/dumumu/edit?js,output
* I know, I know, arrays are evil, https://www.firebase.com/blog/2014-04-28-best-practices-arrays-in-firebase.html, but I think they fit in this scenaris
Here's one way to add an event:
function addEvent(title, topics) {
var event =ref.child('events').push({ title: title });
topics.forEach(function(topic) {
event.child('topics').child(topic).set(true);
ref.child('topics').child(topic).child(event.key()).set(true);
});
}
Seems pretty simple for me. For an interesting twist, you can use the new multi-location updates we launched yesterday (September 2015):
function addEvent(title, topics) {
var updates = {};
var eventId = ref.push().key();
updates['events/'+eventId+'/title'] = title;
topics.forEach(function(topic) {
updates['events/'+eventId+'/topics/'+topic] = true;
updates['topic/'+topic+'/'+eventId] = true;
});
ref.update(updates);
}
The latter is a bit more code. But it's a single write operation to Firebase, so there's no chance of the user closing the app between write operations.
You invoke both the same of course:
addEvent('Learn all about Firebase', ['Firebase']);
addEvent('Cloudspin', ['Firebase', 'Google', 'Cloud']);
And the data structure becomes:
{
"events": {
"-K-4HCzj_ziHkZq3Fpat": {
"title": "Learn all about Firebase",
"topics": {
"Firebase": true
}
},
"-K-4HCzlBFDIwaA8Ajb7": {
"title": "Cloudspin",
"topics": {
"Cloud": true,
"Firebase": true,
"Google": true
}
}
},
"topic": {
"Cloud": {
"-K-4HCzlBFDIwaA8Ajb7": true
},
"Firebase": {
"-K-4HCzj_ziHkZq3Fpat": true,
"-K-4HCzlBFDIwaA8Ajb7": true
},
"Google": {
"-K-4HCzlBFDIwaA8Ajb7": true
}
}
}
Querying/reporting
With Firebase (and most NoSQL databases), you typically have to adapt your data structure for the reporting you want to do on it.
Abe wrote a great answer on this recently, so go read that for sure: Firebase Data Structure Advice Required
Update: change the topics for an event
If you want to change the topics for an existing event, this function is once way to accomplish that:
function updateEventTopics(event, newTopics) {
newTopics.sort();
var eventId = event.key();
var updates = {};
event.once('value', function(snapshot) {
var oldTopics = Object.keys(snapshot.val().topics).sort();
var added = newTopics.filter(function(t) { return oldTopics.indexOf(t) < 0; }),
removed = oldTopics.filter(function(t) { return newTopics.indexOf(t) < 0; });
added.forEach(function(topic) {
updates['events/'+eventId+'/topics/'+topic] = true;
updates['topic/'+topic+'/'+eventId] = true;
});
removed.forEach(function(topic) {
updates['events/'+eventId+'/topics/'+topic] = null;
updates['topic/'+topic+'/'+eventId] = null;
});
ref.update(updates);
});
}
The code is indeed a bit long, but that's mostly to determine the delta between the current topics and the new topics.
In case you're curious, if we run these API calls now:
var event = addEvent('Cloudspin', Date.now() - month, ['Firebase', 'Google', 'Cloud']);
updateEventTopics(event, ['Firebase', 'Google', 'GCP']);
The changeEventTopics() call will result in this update():
{
"events/-K-93CxuCrFDxM6k0B14/topics/Cloud": null,
"events/-K-93CxuCrFDxM6k0B14/topics/GCP": true,
"topic/Cloud/-K-93CxuCrFDxM6k0B14": null,
"topic/GCP/-K-93CxuCrFDxM6k0B14": true
}