In my app, I'm using RTDB multiple instances, together with RTDB management APIs, to try to dynamically balance the load.
So my point is, because I don't know future load, I will just start with one RTDB instance, and create multiple ones if a specified threshold of usage is exceeded.
In my case, this requires the following:
create new RTDB instance through management API
apply rules to that RTDB instance
apply cloud functions to that RTDB instance
1 and 2 could be programmatically done inside a cloud function, but I'm having troubles with 3.
Is this possible?
Are there any workaround or future plans to support 3?
I'm thinking about two options: deploy a function from a function, or allow RTDB triggers to apply to every instances.
As you can check here in the management API documentation, the response body of the create method returns a DatabaseInstance object which has the following structure:
{
"name": string,
"project": string,
"databaseUrl": string,
"type": enum (DatabaseInstanceType),
"state": enum (State)
}
So, you can get the databaseUrl value, store it somewhere in your code and send it to your cloud function as a parameter later, assuming it is a http function. In the function all you have to do is use the following code to access it:
let app = admin.app();
let ref = app.database('YOUR_SECOND_INSTANCE_URL_HERE').ref();
EDIT
For triggered functions this is not possible, since you would need to know the instances names or URL when deploying to function to apply the trigger to them. If you already have the instances names you could try doing something like this community answer, although I am not sure it will suit your app's needs.
Related
I have been looking around for the ways to retrieve the bucket name in Firebase functions.
The documentation says you can do something like:
functions.storage.bucket("bucket_name").object()...
However, in all examples I have seen the "bucket name" is hard-coded. In my project, images are stored in the buckets named as user-ids. So when a write event is triggered, I want to retrieve this user id. Is there a way to do it? Something like this (below)?
exports.optimizeImages = functions.storage.bucket("{uid}").object().onFinalize(async (object) => {
const uid = ???
...
})
When you declare a storage trigger, you are only attaching it to a single bucket. If you want to trigger on multiple buckets, you have to declare multiple triggers. As such, each trigger function should always know which bucket it was fired for - you can simply hard coding it in the function implementation (it will be the same as what you specified in the function builder - just reuse that value).
If you must share the exact same function implementation with multiple triggers on multiple buckets, you can certainly parse the object.bucket property. That seems like a decent way to go.
I've a specific use case where I have multiple realtime DBs on a single project (and this number will grow) and I want to set up cloud functions triggers on all of them, currently I'm hoping if there's a way to get the DB name in the callback on which the cloud function is triggered?
import * as functions from 'firebase-functions';
import mongoose from 'mongoose';
export const updateData = functions.database.ref('/someendpoint/{code}').onUpdate(async (change, context) => {
$dbName = getFireBaseDBName(); //some function to get the DB name - This is the step that I would like to know how
await mongoose.connect(`mongo-db-string-connection/${dbName}`, {useNewUrlParser: true});
const Code = context.params.code;
const Schema = new mongoose.Schema({}, { collection: `someendpoint`, strict: false });
const Model = mongoose.model(`someendpoint`, Schema);
const after = change.after.val();
await Model.deleteMany({code: Code});
await Model.create({after, ...{code:Code}});
});
I need the DB name so that I can save to the database with the same name on Mongo.
For example:
Given I have a firebase project 'My-Project' and I have multiple Realtime Database instances on them say:
'db1', 'db2', 'db3'
When the trigger fires, I want to save/update/delete the data in MongoDB database so that it stays in sync with my Firebase Realtime database.
So it's crucial that not only do I get the data stored in db1 but also I get the name 'db1' so that the right data can be altered in Mongo.
Please keep in mind that more databases will be added to My-Project so
somewhere down the line it'll be 'db100.
First thing - I'll say that the way you're using database shards for multi-tenancy isn't really the way they're meant to be used. The Firebase team recommends using separate projects for multi-tenancy, one for each tenant, in order to keep users and their data isolated. The reason that database shards exist is to help developers deal with the scaling limitations of Realtime Database.
All that said, the triggers for Realtime Database don't directly provide the name of the shard in the callback. You will need to write one function for each shard, as required by the API, and described in the documentation.
To control when and where your function should trigger, call ref(path)
to specify a path, and optionally specify a database instance with
instance('INSTANCE_NAME'). If you do not specify an instance, the
function deploys to the default database instance for the Firebase
project For example:
Default database instance: functions.database.ref('/foo/bar')
Instance named "my-app-db-2": functions.database.instance('my-app-db-2').ref('/foo/bar')
Since you have to hard code the name of the shard in the function code, the you might as well just type it again inside the function itself. Or put them in global variables, and use them inside each function.
If you want to see an example of how to share code between each function declared for each instance, read this other question: How to trigger firebase function on all database instances rather than default one?
I am using the firebase-tools shell CLI to test Firestore cloud functions.
My functions respond to the onCreate trigger for all documents in a certain collection, by using a wildcard, and then mutate that document with an update call.
firestore
.document(`myCollection/{documentId}`)
.onCreate(event => {
const ref = event.data.ref
return ref.update({ some: "mutation"})
})
In the shell I run something like this, (passing some fake auth data required by my database permissions):
myFunction({some: "data"}, { auth: { variable: { uid: "jj5BpbX2PxU7fQn87z10d4Ks6oA3" } } } )
Hoever this results in an error, because the update tries to mutate a document that is not in the database.
Error: no entity to update
In the documentation about unit testing it is explained how you would create mocks for event.data in order to execute the function without touching the actual database.
However I am trying to invoke a real function which should operate on the database. A mock would not make sense, otherwise this is nothing more then a unit test.
I'm wondering what the strategy should be for invoking a function like this?
By using an existing id of a document the function can execute successfully, but this seems cumbersome because you need look it up in the database for every test, and it might not be there anymore at some point.
I think it would be very helpful if the shell would somehow create a new document from the data you pass in, and run the trigger from that. Would this be possible maybe, or is there another way?
The Cloud Functions emulator can only emulate events that could happen within your project. It doesn't emulate the actual change to the database that would have triggered it.
As you're discovering, when your function depends on that actual change previously occurring, you can run into problems. The fact of the matter is that it's entirely possible that the created document may have already been deleted by the time you're handling the event in the function (imagine a user acts quickly to delete, but the event is delayed for whatever reason).
All that said, perhaps you want to use set() with SetOptions that indicate you want to merge instead of overwrite. Bear in mind that if the document was previously deleted (with good reason) before the event triggered, you'll unconditionally recreate the document, which may not be what the user wanted.
I am in a situation where I need to read the Firebase Realtime Database with a "cron" job.
I analysed and played with this repository, but I can't seem to understand how I can simply retrieve the list of ALL of the users or all of the data in the Realtime Database. The only function of the functions.database.ref is onWrite, which doesn't help me.
I know that the Cloud Functions are event-oriented and they were written to respond to some triggers etc, but as far as I here, this is not impossible and we can apparently access the data.
In their 'delete accounts' example, they used this:
https://www.googleapis.com/identitytoolkit/v3/relyingparty/downloadAccount?fields=users/localId,users/lastLoginAt,nextPageToken&access_token=
And it seems quite undocumented and not at all explained. Why do we have to use the identity toolkit? How could I simply READ data? What I've tried to do until the moment:
Use functions.database.ref('users').once('value', function(snapshot){})and it didn't work
Use their cron job example code (identity toolkit), but I get a highly strange response in the logs from Firebase: Missing required header: Metadata-Flavor and the rest of the tracking below:
You can't use functions.database.ref() for reading data. That's only for specifying the path where you want to changes to the database to trigger your function. To write a cron-like function, you're not going to write a database trigger. You probably want an HTTP trigger instead, as shown in the sample code. You trigger than HTTP function with a third party scheduler as described in the readme.
To read data from the database in that function, you should use the admin SDK, as shown in that code sample. It's initialized here, and appears in lots of other code samples as well:
const admin = require('firebase-admin');
admin.initializeApp(functions.config().firebase);
Then you can get a reference to the database:
const ref = admin.database().ref(`/path/to/data`);
ref here is a Reference object. You can read data with that. Be sure to look at the other code samples too - many of them read a database with the admin SDK like this.
It's a known issue that firebase doesn't have easy way to count items. I'm planning to create an app that relies heavily on counts and other aggregates. I fear creating this app's counters with the rules as suggested here will be incredibly complex and hard to maintain.
So I thought about this pattern:
I will keep a server that will listen to all items entered in the database and this server will update all counters and aggregates. The server will hold the UID of a special admin that only he can update counters.
This way, users will not have to download entire nodes in order to get a count, plus I won't have to deal with issues that arise from maintaining counters by clients.
Does this pattern make sense? Am I missing something?
Firebase has recently released Cloud Functions. As mentioned on the documentation:
Cloud Functions is a hosted, private, and scalable Node.js environment
where you can run JavaScript code.
With Cloud Functions, you don't need to create your own Server. You can simply write JavaScript functions and upload it to Firebase. Firebase will be responsible for triggering functions whenever an event occurs.
For example, let's say you want to count the number of likes in a post. You should have a structure similar to this one:
{
"Posts" : {
"randomKey" : {
"likes_count":5,
"likes" : {
"userX" : true,
"userY" : true,
"userZ" : true,
...
}
}
}
}
And your JavaScript function would be written like this:
const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp(functions.config().firebase);
// Keeps track of the length of the 'likes' child list in a separate attribute.
exports.countlikes = functions.database.ref('/posts/$postid/likes').onWrite(event => {
return event.data.ref.parent().child('likes_count').set(event.data.numChildren());
});
This code increases the likes_count variable every time there is a new write on the likes node.
This sample is available on GitHub.