Firebase CronJobs - how to allow http triggers from one cronjob site - firebase

Since Firebase (Real-time DB) does not support cron-jobs I am using cron-job.org to schedule http triggers. This function should fire once at the end of every day, so at around 11:55pm. The thing is I would like only cron-job.org to be able to trigger the http endpoint and no one else (e.g. someone malicious trying to trigger it many times a day). How can I achieve this in my cloud function?
I have setup the cronjob and this is all the code I have right now:
exports.findAndDeleteSevenDayJobs = functions.https.onRequest((req, res) => {
console.log('req',req);
});
Also, cron-job.org offers this:
And I have no idea how to use it.

To create cron-jobs in firebase RDB use a third party service like cron-job.org
1) CREATING THE KEY
To make everything secure you have to generate a secure key, from now on called YourSelfGeneratedButSecureKey.
You can generate one in your terminal by typing: node -e "console.log(require('crypto').randomBytes(20).toString('hex'))"
2) CREATING CRON JOB
Create a new cron-job that will hit your cloud function end-point and attach the created key as a url-query like so:
https://{projectSpecific}.cloudfunctions.net/{nameOfFunction}?key={YourSelfGeneratedButSecureKey}
Setup the key into your env by using the following command in your terminal:
firebase functions:config:set cron.key="{YourSelfGeneratedButSecureKey}"
3) CLOUD FUNCTION
To make sure everything is maximum security you can install secure-compare by typing npm install --save secure-compare;
Then in your cloud function:
const secureCompare = require('secure-compare');
exports.{nameOfFunction} = functions.https.onRequest((req, res) => {
const { key } = req.query;
if (!secureCompare(key, functions.config().cron.key)) {
console.log('Key in request and in env do NOT match');
res.status(403).send('Security key does not match.');
return null;
}
// DO REPETITIVE STUFF SECURELY
});

Related

Deploy a Cloud Function to different Firebase projects

I have two firebase projects, one for development (project-dev) and one for production (project-prod).
I initialized firebase CLI for writing and deploying cloud functions linked to project-prod. Then I ran firebase use --add and added project-dev so that I can deploy the same function to both projects; I don't want to rewrite the same function twice.
At this point, I faced the problem. Into the function, I have to write something to the realtime database, but when I deploy the function to project-dev it writes to project-prod's database.
What I want to achieve is that the function has to refer to the database of the project that it is deployed to. So that I have one function and when it is deployed to project-dev it writes to project-dev's database and when it is deployed to project-prod it writes to project-prod's database.
Is it possible to achieve that? If not, what's the way to go?
EDIT
Function code:
exports.AddOrders= functions.https.onRequest(async (req, res) => {
async function addOrder(key, value) {
var ref = app.database().ref("/orders");
return ref.child(key).set(value);
}
var orders = req.body['orders'];
var promises = [];
for (var key in orders) {
promises.push(addOrder(key, orders[key]));
}
Promise.all(promises).then(
_ => {
res.sendStatus(200);
return null;
}
).catch(err => {
res.send(err);
})
});
(This function works fine, the problem is that it writes on the wrong database)
After you add a project with firebase use --add <projektName> you need to select it with firebase use <projectName>. Then you can deploy to the selected project with firebase deploy or firebase deploy --only functions.
This is the answer for anyone wondering what the problem was.
He was initialising the admin sdk with the credentials of a service account tied to the production project. The solution was to change it to admin.initializeApp() without passing any arguments. This made Firebase use the default credentials for each project.
I know its a common mistake but again, here is the link to the corresponding documentation https://firebase.google.com/docs/admin/setup

How to recursively delete collection in firestore?

I needed a way to quickly clear an entire firestore DB but couldn't find great documentation for how to do it. Eventually I found on stack overflow this answer (Clear Firestore database of all data?) but was a bit nervous about how long it would take to clear my entire DB of millions of documents, so I want a way to just recursively delete a collection at a time.
Background: I've been running some tests migrating large amounts of data from an old DB to firestore, and after each run I want a clean slate to work with in firestore. Do NOT use this on production data!
This is now documented via recursiveDelete function:
https://googleapis.dev/nodejs/firestore/latest/Firestore.html#recursiveDelete
Note that this is a relatively new feature, so you need to make sure your Firebase libraries are updated.
// Setup
const admin = require('firebase-admin');
const serviceAccount = require("./files/my-file.json");
admin.initializeApp({
credential: admin.credential.cert(serviceAccount)
});
const firestore = admin.firestore();
// Delete
const documentRef = firestore
.collection("users")
.doc("M3S2iPhsiu2ZQmOK8ZcC");
await firestore.recursiveDelete(documentRef);
I was able to do this by running the firestore cli delete command, specifying each collection by path. Make sure not to start the path with a leading slash, or it will think you are referring to a directory on your computer. Example of how to run:
firebase firestore:delete "path/to/collection" --recursive
The firestore:delete command is sort of documented here as well: https://firebase.google.com/docs/firestore/manage-data/delete-data#delete_data_with_the_firebase_cli
Update
Please note that the command may fail after deleting about 20000 documents. I think it might be a limit it hits. I just re-run with the same collection path a few times to fully delete collections that are larger than 20k docs.
From the Fireship website : https://fireship.io/snippets/delete-firestore-collection/ there is 2 options to delete collections in firestore :
Option 1 :
You can manually delete a collection or subcollection from the Firebase Console OR by using the CLI :
firebase firestore:delete path-to-delete
Option 2 :
It is possible to interact with Firebase Tools from a Cloud Function. This works especially well with Callable functions because you most certainly want to enforce some form of user authorization.
First, obtain CI token to authenticate firebase tools.
cd functions
npm i firebase-tools -D
firebase login:ci
# your_token
firebase functions:config:set ci_token='your_token'
The function should validate the user has permission to run the operation. If allowed, it runs the CLI command recursively on the collection and its nested subcollections.
const project = process.env.GCLOUD_PROJECT;
const token = functions.config().ci_token;
exports.deleteCollection = functions.runWith({ timeoutSeconds: 540})
.https.onCall((data, context) => {
const path = data.path;
const allowed = context.auth.uid === path.split('/')[0]; // TODO your own logic
if (!allowed) {
throw new functions.https.HttpsError(
'permission-denied',
'Hey, that is not cool buddy!'
);
}
return firebase_tools.firestore
.delete(path, {
project,
token,
recursive: true,
yes: true,
})
.then(() => ({ result: 'all done!' }));
});

Error: Could not load the default credentials (Firebase function to firestore)

I am attempting to write an onCall function for Firebase Cloud Functions that performs advanced querying tasks on a firestore database (i.e. checking a text query up against AutoML natural lang to get a category, etc) but I keep running into a problem trying to query the database from the function:
Error getting documents :: Error: Could not load the default credentials. Browse to https://cloud.google.com/docs/authentication/getting-started for more information.
at GoogleAuth.getApplicationDefaultAsync (/srv/node_modules/google-auth-library/build/src/auth/googleauth.js:161:19)
at <anonymous>
at process._tickDomainCallback (internal/process/next_tick.js:229:7)
Function:
const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp();
const db = admin.firestore();
exports.query = functions.https.onCall((data, context) => {
const text = data.text;
var results = [];
const promise = db.collection('providers').get()
promise.then((snapshot) => {
console.log('marker');
snapshot.forEach((doc) => {
results.push({id: doc.id, data: doc.data()});
});
console.log('yessir');
return {results: results};
}).catch((err) => {
console.log('Error getting documents :: ', err)
console.log('nosir');
return {results: "no results"};
});
});
Longer output:
Function execution started
Function execution took 8ms, finished with status code: 200
Error getting documents :: (etc, same error)
nosir
Example 2 (no change in running):
Function execution started
Function execution took 1200 ms, finished with status code: 200
marker
yessir
I can't figure out where this problem is coming from or how to resolve it.
Any help?
Regards.
What I first did to solve it was add my firebase admin sdk key to my project.
I downloaded it at
https://console.firebase.google.com/u/0/project/**YOUR_PROJECT_ID**/settings/serviceaccounts/adminsdk
then at admin.initializeApp(); I changed to:
admin.initializeApp({
credential: admin.credential.cert(require('../keys/admin.json'))
});
My folder structure is
├── key
│ ├── admin.json
├── src
│ ├── index.ts
HOWEVER, a better practice and safer approach, as some mentioned already:
You could use environment variables to store your credentials, this way you won't commit it to a repository such as Github, keep it safer from safety breaches and won´t make it hardcoded.
Depending on your project and where you'll deploy it there's a different way to do it.
There are many tutorials around on how to create and access env variables (like this one), but you could use a name it like the example below:
GOOGLE_APPLICATION_CREDENTIALS="/home/admin.json"
I had the same error "Could not load the default credentials".
The error occured after updating my project dependencies with npm update.
More precisely firebase-admin and firebase-functions.
Before update:
"dependencies": {
"#google-cloud/firestore": "^1.3.0",
"firebase-admin": "~7.0.0",
"firebase-functions": "^2.2.0"
}
After update:
"dependencies": {
"#google-cloud/firestore": "^1.3.0",
"firebase-admin": "^8.6.0",
"firebase-functions": "^3.3.0"
}
I added the serviceAccountKey.json to my project and changed the imports with the code provided at the service account setting of my firebase project.
From :
var admin = require('firebase-admin')
admin.initializeApp()
To:
var admin = require('firebase-admin');
var serviceAccount = require('path/to/serviceAccountKey.json');
admin.initializeApp({
credential: admin.credential.cert(serviceAccount),
databaseURL: 'https://my-project.firebaseio.com'
});
See #Fernando Rocha's answer below to access the account setting of your firebase project.
#aldobaie's answer helped me figure out what was going on for my use case. For those who are not looking to add async/await to all their calls, remember that the firestore calls return promises, so prepending them with return has the same effect.
In my case:
function doSomething(...) {
return admin.firestore().collection(...).doc(...).get()
.then((doc) => {...})
.catch(err => {...})
}
module.exports = functions.firestore.document('collection/{docId}').onWrite((change, context) => {
return doSomething()
})
I think the accepted answer goes against Firebase's recommend configuration. The function environment has access to admin credentials already, and passing your key in the code is not recommended.
I do it like this:
const functions = require('firebase-functions')
const admin = require('firebase-admin')
admin.initializeApp(functions.config().firebase)
I ran into the same problem myself. Sometimes the function works and many times it would through the Error: Could not load the default credentials error.
The problem I believe have been solved by watching for the Callbacks. You have to keep the function running until the callbacks have been called using the await and async prefixes.
Firebase Cloud Functions don't allow the access to the processor through callbacks once it's been terminated! That's why we get the Error: Could not load the default credentials error.
So, whenever you have a .then() function prefix it with await and prefix the function it's inside it with async and prefix any call to the function with await.
async function registerUser(..) {
...
await admin.firestore().collection(..)...
...
}
I hope this helps you out!
Another option is to set the service account key in an environmental variable instead of setting it with a call to firebaseAdmin.initializeApp({ credential }).
Linux
export GOOGLE_APPLICATION_CREDENTIALS="[PATH]"
export GOOGLE_APPLICATION_CREDENTIALS="/home/user/Downloads/[FILE_NAME].json"
Windows PowerShell
$env:GOOGLE_APPLICATION_CREDENTIALS="[PATH]"
$env:GOOGLE_APPLICATION_CREDENTIALS="C:\Users\username\Downloads\[FILE_NAME].json"
Postscript: An even better option might be to use the local emulator suite.
Alright, so I had this error as well and spent a frustrated few days going over multiple sites, articles, videos, etc to try and figure out the cause of this problem so I could get an adequate answer for both myself and everyone else who is struggling.
There are answers to this question in this thread already. However, I tried following most of them to no avail. Some have security issues and others are just too vague to follow. I decided to post a thorough answer which also addresses the security issues you would have if you followed some of the other answers.
Alright now that I've gotten that out of the way lets get started!
First of all your going to need to go to this link - Getting started with authentication
You should see this in the center of your screen -
Next, click on the button I've marked in green. This will bring you to the create service account key page.
You should see a similar screen to the below image -
For the Service Account option, select new service account.
Create a name for your service account. This is not important, name it whatever you like.
For the role option select Project -> Owner
Finally, select JSON option for key type and then hit create.
This should create and download a .json file. Place this file somewhere smart and safe. I created a folder called 'credentials' in the root of my project and placed it in there.
Also I renamed the file to something more readable. While this isn't necessary, following good file/folder naming and structuring practices is important and I would advise you to rename it to something more readable.
(Its important to note that this file is personal and should not be included in any github repositories/firebase production/etc. This file is for you and you alone!)
Next open a command prompt window and type in the following command -
set GOOGLE_APPLICATION_CREDENTIALS=C:\Users\Username\Path\To\File\filename.json
This will create an environment variable that is linked securely to your credentials which firebase will recognize and use when you make calls to authenticate yourself.
(Note - This is the command for windows. If your using mac/linux go to the 'Getting started with Authentication' page mentioned earlier to get the appropriate command for your operating system)
There you go, the issue should now be fixed. If anyone has any further questions or problems feel free to comment below and i'll do my very best to help you. I know how frustrating it can be to be stuck with an error like this.
I hope this helps someone at the very least. Happy Programming.
C.Gadd
I do not want to use #Fernando solution even though there is nothing wrong.
I have prd and non-prd environment. I use firebase use command to push the changes to the correct environment. When I deploy, firebase uses the default service account. Also I do not want to have the keys in the project folder or in my git repo.
The way I solved might not work for others, but want to share here.
The issue came to me when I updated the permission of the firebase project to give a viewer with editor permission. I made that person the owner and rolled back to editor. It went away. It is not justifying as a fix, but worked for me and I do not have to download the key.
Instead of setting serviceAccountKey.json file, you can first set .env values from it and then use those:
import * as firebaseAdmin from "firebase-admin";
const adminCredentials = {
credential: firebaseAdmin.credential.cert({
projectId: process.env.NEXT_PUBLIC_FIREBASE_PROJECT_ID,
clientEmail: process.env.FIREBASE_CLIENT_EMAIL,
privateKey: JSON.parse(process.env.FIREBASE_PRIVATE_KEY || ""),
}),
databaseURL: process.env.NEXT_PUBLIC_FIREBASE_DATABASE_URL,
};
if (!firebaseAdmin.apps.length) {
firebaseAdmin.initializeApp(adminCredentials);
}
const firestore = firebaseAdmin.firestore();
Old answer:
This is a known bug in Firebase. see the progress here: https://github.com/firebase/firebase-tools/issues/1940
However, meantime there are few options to resolve this:
1 Explicitly passed via code
var admin = require("firebase-admin");
var serviceAccount = require("path/to/serviceAccountKey.json");
admin.initializeApp({
credential: admin.credential.cert(serviceAccount),
databaseURL: "https://your-app.firebaseio.com"
});
Not recommended this hard-coding. This json file will not be accessible on server.
2 Passed via GOOGLE_APPLICATION_CREDENTIALS
I'd recommend this way, set environmental variable:
GOOGLE_APPLICATION_CREDENTIALS=path/to/serviceAccountKey.json
For windows: (considering json is at your root path of project.
using powershell:
$env:GOOGLE_APPLICATION_CREDENTIALS='serviceAccountKey.json'
using NPM script: (notice no space before &&)
"serve": "set GOOGLE_APPLICATION_CREDENTIALS=serviceAccountKey.json&& npm run start",
(for some reason cross-env didn't work)
3 Available at a well-known filesystem path due to gcloud
by installing gcloud sdk and running gcloud auth application-default login
4 Available from the Compute Engine metadata API when running on GCP
I had same problem in firebase Error: "Could not get default credentials."
Then go to firebase console and go to project setting, where you can find Service Accounts option. Click there and you will see the Generate new private key under your project setting.
Copy code for your project language and add it to your project file.
var admin = require("firebase-admin");
var serviceAccount = require("path/to/serviceAccountKey.json");
admin.initializeApp({
credential: admin.credential.cert(serviceAccount),
databaseURL: "https://your-database-url-that-is-given-under-admin-sdk-snippets"
});
After Generating the key you will have option to download. and put it in the project folder. Also set path var serviceAccount = require("path/to/serviceAccountKey.json");
That's it your are ready.
None of above.
You may just:
firebase login - It will open browser login
As soon as you do login, returnto console and run:
firebase init - It will run as successfull.
I had the same issue.
Go on your settings page on Firebase => Service and Account.
Firebase Setting 1. Parameters 2. Account 3. Download the file and rename it [admin.json]
Copy the code and paste it
Requires 'admin.json' and paste, and run Firebase deploy.
admin.initializeApp(functions.config().firebase);
also works.
This error can also occur when the cloud function is not terminated properly.
Whenever you write a cloud function make sure you return promise after the cloud function processing is over, so that cloud function knows that your process is complete.
If you don't return promise then there might be chances where your cloud function might terminate before the processing is complete.
You can refer this to know how to terminate the cloud function.
Terminate cloud functions
Download your firebase service account into your project and reference it like this:
<code>
var admin = require("firebase-admin");
var serviceAccount = require("path/to/serviceAccountKey.json");
admin.initializeApp({
credential: admin.credential.cert(serviceAccount),
databaseURL: "<database-url>"
});
</code>
For those who come here from a serp trying to figure out why their google cloud function fails with:
Error: Could not load the default credentials. Browse to
https://cloud.google.com/docs/authentication/getting-started for more
information. at GoogleAuth.getApplicationDefaultAsync
but none of the above helped, you can try to update all(?) of your #google/whatever dependencies:
npm i -E #google/firestore#latest. Then rebuild, deploy, try again. It happened to me a few times recently and this worked.
I just had the same problem. To solve it, just update your node packages by npm update inside your project-dir/functions/ directory. Finally, deploy again.
On MacOS I had to do the following:
export GOOGLE_APPLICATION_CREDENTIALS=/Users/myname/serviceAccountKey.json
I was getting credential error because the locally running functions emulator could not securely talk to firebase auth running in production.
Google Cloud Reference
For those who still get the same problem event after downloading account key and using it inside your code, make sure it is inside your functions folder.
One thing it's a bit difficult to find in the docs is the firebase-admin SDK only uses the emulators when environment variables tell it to. If you use the service account JSON key as described in some answers here, firebase-admin will talk to prod (on Google Cloud) rather than the emulated version, even if everything else you're doing is on the emulators.
Since most likely you would rather use the emulators for local testing, here's how I set my environment variables in Mac ~/.zshrc:
export GCLOUD_PROJECT="your-project-id"
export FIRESTORE_EMULATOR_HOST=localhost:8080
export FIREBASE_AUTH_EMULATOR_HOST=localhost:9099
export FIREBASE_DATABASE_EMULATOR_HOST=localhost:9000
The GCLOUD_PROJECT id could be your project id, but apparently any id will work as long as it is a well-formed Firebase project id, so these same environment variables can be used to test all your projects on the Firebase emulators. Try setting these environment variables first for emulator use before you try any of the other solutions.
Another oddity is firebase emulators:start needs these environment variables set, but firebase emulators:exec sets them automagically. When you are in a CI scenario :exec is the better choice, but when actively running tests as you write code having the emulators stay up and running with :start is a faster loop and you'll need the environment variables for it to work properly. By having these in environment variables, your code won't need to change at all when deployed to the Cloud.
I just had this issue and fixed it with
firebase login

Can Firebase RemoteConfig be accessed from cloud functions

I'm using Firebase as a simple game-server and have some settings that are relevant for both client and backend and would like to keep them in RemoteConfig for consistency, but not sure if I can access it from my cloud functions in a simple way (I don't consider going through the REST interface a "simple" way)
As far as I can tell there is no mention of it in the docs, so I guess it's not possible, but does anyone know for sure?
firebaser here
There is a public REST API that allows you to read and set Firebase Remote Config conditions. This API requires that you have full administrative access to the Firebase project, so must only be used on a trusted environment (such as your development machine, a server you control or Cloud Functions).
There is no public API to get Firebase Remote Config settings from a client environment at the moment. Sorry I don't have better news.
This is probably only included in newer versions of firebase (8th or 9th and above if I'm not mistaken).
// We first need to import remoteConfig function.
import { remoteConfig } from firebase-admin
// Then in your cloud function we use it to fetch our remote config values.
const remoteConfigTemplate = await remoteConfig().getTemplate().catch(e => {
// Your error handling if fetching fails...
}
// Next it is just matter of extracting the values, which is kinda convoluted,
// let's say you want to extract `game_version` field from remote config:
const gameVersion = remoteConfigTemplate.parameters.game_version.defaultValue.value
So parameters are always followed by the name of the field that you defined in Firebase console's remote config, in this example game_version.
It's a mouthful (or typeful) but that's how you get it.
Also note that if value is stored as JSON string, you will need to parse it before usage, commonly: JSON.parse(gameVersion).
Similar process is outlined in Firebase docs.

How can I get current projectId inside Google Cloud Functions?

The question is simple. I need projectId in order to establish connection insde Google Cloud Function. I found the documentation where it said, that projectId is optional parameter and will be checked from GCLOUD_PROJECT, but on deployed function it didn't work.
So it is the question now how can I get the projectId env variable in order to pass it as argument for Datastore connection instance, or what should be done to not pass this id and establish connection with datastore inside Google Cloud Function?
Update 1
I found that I actually can get variable from process.env.GCLOUD_PROJECT, just like any other env.variable.
But, know is the last question, is it actually possible to use #google-cloud/datastore without any configuration object?
You can get project ID as const projectId = process.env.GCP_PROJECT.
But for some reason the environmet variable "GCP_PROJECT" is not set when running the emulator. Instead the deprecated variable name "GCLOUD_PROJECT" is the one to select. So you might try them both.
const projectId = process.env.GCP_PROJECT || process.env.GCLOUD_PROJECT
The answer depends on which runtime you are using.
If your using Python 3.7 and Go 1.11 you are in luck. Use
process.env.GCP_PROJECT
If using any of the new runtimes, bad luck, either access the metadata server, or google suggests
Note: If your function/app requires one of the environment variables from
an older runtime, you can set the variable when deploying your
function. For example:
gcloud functions deploy envVarMemory \
--runtime nodejs10 \
--set-env-vars FUNCTION_MEMORY_MB=2Gi \
--memory 2Gi \
--trigger-http
If using terraform:
resource "google_cloudfunctions_function" "my-function" {
name = "my-function"
runtime = "nodejs16"
environment_variables = {
GCP_PROJECT = var.project_id
}
#...
}
Under Cloud Functions v2 public preview (Oct 2022), there's a section called built-in parameters that allows require of Project ID, Database URL and Storage bucket.
Presumably also Location ID ("Default GCP resource location") would be listed here, though that's not in the current documentation.
I did not get that to work.
However, this does:
const firebaseConfig = JSON.parse( process.env.FIREBASE_CONFIG );
const locationId_maybe = firebaseConfig.locationId || null;
const projectId = firebaseConfig.projectId;
const databaseURL = firebaseConfig.databaseURL;
The env.var. is always defined. locationId is defined only if there's an active Firebase project. This makes sense, since it needs access to the GCP project in the cloud.
Firebase CLI 11.15.0
As I wrote in update 1 we can get all the information about environment from the process.env variable.
And the second question about configuration object for #google-cloud/datastore, it actually can work without any options. It will trying to fetch all required parameters from environment variables. So, it didn't work beacuase of error in my code.
if it's an HTTP function, we can use the URL to figure out the current project and region with req.header('host')
For any runtimes after node 8 you can use https://www.npmjs.com/package/gcp-metadata
const gcpMetadata = require('gcp-metadata');
export const getGCPProjectIdFromMetadata = async () => {
const isAvailable = await gcpMetadata.isAvailable();
if (!isAvailable) {
return undefined;
}
return gcpMetadata.project('project-id');
};
This will also work across any other instances you wish to launch (such as app engine).
When having deployed with firebase deploy there's:
functions.config().firebase.projectId
functions.config().firebase.databaseURL
functions.config().firebase.storageBucket
functions.config().firebase.locationId

Resources