File upload with filestreams and Firebase cloud functions + cloud storage - firebase

I've written the following code for my QR file upload using firebase cloud functions
const functions = require('firebase-functions');
const qrcode = require('qrcode')
const admin = require('firebase-admin');
const spawn = require('child-process-promise').spawn;
const serviceAccount = require("./secret.json");
const gcs = require('#google-cloud/storage')();
admin.initializeApp(functions.config({
credential: admin.credential.cert(serviceAccount),
storageBucket: "https://SECRET.firebaseio.com"
}));
exports.qrcode = functions.https.onRequest((req, res) => {
const storage = admin.storage().bucket();
const dummyFile = storage.file('dummy.png')
new Promise ((resolve, reject) => qrcode.toFileStream(
dummyFile.createWriteStream()
.on('finish', resolve)
.on('error', reject),
'DummyQR'))
.then(console.log("success")) //Doing stuff here later
.catch(console.log)
res.send("<script>window.close();</script>")
});
According to the docs I should be able to connect to the bucket by simply calling admin.storage().bucket(); (https://firebase.google.com/docs/storage/admin/start) however I get the following error:
Error: Error occurred while parsing your function triggers.
Error: Bucket name not specified or invalid. Specify a valid bucket name via the storageBucket option when initializing the app, or specify the bucket name explicitly when calling the getBucket() method.
and so I'm stuck and not sure how to proceed. If I try to manually enter the bucket admin.storage().bucket("https://SECRET.firebaseio.com"); I get the error
{ ApiError: Not Found
at Object.parseHttpRespBody (/user_code/node_modules/firebase-admin/node_modules/#google-cloud/storage/node_modules/#google-cloud/common/src/util.js:193:30)
at Object.handleResp (/user_code/node_modules/firebase-admin/node_modules/#google-cloud/storage/node_modules/#google-cloud/common/src/util.js:131:18)
at /user_code/node_modules/firebase-admin/node_modules/#google-cloud/storage/node_modules/#google-cloud/common/src/util.js:496:12
at Request.onResponse [as _callback] (/user_code/node_modules/firebase-admin/node_modules/retry-request/index.js:195:7)
at Request.self.callback (/user_code/node_modules/firebase-admin/node_modules/request/request.js:185:22)
at emitTwo (events.js:106:13)
at Request.emit (events.js:191:7)
at Request.<anonymous> (/user_code/node_modules/firebase-admin/node_modules/request/request.js:1157:10)
at emitOne (events.js:96:13)
at Request.emit (events.js:188:7)
code: 404,
errors: [ { domain: 'global', reason: 'notFound', message: 'Not Found' } ],
response: undefined,
message: 'Not Found' }

I had this same problem. I just added the storageBucket name when initializing the app, and if you are using the getSignedUrl method you need to include a service account, otherwise that url it's gonna get expired after a week (as it was in my case).
const serviceAccount = require('/your/service/account/key/path');
admin.initializeApp({
credential: admin.credential.cert(serviceAccount),
storageBucket: "your-storage-bucket-name.appspot.com",
});
don't forget to update your firebase functions with
firebase deploy --only functions
on the command line
UPDATE 6 APRIL 2020
Signed URLs actually expire after 7 days. If you need URLs for more time than that, you should read this answer and this thread

It looks like you're not initializing the admin SDK correctly. Just call initializeApp with no parameters to get all the correct defaults:
admin.initializeApp();
This will use the default service account provided by Cloud Functions for your project. This account has permission to do most of what you need to do in your function without any additional configuration.

In my case it was similar issue to this one (Unable to use cloud storage without specifying storageBucket when initialising app with firebase-admin).
I had following setup:
const admin = require("firebase-admin")
admin.initializeApp()
export * from "./api/api"
export * from "./cron/cronJobs"
export * from "./posts/postsCloudFunctions"
Changed it to:
const admin = require("firebase-admin")
export * from "./api/api"
export * from "./cron/cronJobs"
export * from "./posts/postsCloudFunctions"
admin.initializeApp()
And it seems the issue is initializeApp() was not being called before accessing the storage bucket.

I have same issue while uploading to image to firebase storage:
Error: Bucket name not specified or invalid. Specify a valid bucket name via the storageBucket option when initializing the app, or specify the bucket name explicitly when calling the getBucket() method.
ndlers\users.js:130:8)
at Busboy.emit (events.js:189:13)
at Busboy.emit (E:\react-project\socialape-functions\functions\node_modules\busboy\lib\main.js:37:33)
at E:\react-project\socialape-functions\functions\node_modules\busboy\lib\types\multipart.js:52:13
at process._tickCallback (internal/process/next_tick.js:61:11)
Then i add storageBucket to admin function
var admin = require("firebase-admin");
var serviceAccount = require("your service account key file path);
admin.initializeApp({
credential: admin.credential.cert(serviceAccount),
databaseURL: "https://your-app-1234.firebaseio.com",
storageBucket: "your-storage-bucket-name given in firebase storage"
});
const db = admin.firestore();
module.exports = { admin, db };
After adding bucket to admin functions it works.

You should change
storageBucket: "your-storage-bucket-name given in firebase storage"
to
storageBucket: "<BUCKET_NAME>.appspot.com".
You can find BUCKET_NAME as the picture below:

Related

Google storage permissions error while generating signed url from cloud function

I'm attempting to use a Firebase Cloud Function to create signed download URLs for files stored in a Storage Bucket. Using the snippet below on my local machine, I'm able to access cloud storage and generate these URLs.
/* eslint-disable indent */
import * as functions from "firebase-functions";
import * as admin from "firebase-admin";
// eslint-disable-next-line #typescript-eslint/no-var-requires
const serviceAccount = require("./test-serviceAccount.json");
admin.initializeApp();
const storage = admin.storage();
const bucket = storage.bucket();
admin.initializeApp({
credential: admin.credential.cert(serviceAccount),
}, "firestore");
export const getFile = functions.https.onRequest(async (request, response) => {
const [files] = await bucket.getFiles();
const fileNames: string[] = [];
files.forEach(async (file) => {
console.log(file.name);
const url = await file.getSignedUrl(
{
version: "v2",
action: "read",
expires: Date.now() + 1000 * 60 * 60 * 24,
}
);
fileNames.push(String(url));
if (files.indexOf(file) === files.length - 1) {
response.send(JSON.stringify(fileNames));
}
});
});
However after deploying to Cloud Functions I get an error when I call the function saying:
Error: could not handle the request
and the following message is logged in the functions console:
Error: The caller does not have permission
at Gaxios._request (/workspace/node_modules/gaxios/build/src/gaxios.js:129:23)
at processTicksAndRejections (internal/process/task_queues.js:95:5)
at async Compute.requestAsync (/workspace/node_modules/google-auth-library/build/src/auth/oauth2client.js:368:18)
at async GoogleAuth.signBlob (/workspace/node_modules/google-auth-library/build/src/auth/googleauth.js:655:21)
at async sign (/workspace/node_modules/#google-cloud/storage/build/src/signer.js:97:35)
I've tried using and not using a .json service account key and made sure that the service account has permissions (it has Service Account Token Creator, Storage Admin, and Editor roles at the moment).
I also read this issue relating to the python SDK for storage, but it seems to have been resolved. The workaround mentioned in that issue (using a .json service account token) also didn't resolve the permissions errors.
After working with Firebase support - here's what worked for me:
import { initializeApp, applicationDefault } from 'firebase-admin/app';
initializeApp({
credential: applicationDefault(),
projectId: '<FIREBASE_PROJECT_ID>',
});
Specifying the projectId in the init call seems to have resolved the issue.
Signed url means it is signed for (or, accessible to) any particular user for particular span of time, maximum 7 days. If you trying to get it for unauthenticated users, it may show such error(s).
It's better to use getDownloadURL() for unauthenticated users. getSignedUrl() should to used for authenticated users only.

Uploading files from Firebase Cloud Functions to Cloud Storage

The documentation is too complex for me to understand. It shows how to download a file from Cloud Storage to Cloud Functions, manipulate the file, and then upload the new file to Cloud Storage. I just want to see the basic, minimum instructions for uploading a file from Cloud Functions to Cloud Storage. Why doesn't this work:
const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp();
exports.storage = functions.firestore.document('Test_Value').onUpdate((change, context) => {
var metadata = {
contentType: 'text',
};
admin.storage().ref().put( {'test': 'test'}, metadata)
.then(function() {
console.log("Document written.");
})
.catch(function(error) {
console.error(error);
})
});
The error message is admin.storage(...).ref is not a function. I'm guessing that firebase-admin includes Firestore but not Storage? Instead of firebase-admin should I use #google-cloud/storage? Why doesn't this work:
const functions = require('firebase-functions');
const admin = require('firebase-admin');
const {Storage} = require('#google-cloud/storage')();
const storage = new Storage();
admin.initializeApp();
exports.storage = functions.firestore.document('Test_Value').onUpdate((change, context) => {
storage.bucket().upload( {'test': 'test'} , {
metadata: {
contentType: 'text'
}
})
});
I can't even deploy this code, the error message is
Error parsing triggers: Cannot find module './clone.js'
Apparently a npm module dependency is missing? But the module isn't called clone.js? I tried requiring child-process-promise, path, os, and fs; none fixed the missing clone.js error.
Why does admin.initializeApp(); lack parameters, when in my index.html file I have:
firebase.initializeApp({
apiKey: 'swordfish',
authDomain: 'myapp.firebaseapp.com',
databaseURL: "https://myapp.firebaseio.com",
projectId: 'myapp',
storageBucket: "myapp.appspot.com"
});
Another issue I'm seeing:
npm list -g --depth=0
/Users/TDK/.nvm/versions/node/v6.11.2/lib
├── child_process#1.0.2
├── UNMET PEER DEPENDENCY error: ENOENT: no such file or directory, open '/Users/TDK/.nvm/versions/node/v6.11.2/lib/node_modules/firebase-admin/package.json
├── firebase-functions#2.1.0
├── firebase-tools#6.0.1
├── firestore-backup-restore#1.3.1
├── fs#0.0.2
├── npm#6.4.1
├── npm-check#5.9.0
├── protractor#5.4.1
├── request#2.88.0
└── watson-developer-cloud#3.13.0
In other words, there's something wrong with firebase-admin, or with Node 6.11.2. Should I use a Node Version Manager to revert to an older version of Node?
Go to https://console.cloud.google.com/iam-admin/iam
Click the pencil icon next to your App Engine default service account
+ ADD ANOTHER ROLE
Add Cloud Functions Service Agent
In my specific use case, I needed to decode a base64 string into a byte array and then use that to save the image.
var serviceAccount = require("./../serviceAccountKey.json");
import * as functions from 'firebase-functions';
import * as admin from 'firebase-admin';
admin.initializeApp({
projectId: serviceAccount.project_id,
credential: admin.credential.cert(serviceAccount),
databaseURL: "https://your_project_id_here.firebaseio.com", //update this
storageBucket: "your_bucket_name_here.appspot.com" //update this
});
function uploadProfileImage(imageBytes64Str: string): Promise<any> {
const bucket = admin.storage().bucket()
const imageBuffer = Buffer.from(imageBytes64Str, 'base64')
const imageByteArray = new Uint8Array(imageBuffer);
const file = bucket.file(`images/profile_photo.png`);
const options = { resumable: false, metadata: { contentType: "image/jpg" } }
//options may not be necessary
return file.save(imageByteArray, options)
.then(stuff => {
return file.getSignedUrl({
action: 'read',
expires: '03-09-2500'
})
})
.then(urls => {
const url = urls[0];
console.log(`Image url = ${url}`)
return url
})
.catch(err => {
console.log(`Unable to upload image ${err}`)
})
}
Then you can call the method like this and chain the calls.
uploadProfileImage(image_bytes_here)
.then(url => {
//Do stuff with the url here
})
Note: You must initialize admin with a service account and specify the default bucket. If you simply do admin.initializeApp() then your image urls will expire in 10 days.
Steps to properly use a service account.
Go to Service Accounts and generate a private key
Put the JSON file in your functions folder (next to src and node_modules)
Go to Storage and copy the URL not including the "gs://" in the front. Use this for the storage bucket url when initializing admin.
Use your project ID above for the database URL.
See Introduction to the Admin Cloud Storage
API for further
details on how to use the Cloud Storage service in Firebase Admin SDK.
var admin = require("firebase-admin");
var serviceAccount = require("path/to/serviceAccountKey.json");
admin.initializeApp({
credential: admin.credential.cert(serviceAccount),
storageBucket: "<BUCKET_NAME>.appspot.com"
});
var bucket = admin.storage().bucket();
// 'bucket' is an object defined in the #google-cloud/storage library.
// See https://googlecloudplatform.github.io/google-cloud-node/#/docs/storage/latest/storage/bucket
// for more details.
Regarding uploading objects, see Cloud Storage Documentation Uploading Objects sample code:
// Imports the Google Cloud client library
const {Storage} = require('#google-cloud/storage');
// Creates a client
const storage = new Storage();
/**
* TODO(developer): Uncomment the following lines before running the sample.
*/
// const bucketName = 'Name of a bucket, e.g. my-bucket';
// const filename = 'Local file to upload, e.g. ./local/path/to/file.txt';
// Uploads a local file to the bucket
await storage.bucket(bucketName).upload(filename, {
// Support for HTTP requests made with `Accept-Encoding: gzip`
gzip: true,
metadata: {
// Enable long-lived HTTP caching headers
// Use only if the contents of the file will never change
// (If the contents will change, use cacheControl: 'no-cache')
cacheControl: 'public, max-age=31536000',
},
});
console.log(`${filename} uploaded to ${bucketName}.`);
I uploaded a file from my hard drive to Firebase Cloud Storage via Google Cloud Functions. First, I found the documentation for Google Cloud Functions bucket.upload.
const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp();
exports.Storage = functions.firestore.document('Storage_Value').onUpdate((change, context) => {
const {Storage} = require('#google-cloud/storage');
const storage = new Storage();
const bucket = storage.bucket('myapp.appspot.com');
const options = {
destination: 'Test_Folder/hello_world.dog'
};
bucket.upload('hello_world.ogg', options).then(function(data) {
const file = data[0];
});
return 0;
});
The first three lines are Cloud Functions boilerplate. The next line
exports.Storage = functions.firestore.document('Storage_Value').onUpdate((change, context) => {
creates the Cloud Function and sets the trigger. The next three lines are more Google Cloud boilerplate.
The rest of the code locates the file hello_world.ogg on my computer's hard drive in the functions folder of my project directory and uploads it to the directory Test_Folder and changes the name of the file to hello_world.dog in my Firebase Cloud Storage. This returns a promise, and the next line const file = data[0]; is unnecessary unless you want to do something else with the file.
Lastly we return 0;. This line does nothing except prevent the error message
Function returned undefined, expected Promise or Value
if (req.rawBody) {
busboy.end(req.rawBody);
}
else {
req.pipe(busboy);
}
As described in this issue: https://github.com/GoogleCloudPlatform/cloud-functions-emulator/issues/161#issuecomment-376563784

Firebase Functions 1.0.0 migration: Trouble with custom initializeApp() with Google Service Account credential

I just updated from the beta (v0.9.1) to v1.0.0, and ran into some problems with initialization. As per the migration guide, functions.config().firebase is now deprecated. Here is my initialization before:
const serviceAccount = require('../service-account.json')
admin.initializeApp({
credential: admin.credential.cert(serviceAccount),
databaseURL: functions.config().firebase.databaseURL,
storageBucket: functions.config().firebase.storageBucket,
projectId: functions.config().firebase.projectId,
})
Here is what I changed it to after upgrading:
admin.initializeApp()
At first this seems to work fine, the databaseURL and storageBucket are being recognized. However, there is a problem regarding the newly absent credential field. That field is required in order to access Google Cloud Storage. I access the #google-cloud/storage API in my app like so:
import { Bucket, File } from '#google-cloud/storage'
import * as admin from 'firebase-admin'
bucket: Bucket = admin.storage().bucket()
this.bucket
.upload(filepath, {
destination: uploadPath,
})
.then((fileTuple: [File]) => {
// after uploading, save a reference to the audio file in the DB
fileTuple[0]
.getSignedUrl({ action: 'read', expires: '03-17-2025' })
.then((url: [string]) => {
dbRef.child('audioFiles').push(url[0])
})
.catch(err => console.log('Error:', err))
console.log(`${filepath} uploaded to ${this.bucket.name}.`)
})
This uploads a file to storage, then it inserts the signed URL into my Realtime Database.
After migration, this chunk of code gives the following error:
SigningError: Cannot sign data without `client_email`
Since client_email was a field on service-account.json, I suspect that I need to reinsert credentials, like so:
admin.initializeApp({
credential: admin.credential.cert(serviceAccount),
})
When I try this, however, I get errors like Error: Can't determine Firebase Database URL. or Error: Bucket name not specified or invalid.. So if I manually insert credential, it appears to expect databaseURL and storageBucket to be manually inserted as well.
So the question is, how do I do this?
Figured it out myself. I had to rewrite my init call like this:
const serviceAccount = require('../service-account.json')
const firebaseConfig = JSON.parse(process.env.FIREBASE_CONFIG)
admin.initializeApp({
credential: admin.credential.cert(serviceAccount),
databaseURL: firebaseConfig.databaseURL,
storageBucket: firebaseConfig.storageBucket,
projectId: firebaseConfig.projectId,
})

firebase functions deployment error: Insufficient permissions to (re)configure a trigger

When I deploy storage trigger functions I get the following error msg and the deployment fails:
Deployment error.
Insufficient permissions to (re)configure a trigger (permission denied for bucket Bucket-Name. Please, give owner permissions to the editor role of the bucket and try again.
Firestore trigger functions can be deployed. I already tried to fix the ACLs using the commands suggested here and here.
Any ideas how to fix that?
Edit: I set up a new project, facing the same error.
My /functions/index.js:
const functions = require('firebase-functions');
const admin = require('firebase-admin');
const serviceAccount = require("./serviceAccountKey.json");
admin.initializeApp({
credential: admin.credential.cert(serviceAccount),
databaseURL: "https://<project-name>.firebaseio.com"
});
const storage = admin.storage();
exports.helloWorld = functions.https.onRequest((request, response) => {
response.send("Im deployed successfully");
});
exports.storageTrigger = functions.storage
.object()
.onChange(event => console.log('I fail to deploy'));

Firebase: Update Firestore Fails If Document Does Not Exist [duplicate]

This is my very basic Cloud Function:
const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp(functions.config().firebase);
const db = admin.firestore()
exports.createdNewAccount = functions.auth.user().onCreate(event => {
return db.collection('users').doc(event.data.uid).update({
"creationDate" : Date.now()
})
})
And I get the error
Error: no entity to update: app
What is wrong with my code?
Most likely, the document for event.data.uid does not exist. The documentation for update() states:
The update will fail if applied to a document that does not exist.
Use set() instead.
I faced a similar error when testing my app locally with the Firebase emulator. My firebase config file looked like:
import firebase from "firebase";
import "firebase/firestore";
const firebaseConfig = {
apiKey: <FIREBASE_API_KEY>,
authDomain: <FIREBASE_AUTH_DOMAIN>,
databaseURL: <FIREBASE_DB_URL>,
projectId: <FIREBASE_PROJECT_ID>,
storageBucket: <FIREBASE_STORAGE_BUCKET>,
messagingSenderId: <FIREBASE_MSG_SENDER_ID>,
appId: <FIREBASE_APP_ID>,
measurementId: <FIREBASE_MEASUREMENT_ID>,
};
// Initialize Firebase
const firebaseApp = firebase.initializeApp(firebaseConfig);
// for local instances, use the emulator to connect the frontend to firestore & functions
if (location.hostname === "localhost") {
firebase.firestore().settings({
host: "localhost:8080",
ssl: false,
});
firebase.functions().useFunctionsEmulator("http://localhost:5001");
}
Turns out my local Firestore database (expected to be running at localhost:8080) wasn't being hit. So my cloud functions were trying to write to a non-existent db. The underlying issue was that I also had my backend initializing to a different database:
// functions/index.js
const admin = require("firebase-admin");
var serviceAccount = require("./serviceAccount.json");
admin.initializeApp({
credential: admin.credential.cert(serviceAccount),
databaseURL: "https://other-firebase-project-id.firebaseio.com",
});
The solution (originally adapted from a Fireship tutorial) was to remove this [incorrect] re-initialization of the database altogether:
// functions/index.js
...
admin.initializeApp();
...
After all, according to the docs, we can initialize the Firebase Admin SDK without parameters since the FIREBASE_CONFIG environment variable is included automatically in Cloud Functions for Firebase functions that are deployed via the Firebase CLI.
FWIW, also be sure to set the correct Firebase project on the CLI. Doing so with this allows the Firebase CLI to hook Functions/Firestore to the right project:
firebase use <desired-firebase-project-id>
If you have multiple Firebase projects, you can list them out with: firebase projects:list

Resources