how to create refFromURL with admin privilege on cloud functions? - firebase

I want to have a reference to an image using its http URL when firestore update cloud function triggered so that i can take the url from change provide by onUpdate() function and use it to get a reference to the image on firebase storage and delete it.

In order to delete a file stored in Cloud Storage for Firebase from a Cloud Function you will need to create a File object based on:
The Bucket instance this file is attached to;
The name of the file,
and then call the delete() method
as detailed in the Node.js library documentation https://cloud.google.com/nodejs/docs/reference/storage/2.0.x/File.
Here is an example of code from the documentation:
const storage = new Storage();
const bucketName = 'Name of a bucket, e.g. my-bucket';
const filename = 'File to delete, e.g. file.txt';
// Deletes the file from the bucket
storage
.bucket(bucketName)
.file(filename)
.delete()
.then(() => {
console.log(`gs://${bucketName}/${filename} deleted.`);
})
.catch(err => {
console.error('ERROR:', err);
});
From your question, I understand that your app clients don't have the bucket and file names as such and only have a download URL (probably generated through getDownloadURL if it is a web app, or the similar method for other SDKs).
So the challenge is to derive the bucket and file names from a download URL.
If you look at the format of a download URL you will find that it is composed as follows:
https://firebasestorage.googleapis.com/v0/b/<your-project-id>.appspot.com/o/<your-bucket-name>%2F<your-file-name>?alt=media&token=<a-token-string>
So you just need to use a set of Javascript methods like indexOf(), substring() and/or slice() to extract the bucket and file names from the download URL.
Based on the above, your Cloud Function code could then look like:
const storage = new Storage();
.....
exports.deleteStorageFile = functions.firestore
.document('deletionRequests/{requestId}')
.onUpdate((change, context) => {
const newValue = change.after.data();
const downloadUrl = newValue.downloadUrl;
// extract the bucket and file names, for example through two dedicated Javascript functions
const fileBucket = getFileBucket(downloadUrl);
const fileName = getFileName(downloadUrl);
return storage
.bucket(fileBucket)
.file(fileName)
.delete()
});

Related

Firebase Storage with Google Actions

I am having some issues connecting my firebase storage with my google action. I need to be able to "download" the json files inside in order to be able to read and pick out what a user may need given data that they provide when they call the action.
Below is the code that I currently have, complied from the different APIs and other stackoverflow questions I have found.
const functions = require('firebase-functions');
const admin = require('firebase-admin');
const Firestore = require('#google-cloud/firestore');
const firestore = new Firestore();
var storage = require('#google-cloud/storage');
const gcs = storage({projectId: 'aur-healthcare-group'});
const bucket = gcs.bucket('gs://aur-healthcare-group');
admin.storage().bucket().file('aur-healthcare-group/aur_members.json').download(function(errr, contents){
if(!err){
var jsObjext = JSON.parse(contents.toString('utf8'));
}
});
The current error I am receiving is "code":3,"message":"Function failed on loading user code. This is likely due to a bug in the user code. Error message: Error: please examine your function logs to see the error cause. When I check the logs I only get the above mentioned message again.
I believe that I am not accessing my firebase storage correctly and have trouble finding a good resource on how to access this correctly. Would somebody be able to give me an example of how to access the storage correctly so I will be able to apply it to my project?
Since you're running in Firebase Functions, you shouldn't need to require the #google-cloud/storage dependency directly. Rather, you can get the correctly authenticated storage component via admin.storage()
Following that, you shouldn't download the file to your function, as you would be better off reading directly into memory via a readStream.
With regards to your existing code error, it may be because you're checking if (!err) when the callback variable is errr.
I've done this in the past and here's a code snippet of how I achieved it. It's written in Typescript specifically, but I think you should be able to port it to JS if you're using that directly.
import * as functions from 'firebase-functions';
import * as admin from 'firebase-admin'
import { Bucket } from '#google-cloud/storage';
admin.initializeApp()
const db = admin.firestore()
const bucket = admin.storage().bucket('project-id.appspot.com') // Use your project-id here.
const readFile = async (bucket: Bucket, fileName: string) => {
const stream = bucket.file(fileName).createReadStream();
return new Promise((resolve, reject) => {
let buffer = '';
stream.on('data', function(d: string) {
buffer += d;
}).on('end', function() {
resolve(buffer)
});
})
}
app.handle('my-intent-handler', async (conv) => {
const contents = await readArticle(bucket, 'filename.txt')
conv.add(`Your content is ${contents}`)
})
exports.fulfillment = functions.https.onRequest(app)

Generating a PDF when a document is created in Firebase Cloud Firestore

I'm developing an app that creates a PDF based on a web form.
I am currently attempting to use pdfmake to generate the PDFs based on a firestore document create trigger
import * as functions from 'firebase-functions';
const admin = require('firebase-admin);
admin.initializeApp();
const PdfPrinter = require('pdfmake');
const fs = require('fs');
export const createPDF = functions.firestore
.document('pdfs/{pdf}')
.onCreate(async (snap, context) => {
var pdfName = context.params.pdf;
var printer = new PdfPrinter();
var docDefinition = {
// Pdf Definitions
};
var options = {
// Pdf Options
};
var pdfDoc = printer.createPdfKitDocument(docDefinition, options);
pdfDoc.pipe(fs.createWriteStream('tempDoc.pdf'));
await pdfDoc.end();
// Upload to Firebase Storage
const bucket = admin.storage().bucket('myproject.appspot.com');
bucket.upload('tempDoc.pdf', {
destination: pdfName + '.pdf',
});
return fs.unlinkSync('document.pdf');
});
The trigger is called, however i get the error "Error: ENOENT: no such file or directory, stat 'document.pdf'"
I have tried it with the onCreate function being async and without.
Any help is greatly appreciated
It's not possible to write to any file location in Cloud Functions outside of /tmp. If your code needs to write a file, it should build paths off of os.tmpdir() as described in the documentation:
The only writeable part of the filesystem is the /tmp directory, which
you can use to store temporary files in a function instance. This is a
local disk mount point known as a "tmpfs" volume in which data written
to the volume is stored in memory. Note that it will consume memory
resources provisioned for the function.
The rest of the file system is read-only and accessible to the
function.

Cloud Functions: zip multiple documents from Cloud Storage

I already searched through a lot of questions on stack overflow but couldn't find a fitting answer from which I can derive the answer I need:
I want to zip multiple files from a folder within Google Cloud Storage/Firebase Storage with a Cloud Function.
I already found the solution for zipping documents from the local filesystem but could not derive how to do it within a Cloud Function for Cloud Storage.
Google Cloud Storage supports the decompressive form of transcoding but not a compressive form of transcoding. However, at Cloud Storage, any user can store a gzip-compressed file.
To zip multiple documents from Cloud Storage using Cloud Functions, you can download the files from Cloud Storage to functions instances using gcs.bucket.file(filePath). download, zip the file, and re-upload the files to the Cloud Storage. Here you will find an example of downloading, transforming, and uploading a file. You can find an example to zip multiple files in this StackOverflow thread. This document explains how you can upload objects to Cloud Storage using Console, Gsutil, Code Sample, or REST APIs.
A bit late, but I had the same problem to solve.
The following Firebase Function:
Runs with 1 GB / 120 seconds timeout (for good measure)
Is triggered by WRITE calls (do this only if you have few calls!)
Ignores all paths except background_thumbnail/
Creates a random working directory and deletes it afterwards
Downloads images from Firebase Storage
Zips these images in a folder: background_thumbnail/<IMAGE>
Uploads created ZIP to Firebase Storage
Creates a signed URL for the ZIP file at Firebase Storage
Stores the signed URL in Firestore.
The code can probably be improved and made more elegant, but it works (for now).
const {v4: uuidv4} = require("uuid"); // for random working dir
const JSZip = require("jszip");
exports.generateThumbnailZip = functions
.runWith({memory: "1GB", timeoutSeconds: 120})
.region("europe-west3")
.storage.object()
.onFinalize(async (object) => {
// background_thumbnail/ is the watched folder
if (!object.name.startsWith("background_thumbnail/")) {
return functions.logger.log(`Aborting, got: ${object.name}.`);
}
const jszip = new JSZip();
const bucket = admin.storage().bucket();
const fileDir = path.dirname(object.name);
const workingDir = path.join(os.tmpdir(), uuidv4());
const localZipPath = path.join(workingDir, `${fileDir}.zip`);
const remoteZipPath = `${fileDir}.zip`;
await mkdirp(workingDir);
// -------------------------------------------------------------------
// DOWNLOAD and ZIP
// -------------------------------------------------------------------
const [files] = await bucket.getFiles({prefix: `${fileDir}/`});
for (let index = 0; index < files.length; index++) {
const file = files[index];
const name = path.basename(file.name);
const tempFileName = path.join(workingDir, name);
functions.logger.log("Downloading tmp file", tempFileName);
await file.download({destination: tempFileName});
jszip.folder(fileDir).file(name, fs.readFileSync(tempFileName));
}
const content = await jszip.generateAsync({
type: "nodebuffer",
compression: "DEFLATE",
compressionOptions: { level: 9 }
});
functions.logger.log("Saving zip file", localZipPath);
fs.writeFileSync(localZipPath, content);
// -------------------------------------------------------------------
// UPLOAD ZIP
// -------------------------------------------------------------------
functions.logger.log("Uploading zip to storage at", remoteZipPath);
const uploadResponse = await bucket
.upload(path.resolve(localZipPath), {destination: remoteZipPath});
// -------------------------------------------------------------------
// GET SIGNED URL FOR ZIP AND STORE IT IN DB
// -------------------------------------------------------------------
functions.logger.log("Getting signed URLs.");
const signedResult = await uploadResponse[0].getSignedUrl({
action: "read",
expires: "03-01-2500",
});
const signedUrl = signedResult[0];
functions.logger.log("Storing signed URL in db", signedUrl);
// Stores the signed URL under "zips/<WATCHED DIR>.signedUrl"
await db.collection("zips").doc(fileDir).set({
signedUrl: signedUrl,
}, {merge: true});
// -------------------------------------------------------------------
// CLEAN UP
// -------------------------------------------------------------------
functions.logger.log("Unlinking working dir", workingDir);
fs.rmSync(workingDir, {recursive: true, force: true});
functions.logger.log("DONE");
return null;
});

Delete Firebase Storage image url with download url

I am using Firebase storage and Realtime Database for storing the image and its download url respectively.The filename is generated in a random manner with which download url is generated and saved to the realtime database.
Scenario:
If user uploads new Image(For e.g Profile Image) I want to delete the old image with the help of downloadImageurl(Download image url is generated when image is uploaded initially and same is saved in the realtime database).How the old image can be deleted?I have tried below code but for it to work I must get filename.
gcs
.bucket("e**********.appspot.com") // find it in Firebase>Storage>"gs://...." copy without gs
//or go to console.cloud.google.com/ buckets and copy name
.file("images/" +event.params.uid+"/"+filename) //file location in my storage
.delete()
.then(() => {
console.log(`gs://${bucketName}/${filename} deleted.`);
})
.catch(err => {
console.error('ERROR-DELETE:', err+ " filename: "+filename);
});
This may help you out.
This code will fetch the name of file from the URL and will delete that file. Currently this solution works for me!
Code
import * as firebase from 'firebase';
...
let name = imagePath.substr(imagePath.indexOf('%2F') + 3, (imagePath.indexOf('?')) - (imagePath.indexOf('%2F') + 3));
name = name.replace('%20',' ');
let storagePath = firebase.storage().ref();
storagePath.child(`images/${name}`).delete();
Depending on what you want:
Keep the original image and being able to delete it manually in the future.
Delete it immediately after the thumbnail is generated.
I suppose you are using this example
1- You have to store the filePath in your db. Then whenever you want to delete it from your front:
import * as firebase from 'firebase';
...
const store = firebase.storage().ref();
// Depending on which db you use and how you store, you get the filePath and delete it:
store.child(image.filePath).delete();
2- Continue the promise from the firebase function like this:
// ...LAST PART OF THE EXAMPLE...
.then(() => {
// Add the URLs to the Database
return admin.database().ref('images').push({path: fileUrl, thumbnail: thumbFileUrl});
}).then(() => {
// ...PART YOU CAN ADD TO DELETE THE IMAGE UPLOADED
const bucket = gcs.bucket(bucket);
bucket.file(filePath).delete();
})
"bucket" is the const previously created:
const bucket = gcs.bucket(event.data.bucket);
as well as "filePath":
const filePath = event.data.name;
To delete an image with the url, you can use refFromUrl() function to get the ref then delete it easily
const storage = firebase.storage();
storage.refFromURL(imageUrl).delete()
const downloadUrl = "https://firebasestorage.googleapis.com/v0/b/***.appspot.com/o/***?alt=media&token=***";
The first *** represents FIREBASE_STORAGE_BUCKET
The second *** represents location of file in bucket
The third *** represents token for public access image/file
As a web developer, you're aware that URI are encoded such as
"#" = "%40",
"$" = "%24",
" " = "%20", etc.
Since we are using JavaScript, what we can do is decode URI like so to get exact path
const path = decodeURIComponent(downloadUrl.split("o/")[1].split("?")[0]);
return await bucket
.file(path)
.delete()
.then(() => true)
.catch((error) => {
throw new TypeError(`deleteImages ${error}`);
});

How to call refFromURL in Firebase Cloud Function

I'm storing references to files in Firebase Cloud Storage using URLs. In firebase client code, you can call firebase.storage().refFromURL(photo.image) to get the actual storage reference and do handy things like call delete with it. How do I accomplish the same thing in a cloud function (specifically a realtime database trigger)? I want to be able to clean up images after deleting the object that references them.
Following Bob Snider's answer, this is a little function (typescript) to extract file full path from URL.
export const getFileFromURL = (fileURL: string): Promise<any> => {
const fSlashes = fileURL.split('/');
const fQuery = fSlashes[fSlashes.length - 1].split('?');
const segments = fQuery[0].split('%2F');
const fileName = segments.join('/');
return fileName;
}
In a cloud function, to delete a file from storage you need the file's bucket name and file name (which includes the path). Those can be obtained on the client side from the storage reference. For example, a JS Storage Reference has properties bucket and fullPath. The string representation of a storage reference has format: gs://example-12345.appspot.com/path/to/file, where the bucket is example-12345.appspot.com and the file "name" is path/to/file.
In the example cloud function shown below, the client is expected to provide the bucket and filename as children of the trigger location. You could also write the URL string to the trigger location and then split it into bucket and filename components in the cloud function.
This code is based on the example in the Cloud Storage guide.
const functions = require('firebase-functions');
const gcs = require('#google-cloud/storage')();
const admin = require('firebase-admin');
admin.initializeApp(functions.config().firebase);
exports.deleteFile = functions.database.ref('/test').onWrite(event => {
const bucket = event.data.child('bucket').val();
const filename = event.data.child('filename').val();
console.log('bucket=', bucket, 'filename=', filename);
return gcs.bucket(bucket).file(filename).delete().then(() => {
console.log(`gs://${bucket}/${filename} deleted.`);
}).catch((err) => {
console.error('ERROR:', err);
});
});
Here is a one-liner.
const refFromURL = (URL) => decodeURIComponent(URL.split('/').pop().split('?')[0])
I've wrote code sample which I using instead refFromURL method from web-firebase in my functions project based on Bob Snyder answer.
function refFromUrl(gsLink) {
var fileEntryTemp = gsLink.file.replace("gs://", "")
var bucketName = fileEntryTemp.substring(0, fileEntryTemp.indexOf("/"));
var filename = gsLink.file.match("gs://" + bucketName + "/" + "(.*)")[1];
var gsReference = admin.storage().bucket().file(filename);
return gsReference;
}
Here is an example how I get a download link based on this ref:
var gsReference = refFromUrl(fileEntry);
gsReference.getSignedUrl({
action: 'read',
expires: '03-09-2491'
}).then(function (url) {
console.log(url);
response.send(url);
}).catch(function (error) {
});
Hope this will save time for somebody
For complicated actions on your database from cloud functions you could use Admin SDK https://firebase.google.com/docs/database/admin/startFor the usage of Cloud Storage in Cloud Function check this out https://firebase.google.com/docs/functions/gcp-storage-eventsCloud Functions may not provide the same capability as client since Cloud Functions is beta for now and people are still working on it.

Resources