I am writing a web service where users can upload and download files to their user directory in Google Cloud Firebase Storage.
Imagine a user uploads a 1GB file to the storage. While the user uploads the file, it is already visible by other "processes".
What are common techniques to identify a file being uploaded 100% from a process that does not have the upload stats?
One approach in a local environment would be to call the file first "my-file.iso.tmp" and later rename them by removing the suffix, since a rename operation is atomic. But that doesn't seem to be a suitable solution for my Firebase Storage problem.
Any insights are highly appreciated!
There isn't any method to rename a file in Firebase storage. If you want to show realtime upload stats across all user's devices then using realtime database could be a way.
var uploadTask = storageRef.child('images/rivers.jpg').put(file);
uploadTask.on('state_changed',
async (snapshot) => {
var progress = (snapshot.bytesTransferred / snapshot.totalBytes) * 100;
console.log('Upload is ' + progress + '% done');
await updateUploadStatus(progress)
}
},
(error) => {
// Handle unsuccessful uploads
},
() => {
// Upload completed
await uploadStatus(100)
uploadTask.snapshot.ref.getDownloadURL().then((downloadURL) => {
console.log('File available at', downloadURL);
});
}
);
async function updateUploadStatus(progress) {
const dbRef = firebase.database().ref(`users/${userID}/${fileID}`)
await dbRef.update(progress)
}
The updateUploadStatus will update progress to realtime database and you can listen it on all other devices where user has logged in (and is active) as follows:
var fileRef = firebase.database().ref(`users/${userID}/${fileID}`);
fileRef.on('value', (snapshot) => {
const data = snapshot.val();
updateFileProgress(data);
});
It's upto you how you get that fileID on other devices or you could listen to users/${userID} node itself. While the progress is not 100 percent you can grey out that file icon or something like that.
Related
I am new to flutter and I am clueless on how to implement a function to download a PDF file from firebase. I went through multiple tutorials regarding downloading files from cloud storage to local storage in flutter app and still unable to make it work.
I want when user presses the button
onPressed: () async {},
Then, it will straight away download the file from my firebase cloud storage to their local files.
I know I have to use writeToFile, but how do I implement the function?
Hope you can help me, thanks!
When looking at the docs for Firebase Cloud Storage, you can download a file from the Cloud Storage to the local documents folder, by using the following function:
//You'll need the Path provider package to get the local documents folder.
import 'package:path_provider/path_provider.dart';
Future<void> downloadFileExample() async {
//First you get the documents folder location on the device...
Directory appDocDir = await getApplicationDocumentsDirectory();
//Here you'll specify the file it should be saved as
File downloadToFile = File('${appDocDir.path}/downloaded-pdf.pdf');
//Here you'll specify the file it should download from Cloud Storage
String fileToDownload = 'uploads/uploaded-pdf.pdf';
//Now you can try to download the specified file, and write it to the downloadToFile.
try {
await firebase_storage.FirebaseStorage.instance
.ref(fileToDownload)
.writeToFile(downloadToFile);
} on firebase_core.FirebaseException catch (e) {
// e.g, e.code == 'canceled'
print('Download error: $e');
}
}
You can find this, and read more about it on this page: https://firebase.flutter.dev/docs/storage/usage
To implement it on the onPressed funtion, i would suggest having the function in the same class as you have the button, and executing the function from the onPressed like this:
onPressed: () {
downloadFileExample();
}
I already searched through a lot of questions on stack overflow but couldn't find a fitting answer from which I can derive the answer I need:
I want to zip multiple files from a folder within Google Cloud Storage/Firebase Storage with a Cloud Function.
I already found the solution for zipping documents from the local filesystem but could not derive how to do it within a Cloud Function for Cloud Storage.
Google Cloud Storage supports the decompressive form of transcoding but not a compressive form of transcoding. However, at Cloud Storage, any user can store a gzip-compressed file.
To zip multiple documents from Cloud Storage using Cloud Functions, you can download the files from Cloud Storage to functions instances using gcs.bucket.file(filePath). download, zip the file, and re-upload the files to the Cloud Storage. Here you will find an example of downloading, transforming, and uploading a file. You can find an example to zip multiple files in this StackOverflow thread. This document explains how you can upload objects to Cloud Storage using Console, Gsutil, Code Sample, or REST APIs.
A bit late, but I had the same problem to solve.
The following Firebase Function:
Runs with 1 GB / 120 seconds timeout (for good measure)
Is triggered by WRITE calls (do this only if you have few calls!)
Ignores all paths except background_thumbnail/
Creates a random working directory and deletes it afterwards
Downloads images from Firebase Storage
Zips these images in a folder: background_thumbnail/<IMAGE>
Uploads created ZIP to Firebase Storage
Creates a signed URL for the ZIP file at Firebase Storage
Stores the signed URL in Firestore.
The code can probably be improved and made more elegant, but it works (for now).
const {v4: uuidv4} = require("uuid"); // for random working dir
const JSZip = require("jszip");
exports.generateThumbnailZip = functions
.runWith({memory: "1GB", timeoutSeconds: 120})
.region("europe-west3")
.storage.object()
.onFinalize(async (object) => {
// background_thumbnail/ is the watched folder
if (!object.name.startsWith("background_thumbnail/")) {
return functions.logger.log(`Aborting, got: ${object.name}.`);
}
const jszip = new JSZip();
const bucket = admin.storage().bucket();
const fileDir = path.dirname(object.name);
const workingDir = path.join(os.tmpdir(), uuidv4());
const localZipPath = path.join(workingDir, `${fileDir}.zip`);
const remoteZipPath = `${fileDir}.zip`;
await mkdirp(workingDir);
// -------------------------------------------------------------------
// DOWNLOAD and ZIP
// -------------------------------------------------------------------
const [files] = await bucket.getFiles({prefix: `${fileDir}/`});
for (let index = 0; index < files.length; index++) {
const file = files[index];
const name = path.basename(file.name);
const tempFileName = path.join(workingDir, name);
functions.logger.log("Downloading tmp file", tempFileName);
await file.download({destination: tempFileName});
jszip.folder(fileDir).file(name, fs.readFileSync(tempFileName));
}
const content = await jszip.generateAsync({
type: "nodebuffer",
compression: "DEFLATE",
compressionOptions: { level: 9 }
});
functions.logger.log("Saving zip file", localZipPath);
fs.writeFileSync(localZipPath, content);
// -------------------------------------------------------------------
// UPLOAD ZIP
// -------------------------------------------------------------------
functions.logger.log("Uploading zip to storage at", remoteZipPath);
const uploadResponse = await bucket
.upload(path.resolve(localZipPath), {destination: remoteZipPath});
// -------------------------------------------------------------------
// GET SIGNED URL FOR ZIP AND STORE IT IN DB
// -------------------------------------------------------------------
functions.logger.log("Getting signed URLs.");
const signedResult = await uploadResponse[0].getSignedUrl({
action: "read",
expires: "03-01-2500",
});
const signedUrl = signedResult[0];
functions.logger.log("Storing signed URL in db", signedUrl);
// Stores the signed URL under "zips/<WATCHED DIR>.signedUrl"
await db.collection("zips").doc(fileDir).set({
signedUrl: signedUrl,
}, {merge: true});
// -------------------------------------------------------------------
// CLEAN UP
// -------------------------------------------------------------------
functions.logger.log("Unlinking working dir", workingDir);
fs.rmSync(workingDir, {recursive: true, force: true});
functions.logger.log("DONE");
return null;
});
So I am very new to the whole coding scene and am trying to learn how to code using react native. Right now, I'm trying to figure out how to upload images using firebase (functions)and google cloud storage.
Below is the backend code that enables me to upload one image per submission to firebase.
I was wondering is it possible to modify this code so that it can upload multiple images per submission? If so, how would I go about doing it?
exports.storeImage = functions.https.onRequest((request, response) => {
return cors(request, response, () => {
const body = JSON.parse(request.body);
fs.writeFileSync("/tmp/uploaded-image.jpg", body.image, "base64", err => {
console.log(err);
return response.status(500).json({ error: err });
});
const bucket = gcs.bucket("myapp.appspot.com");
const uuid = UUID();
return bucket.upload(
"/tmp/uploaded-image.jpg",
{
uploadType: "media",
destination: "/places/" + uuid + ".jpg",
metadata: {
metadata: {
contentType: "image/jpeg",
firebaseStorageDownloadTokens: uuid
}
}
},
(err, file) => {
if (!err) {
return response.status(201).json({
imageUrl:
"https://firebasestorage.googleapis.com/v0/b/" +
bucket.name +
"/o/" +
encodeURIComponent(file.name) +
"?alt=media&token=" +
uuid,
imagePath: "/places/" + uuid + ".jpg"
});
} else {
console.log(err);
return response.status(500).json({ error: err });
}
}
);
})
.catch(error => {
console.log("Token is invalid!");
response.status(403).json({error: "Unauthorized"});
});
});
});
I don't have a React Native environment easily available, but I believe you can do it from the client with code like this:
await firebase.storage().ref('test/test.jpg').putFile('/path/to/test.jpg');
let downloadUrl = await firebase.storage().ref('test/test.jpg').getDownloadURL();
console.log('downloadUrl :', downloadUrl); // do whatever you need with it
To upload another image you just call the code twice, you can even do it in concurrently if you want.
When you use Firebase you should do most of the operations directly from the client, so you just need backend (including cloud functions) code if you need to do some heavy processing, use the admin SDK, integrate with third party apps, or stuff like that. For simple database or storage operations the client will suit you much better.
Also, you don't need to compose the download URL yourself, getDownloadUrl() does that for you. And if you access storage from the client it automatically integrates with Firebase Auth so you can protect your data.
I am using Firebase storage and Realtime Database for storing the image and its download url respectively.The filename is generated in a random manner with which download url is generated and saved to the realtime database.
Scenario:
If user uploads new Image(For e.g Profile Image) I want to delete the old image with the help of downloadImageurl(Download image url is generated when image is uploaded initially and same is saved in the realtime database).How the old image can be deleted?I have tried below code but for it to work I must get filename.
gcs
.bucket("e**********.appspot.com") // find it in Firebase>Storage>"gs://...." copy without gs
//or go to console.cloud.google.com/ buckets and copy name
.file("images/" +event.params.uid+"/"+filename) //file location in my storage
.delete()
.then(() => {
console.log(`gs://${bucketName}/${filename} deleted.`);
})
.catch(err => {
console.error('ERROR-DELETE:', err+ " filename: "+filename);
});
This may help you out.
This code will fetch the name of file from the URL and will delete that file. Currently this solution works for me!
Code
import * as firebase from 'firebase';
...
let name = imagePath.substr(imagePath.indexOf('%2F') + 3, (imagePath.indexOf('?')) - (imagePath.indexOf('%2F') + 3));
name = name.replace('%20',' ');
let storagePath = firebase.storage().ref();
storagePath.child(`images/${name}`).delete();
Depending on what you want:
Keep the original image and being able to delete it manually in the future.
Delete it immediately after the thumbnail is generated.
I suppose you are using this example
1- You have to store the filePath in your db. Then whenever you want to delete it from your front:
import * as firebase from 'firebase';
...
const store = firebase.storage().ref();
// Depending on which db you use and how you store, you get the filePath and delete it:
store.child(image.filePath).delete();
2- Continue the promise from the firebase function like this:
// ...LAST PART OF THE EXAMPLE...
.then(() => {
// Add the URLs to the Database
return admin.database().ref('images').push({path: fileUrl, thumbnail: thumbFileUrl});
}).then(() => {
// ...PART YOU CAN ADD TO DELETE THE IMAGE UPLOADED
const bucket = gcs.bucket(bucket);
bucket.file(filePath).delete();
})
"bucket" is the const previously created:
const bucket = gcs.bucket(event.data.bucket);
as well as "filePath":
const filePath = event.data.name;
To delete an image with the url, you can use refFromUrl() function to get the ref then delete it easily
const storage = firebase.storage();
storage.refFromURL(imageUrl).delete()
const downloadUrl = "https://firebasestorage.googleapis.com/v0/b/***.appspot.com/o/***?alt=media&token=***";
The first *** represents FIREBASE_STORAGE_BUCKET
The second *** represents location of file in bucket
The third *** represents token for public access image/file
As a web developer, you're aware that URI are encoded such as
"#" = "%40",
"$" = "%24",
" " = "%20", etc.
Since we are using JavaScript, what we can do is decode URI like so to get exact path
const path = decodeURIComponent(downloadUrl.split("o/")[1].split("?")[0]);
return await bucket
.file(path)
.delete()
.then(() => true)
.catch((error) => {
throw new TypeError(`deleteImages ${error}`);
});
I have 100 images that I want to store in firebase storage, but I also need to extract the urls from them. Is there an automatic way of doing it?
If not is there a better service provider that allows uploading a lot of images and extracting the url all automatically??
I highly recommend using Firebase Storage and the Firebase Realtime Database together to accomplish this. Some code to show how these pieces interact is below (Swift):
Shared:
// Firebase services
var database: FIRDatabase!
var storage: FIRStorage!
...
// Initialize Database, Auth, Storage
database = FIRDatabase.database()
storage = FIRStorage.storage()
Upload:
let fileData = NSData() // get data...
let storageRef = storage.reference().child("myFiles/myFile")
storageRef.putData(fileData).observeStatus(.Success) { (snapshot) in
// When the image has successfully uploaded, we get it's download URL
// This "extracts" the URL, which you can then save to the RT DB
let downloadURL = snapshot.metadata?.downloadURL()?.absoluteString
// Write the download URL to the Realtime Database
let dbRef = database.reference().child("myFiles/myFile")
dbRef.setValue(downloadURL)
}
Download:
let dbRef = database.reference().child("myFiles")
dbRef.observeEventType(.ChildAdded, withBlock: { (snapshot) in
// Get download URL from snapshot
let downloadURL = snapshot.value() as! String
// Create a storage reference from the URL
let storageRef = storage.referenceFromURL(downloadURL)
// Download the data, assuming a max size of 1MB (you can change this as necessary)
storageRef.dataWithMaxSize(1 * 1024 * 1024) { (data, error) -> Void in
// Do something with downloaded data...
})
})
For more information, see Zero to App: Develop with Firebase, and it's associated source code, for a practical example of how to do this.