How to access file data from firebase storage without downloading - firebase

I have a URL for my file type .ies that is uploaded on firebase storage and I want to read the content inside the file but when I hit URL on the browser it only returns the information about the file, not the content
Example link: https://firebasestorage.googleapis.com/v0/b/westgatedash-d1341.appspot.com/o/documents%2FIES%2FCDL2-45W-M-120V_IESNA2002.IES
but when I use query params alt=media it downloads that I don't want.
Are there any query params to get the data of the file? or any way to achieve the goal ?

According to the documentation, each call is different and for different purposes:
Listing objects:
To list the objects in a bucket:
curl -X GET -H "Authorization: Bearer OAUTH2_TOKEN" \
"https://storage.googleapis.com/storage/v1/b/BUCKET_NAME/o"
Downloading objects:
curl -X GET \
-H "Authorization: Bearer OAUTH2_TOKEN" \
-o "SAVE_TO_LOCATION" \
"https://storage.googleapis.com/storage/v1/b/BUCKET_NAME/o/OBJECT_NAME?alt=media"
The difference when hitting the endpoint is, precisely, the alt=media param. What I understand is that you might want to stream the object? If so, you can achieve it through the client libraries. For instance:
To handle the file on a Cloud Function you can follow this guide you will find more detailed info. As a summary, to download a file:
// Download file from bucket.
const bucket = admin.storage().bucket(fileBucket);
const tempFilePath = path.join(os.tmpdir(), fileName);
const metadata = {
contentType: contentType,
};
await bucket.file(filePath).download({destination: tempFilePath});
console.log('Image downloaded locally to', tempFilePath);
This will save the image to a temporary folder since this is the recommended way to do it.
Use gcs.bucket.file(filePath).download to download a file to a temporary directory on your Cloud Functions instance. In this location, you can process the file as needed.
EDIT
According to Downloads Documentation:
You can send download requests to Cloud Storage in the following ways:
Simple download: Downloading objects to a destination.
Streaming download: Downloading data to a process.
Sliced object download: Downloading large objects.
In the Docs, following the GitHub Repo: For C++ it shows a stream downloading:
void ReadObject(google::cloud::storage::Client client,
std::vector<std::string> const& argv) {
//! [read object] [START storage_download_file]
namespace gcs = google::cloud::storage;
[](gcs::Client client, std::string const& bucket_name,
std::string const& object_name) {
gcs::ObjectReadStream stream = client.ReadObject(bucket_name, object_name);
int count = 0;
std::string line;
while (std::getline(stream, line, '\n')) {
++count;
}
std::cout << "The object has " << count << " lines\n";
}
//! [read object] [END storage_download_file]
(std::move(client), argv.at(0), argv.at(1));
}
But for NodeJs, for some reason, it doesn't. I was not able to find any example of that. On the other hand there are other questions in SatckOverflow with the same question, i.e: this:
const {Storage} = require('#google-cloud/storage');
const storage = new Storage();
const bucket = storage.bucket(bucket);
const remoteFile = bucket.file(file);
let buffer = '';
remoteFile.createReadStream()
.on('error', function(err) {console.log(err)})
.on('data', function(response) {
buffer += response
})
.on('end', function() {
//console.log(buffer);
res.send(buffer);
})

Related

Firebase web client gzip upload to Cloud Storage

I am using the Firebase web client and would like to upload a gzipped file to Cloud Storage. With the Nodejs client it is possible to simply specify gzip as an option like so: await bucket.upload(filepath, { destination, gzip: true });. Is it possible to gzip data upload to Cloud Storage using the Firebase web client? For example, how would I change the following to gzip the file?
const ref = firebase.storage().ref().child("test.json");
const blob = new Blob([JSON.stringify({ hello: "world" })], { type: "application/json" });
await ref.put(blob);
You can take a look at the documentation which shows you a sample to upload a picture:
// Create a root reference
var storageRef = firebase.storage().ref();
// Create a reference to 'mountains.jpg'
var mountainsRef = storageRef.child('mountains.jpg');
// Create a reference to 'images/mountains.jpg'
var mountainImagesRef = storageRef.child('images/mountains.jpg');
// While the file names are the same, the references point to different files
mountainsRef.name === mountainImagesRef.name // true
mountainsRef.fullPath === mountainImagesRef.fullPath // false
// Create file metadata including the content type
var metadata = {
contentType: 'image/jpeg',
};
// Upload the file and metadata
var uploadTask = storageRef.child('images/mountains.jpg').put(file, metadata);
But in your case you may want to upload gzip file. Just change the metadata.

How to backup Firebase Realtime Database more frequently than once in 24 hours?

I have been trying to find a solution for this since a couple of days now. I want to backup my data of Realtime Database more frequently than what is provided by Firebase i.e. once in 24 hours. I have done a similar thing with my FireStore, deployed an app and a cron job to backup every 8 hours and save that to my GS bucket. But I couldn't find anything at all which is a similar for Realtime Database.
Here's what I have tried-
I am using firebase functions here (I will add a cron job later). What I am doing is fetching every child of my db's root node and zipping it with zlib. Here's what my code looks like-
const functions = require('firebase-functions');
const admin = require("firebase-admin");
const cors = require('cors')({origin: true});
const zlib = require('zlib');
exports.backup = functions.https.onRequest((request, response) => {
cors(request, response, async () => {
const db = admin.database();
const ref_deb = db.ref("/");
ref_deb.once('value').then(function (snapshot) {
console.log("Database value " + snapshot.val());
let jsonFileContent = snapshot.val();
if (jsonFileContent != null) {
//console.log(io);
console.log("getCompressedJSONFile function started", jsonFileContent);
let bufferObject = new Buffer.from(JSON.stringify(jsonFileContent));
zlib.gzip(bufferObject, function (err, zippedData) {
if (err) {
console.log("error in gzip compression using zlib module", err);
response.send("Error").status(500);
} else {
console.log("zippedData", zippedData);
response.send(zippedData).status(200);
}
})
} else {
response.send("Error").status(500)
}
}, function (error) {
console.error(error);
});
});
});
I am getting the response (obviously encoded and not readable)
Now I want to upload it to my GS bucket. How can I do that? Is it possible to do it by passing the zippedContents from zlib without creating a file? Any help will be appreciated.
My database is less than 0.5 MB uncompressed so backing everything at once shouldn't be an issue.
There is many questions in your question.
First, you can't trigger manually a backup like you can do it into Firestore (Firebase).
Then, you can stream directly your file from your function to storage, but I don't recommend it. Indeed, there is no CRC when you stream data, and, if an issue occur in your ZIP file, the whole file is corrupted and you can't recover your data
If your file is small, I recommend you to store it into /tmp directory, a writable in-memory file system (tmpfs), and then copy the file to the bucket.

Firebase Storage - Image preview is permenantly loading

I've started working with firebase storage and firebase functions recently. Right now I've been developing file upload from functions to storage .
I've got it working (upload is done and file appears on the storage section), yet, the image, stays like this forever (loading forever on the right side):
I though that it was an error from my code. Yet, if I open Google Cloud Platform - Storage, the image appears and I can open it and preview it.
In firebase storage, if I open the image (select on it and click open), it returns the following url: https://console.firebase.google.com/u/0/undefined
What may I been doing wrong? Here's the code I'm using:
function uploadImage() {
const newImageData = "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAOEAAADhCAMAAAAJbSJIAAAAgVBMVEX///8AAAAEBASAgIDr6+vw8PBYWFjU1NTGxsbz8/P29vb8/Py1tbVhYWHd3d1ra2vk5OS/v78pKSlTU1NOTk6Tk5OpqanNzc13d3dKSkplZWWbm5s5OTkfHx+NjY2GhoYcHBw9PT0TExOioqJ7e3soKCiurq5CQkI6OjoXFxcwMDAuPQWoAAAIJ0lEQVR4nO2daXuqPBCGVfYtIbKLgorLKf//B75ga2sPAdmS8F5n7m/VXgyPISGZzExWKwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAOAVy6wwlCZG/YUl+vamYLoy0lAkORWbdZNN/YUUVf8ju6bomx2Kq+mEBL5K0UVD9QNCdM0Vfds9QSQMnYTWaN1sEicMCRJ9+29QSCqp28HiftiqUkoU0TLaMDLsl8PbrtGWpY8zQ7QYCoE/pe3+ZusHogW9Yln2nxnVPfljW8t4l8go2zPQV7PPkCxa3kqOvOldr52NF4nVKNuYobxPsC1OoxVilu33ZINDQf1Rw2cO+mrOWBMhMD5w0ldziLnrQyeO+mpOfOdzZspZX03KcQGC2I+gNDC3ZkQ7IQLX6x0niW7fZd83ief7Pg7tH0JcfeIlQy+kcllCyk7/O/LUWIsiDbkVv9/bcv0Rqr+MVa//BR0Ob/++AstdKLuK8n4ZZCiKK4e7cikSez2iTjxm2DPjPj8e8wf1zWviXPqpPuHyeuqXb2ZK6WxaqJCuZe7m7qTTp1da6ty7prtbMoOOVro64VktonlmyFZUqB0NybQrkna7l1ndZIhc2k0xbESjdTEf63M7yBQ9bjO2Z+enamvCXcTCphG1zZ3YNSJ9OroJmRkM6UMOZmXPoppzWE4WEX1oY7Xmv1Fs7Zn9nl9gWt+/MTImNU2dprze+6GfmmYlRrYoHZ9dF/whbJrdMTLVdMzEPDZRlOZr48DIVMMQ2wnUN5SpIiNLjaF7z74X1uiNwWbDyNI/qHCd8XBGW1nDLj+F64K9RKtomuWocMvq3fvDjbIk5ahwvc0YGXuS0dbcPBWu1zlLx4mbU23yVbi+s3srkjvdJGeF1WqGTTO6rZsH3BVWjyqae1C1EP0BFaVwvQ60OR1EshZ0GROisJqG23Pt1Gp2q49GqML1OsmK6aMOKbK3OzbCFFZsLwEZ75YySHDpE1olUmFl/XrP02jE5aM0v1972phd2ydDQkvOnjRk4aFL3pDAjiUo/LoTHOuKLMtGzet+lPn4pPpG0eMRQTnLUfi8oTreWc0j7UmUq22x0f9PhbMDCkEhKBQPKJxNYTkgEmYCXiMShZvCXEkHTUXGcPZSpbFS5KZQWq3Qu4XORGIbUTa9eCqsFqt6OGemxSvbUH8sqwUrXK0sl/gM9PnE/XKNCFdYYZozB7VvsPkzU1+Cwhrrph7mGHfOB/X226+1FIU1duztr+Nlnq97L7YbV12SworITqXdvmcw5QvlfielNtU5sDCFNYpO0iJTk2MvbcdEzYqUtIdVLVDhAwvVubIk8D3Po421j88DUufHvvEnL1XhF2Yd7+xqTR6f9wq1XbjCGQCFs/EPKsR8sq6Vxi4bvxVwzCOJBTVXLxzX+BxSrgPKm4arFwM3J1lzQk/D5eunOTjsIqN0h57Gyd0TlbBJYtVa9xFF+NpUedZSM5Ypd6UfifEm4tts5QFkdOvO4RTlLz3GoTZDVpAWxu/WJAI9wn/uRaiPj65x9bC496ixIdjnvXek7Db8HWLfMsnpWWFjAV79w9ZxqlVtzwtXq2XH2Q7I71+Aws/72FyvVz/IsrzlgnmWBX71P5vBV16IQhr3uCUYbxBLVjgPoBAUgkLxgEJQCArFAwpBISgUDzeF1+E7vWMor8IUZi5mFUrzwxa7jRRLnntP0YCqUaNwItG7a3LBUqNT1B5K0fuHboj7RSQM5YjDT7edaIV1HlbeGAwmc82/c8XEK1ytTDkcXKeuEzWUlxf1JRdzOJlq7sXv/YGlKKxwi91pQtBXHfZ12hUNn/mCFNZEaYxPY6YC5QnH9JSwhSl8qCRpFu9Pfb3Yh9M+zlLSmvAmTmGbL/uBoUfkFhY48dvDa30/wUV4I5HembMoLs7b6xGKISOE9Ce5p6qql3//XX3XY88RNVICllvbpM7Hk+WhOab8aps0nzIeRZSoZZQYWaIkyLMNNfnEbprtHAAmQCtlyL6OEq3Q34WRLcqPyb4V+RqlGVuHLMP3FEofXDPrhqsVPYIHM6xtQg86SZgZ1Kj21iWjEjVu3jL5Y3eOgEw3WK3l4vnrRVlx65qTXRFaWlGqJ/Fq1qivVUdKHMvyW3JXMJZPBs9W6BhyZ6YYZlrSW++wXHVIiaCpIg1EpO61F9syePK7fMokt8noekMWInb+rnRLzLgqu/ved3jw8tQePrq6dpp771eUDvPDA+TT25uo2Ho4v4V9n1gjvOXY6+U+P3E4OqC7K75w/DglSZymQftNyUGaxkly+ujtcOVSi7K3xF987KRXdh+jrsKn2OZK57Pl1KTkJLCaD7PejqHjcDwYsXXGyBBWs98WrBtvieWN89FkJucTkTDif9ysgTgKnDwZHMfwY5FGwucQJAqdaS2zKhR1AiIoBIWgEBSCQlAICkEhKASFi1DIokobDV/Y2ePjvG7D4eaA+gcVRnyi9zdj6p/Pg8vncOedqCX+amXxOYA85exle8Xg4XDDYrxQX2g9iyBMYM8uLqEXiHVKyZbbkfFt5GwLCZ9ZhbANIJwrfp3GnU/04xsidpM3X9yb8BcoYJQzEwjvg09M++3ZNyNIbP6bMa1YCjVIchK2IvBFT8PaDaig85bDbmHyHljhaZ7+eDyFS9RXo9jx9K3hMrY57tcPh6SX/fiWPO4vKZ8TeKdg6SRTx4g8qhnRl/p4/oWJIiKpQx7YUpVIJGCrfgqG7CIb9zjSaeNgG7kzhaZyx7JM07Ljy4XWnuXlEtv1P7B9Mv8DltyUV+hIpoIAAAAASUVORK5CYII="
var mimeTypes = require('mimetypes');
var image = newImageData,
mimeType = image.match(/data:([a-zA-Z0-9]+\/[a-zA-Z0-9-.+]+).*,.*/)![1],
fileName = 'test.' + mimeTypes.detectExtension(mimeType),
base64EncodedImageString = image.replace(/^data:image\/\w+;base64,/, ''),
imageBuffer = new Buffer(base64EncodedImageString, 'base64');
// Instantiate the GCP Storage instance
const { Storage } = require('#google-cloud/storage');
const googleCloudStorage = new Storage(firebaseSettings);
const bucket = googleCloudStorage.bucket('projectID.appspot.com');
var file = bucket.file(fileName);
return file.save(imageBuffer, {
metadata: { contentType: mimeType, cacheControl: "public, max-age=300" },
public: true,
validation: 'md5'
}, function (error: any) {
if (error) {
throw 'error';
}
return "https://storage.googleapis.com/share-expanses-dcc9f.appspot.com/" + fileName;
});
}
Thanks for the help
Haven't been able to test the solution given by Firebase, but here's the transcript of the response:
The problem that you are facing could be because of two reasons. The
first one is how you are uploading the files, via the Firebase
Console, using any Admin SDK, or via the gsutil command. If using the
Admin SDK option, the problem is a known issue where the required
metadata doesn’t exist, fortunately there is a workaround, you can try
this script to solve this issue.
Now, the second one is related to the network if you are using
comcast, please, try on a different network to see if this issue is
related to that.
When you save an image to firebase, you need to provide an access token in metadata : firebaseStorageDownloadTokens. It has to be an uuid.
More info can be found here : https://www.sentinelstand.com/article/guide-to-firebase-storage-download-urls-tokens
const { v4: uuid } = require("uuid")
function uploadImage() {
const newImageData = "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAOEAAADhCAMAAAAJbSJIAAAAgVBMVEX///8AAAAEBASAgIDr6+vw8PBYWFjU1NTGxsbz8/P29vb8/Py1tbVhYWHd3d1ra2vk5OS/v78pKSlTU1NOTk6Tk5OpqanNzc13d3dKSkplZWWbm5s5OTkfHx+NjY2GhoYcHBw9PT0TExOioqJ7e3soKCiurq5CQkI6OjoXFxcwMDAuPQWoAAAIJ0lEQVR4nO2daXuqPBCGVfYtIbKLgorLKf//B75ga2sPAdmS8F5n7m/VXgyPISGZzExWKwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAOAVy6wwlCZG/YUl+vamYLoy0lAkORWbdZNN/YUUVf8ju6bomx2Kq+mEBL5K0UVD9QNCdM0Vfds9QSQMnYTWaN1sEicMCRJ9+29QSCqp28HiftiqUkoU0TLaMDLsl8PbrtGWpY8zQ7QYCoE/pe3+ZusHogW9Yln2nxnVPfljW8t4l8go2zPQV7PPkCxa3kqOvOldr52NF4nVKNuYobxPsC1OoxVilu33ZINDQf1Rw2cO+mrOWBMhMD5w0ldziLnrQyeO+mpOfOdzZspZX03KcQGC2I+gNDC3ZkQ7IQLX6x0niW7fZd83ief7Pg7tH0JcfeIlQy+kcllCyk7/O/LUWIsiDbkVv9/bcv0Rqr+MVa//BR0Ob/++AstdKLuK8n4ZZCiKK4e7cikSez2iTjxm2DPjPj8e8wf1zWviXPqpPuHyeuqXb2ZK6WxaqJCuZe7m7qTTp1da6ty7prtbMoOOVro64VktonlmyFZUqB0NybQrkna7l1ndZIhc2k0xbESjdTEf63M7yBQ9bjO2Z+enamvCXcTCphG1zZ3YNSJ9OroJmRkM6UMOZmXPoppzWE4WEX1oY7Xmv1Fs7Zn9nl9gWt+/MTImNU2dprze+6GfmmYlRrYoHZ9dF/whbJrdMTLVdMzEPDZRlOZr48DIVMMQ2wnUN5SpIiNLjaF7z74X1uiNwWbDyNI/qHCd8XBGW1nDLj+F64K9RKtomuWocMvq3fvDjbIk5ahwvc0YGXuS0dbcPBWu1zlLx4mbU23yVbi+s3srkjvdJGeF1WqGTTO6rZsH3BVWjyqae1C1EP0BFaVwvQ60OR1EshZ0GROisJqG23Pt1Gp2q49GqML1OsmK6aMOKbK3OzbCFFZsLwEZ75YySHDpE1olUmFl/XrP02jE5aM0v1972phd2ydDQkvOnjRk4aFL3pDAjiUo/LoTHOuKLMtGzet+lPn4pPpG0eMRQTnLUfi8oTreWc0j7UmUq22x0f9PhbMDCkEhKBQPKJxNYTkgEmYCXiMShZvCXEkHTUXGcPZSpbFS5KZQWq3Qu4XORGIbUTa9eCqsFqt6OGemxSvbUH8sqwUrXK0sl/gM9PnE/XKNCFdYYZozB7VvsPkzU1+Cwhrrph7mGHfOB/X226+1FIU1duztr+Nlnq97L7YbV12SworITqXdvmcw5QvlfielNtU5sDCFNYpO0iJTk2MvbcdEzYqUtIdVLVDhAwvVubIk8D3Po421j88DUufHvvEnL1XhF2Yd7+xqTR6f9wq1XbjCGQCFs/EPKsR8sq6Vxi4bvxVwzCOJBTVXLxzX+BxSrgPKm4arFwM3J1lzQk/D5eunOTjsIqN0h57Gyd0TlbBJYtVa9xFF+NpUedZSM5Ypd6UfifEm4tts5QFkdOvO4RTlLz3GoTZDVpAWxu/WJAI9wn/uRaiPj65x9bC496ixIdjnvXek7Db8HWLfMsnpWWFjAV79w9ZxqlVtzwtXq2XH2Q7I71+Aws/72FyvVz/IsrzlgnmWBX71P5vBV16IQhr3uCUYbxBLVjgPoBAUgkLxgEJQCArFAwpBISgUDzeF1+E7vWMor8IUZi5mFUrzwxa7jRRLnntP0YCqUaNwItG7a3LBUqNT1B5K0fuHboj7RSQM5YjDT7edaIV1HlbeGAwmc82/c8XEK1ytTDkcXKeuEzWUlxf1JRdzOJlq7sXv/YGlKKxwi91pQtBXHfZ12hUNn/mCFNZEaYxPY6YC5QnH9JSwhSl8qCRpFu9Pfb3Yh9M+zlLSmvAmTmGbL/uBoUfkFhY48dvDa30/wUV4I5HembMoLs7b6xGKISOE9Ce5p6qql3//XX3XY88RNVICllvbpM7Hk+WhOab8aps0nzIeRZSoZZQYWaIkyLMNNfnEbprtHAAmQCtlyL6OEq3Q34WRLcqPyb4V+RqlGVuHLMP3FEofXDPrhqsVPYIHM6xtQg86SZgZ1Kj21iWjEjVu3jL5Y3eOgEw3WK3l4vnrRVlx65qTXRFaWlGqJ/Fq1qivVUdKHMvyW3JXMJZPBs9W6BhyZ6YYZlrSW++wXHVIiaCpIg1EpO61F9syePK7fMokt8noekMWInb+rnRLzLgqu/ved3jw8tQePrq6dpp771eUDvPDA+TT25uo2Ho4v4V9n1gjvOXY6+U+P3E4OqC7K75w/DglSZymQftNyUGaxkly+ujtcOVSi7K3xF987KRXdh+jrsKn2OZK57Pl1KTkJLCaD7PejqHjcDwYsXXGyBBWs98WrBtvieWN89FkJucTkTDif9ysgTgKnDwZHMfwY5FGwucQJAqdaS2zKhR1AiIoBIWgEBSCQlAICkEhKASFi1DIokobDV/Y2ePjvG7D4eaA+gcVRnyi9zdj6p/Pg8vncOedqCX+amXxOYA85exle8Xg4XDDYrxQX2g9iyBMYM8uLqEXiHVKyZbbkfFt5GwLCZ9ZhbANIJwrfp3GnU/04xsidpM3X9yb8BcoYJQzEwjvg09M++3ZNyNIbP6bMa1YCjVIchK2IvBFT8PaDaig85bDbmHyHljhaZ7+eDyFS9RXo9jx9K3hMrY57tcPh6SX/fiWPO4vKZ8TeKdg6SRTx4g8qhnRl/p4/oWJIiKpQx7YUpVIJGCrfgqG7CIb9zjSaeNgG7kzhaZyx7JM07Ljy4XWnuXlEtv1P7B9Mv8DltyUV+hIpoIAAAAASUVORK5CYII="
var mimeTypes = require('mimetypes');
var image = newImageData,
mimeType = image.match(/data:([a-zA-Z0-9]+\/[a-zA-Z0-9-.+]+).*,.*/)![1],
fileName = 'test.' + mimeTypes.detectExtension(mimeType),
base64EncodedImageString = image.replace(/^data:image\/\w+;base64,/, ''),
imageBuffer = new Buffer(base64EncodedImageString, 'base64');
// Instantiate the GCP Storage instance
const { Storage } = require('#google-cloud/storage');
const googleCloudStorage = new Storage(firebaseSettings);
const bucket = googleCloudStorage.bucket('projectID.appspot.com');
var file = bucket.file(fileName);
return file.save(imageBuffer, {
metadata: {
contentType: mimeType,
cacheControl: "public,
max-age=300",
// THIS IS THE LINE YOU NEED TO ADD
firebaseStorageDownloadTokens: uuid(),
},
public: true,
validation: 'md5'
}, function (error: any) {
if (error) {
throw 'error';
}
return "https://storage.googleapis.com/share-expanses-dcc9f.appspot.com/" + fileName;
});
}
After that you'll need to click on "Create access token"
#jean-smaug answer is almost complete. Based on the page he linked (https://www.sentinelstand.com/article/guide-to-firebase-storage-download-urls-tokens), the only missing thing is to wrap the firebaseStorageDownloadTokens property inside a metadata object. I've just tested it and it's working fine 👌 No need to create access token afterwards.
In my case I added metadata while uploading and it loading as it showed in image but when I'm refresh page after 3 min I found that it upload correctly , so as Cafn explain if it not matter of metadata you should wait until it loaded
$uploadedObject=$bucket->upload($imageFile, [
'name' => 'Image_Name',
"metadata" => [ "contentType"=> 'image/png'],
]);

How to get public download link within a firebase storage trigger function: "onFinalize"?

I am writing a firebase cloud function that records the download link of a recentally uploaded file to real-time database:
exports.recordImage = functions.storage.object().onFinalize((object) => {
});
"object" gives me access to two variables "selfLink" and "mediaLink" but both of them when entered in a browser they return the following:
Anonymous caller does not have storage.objects.get access to ... {filename}
So, they are not public links. How can I get the public download link within this trigger function?
You have to use the asynchronous getSignedUrl() method, see the doc of the Cloud Storage Node.js library: https://cloud.google.com/nodejs/docs/reference/storage/2.0.x/File#getSignedUrl.
So the following code should do the trick:
.....
const defaultStorage = admin.storage();
.....
exports.recordImage = functions.storage.object().onFinalize(object => {
const bucket = defaultStorage.bucket();
const file = bucket.file(object.name);
const options = {
action: 'read',
expires: '03-17-2025'
};
// Get a signed URL for the file
return file
.getSignedUrl(options)
.then(results => {
const url = results[0];
console.log(`The signed url for ${filename} is ${url}.`);
return true;
})
});
Note that, in order to use the getSignedUrl() method, you need to initialize the Admin SDK with the credentials for a dedicated service account, see this SO Question & Answer firebase function get download url after successfully save image to firebase cloud storage.
*use this function:
function mediaLinkToDownloadableUrl(object) {
var firstPartUrl = object.mediaLink.split("?")[0] // 'https://storage.googleapis.com/download/storage/v1/b/abcbucket.appspot.com/o/songs%2Fsong1.mp3.mp3'
var secondPartUrl = object.mediaLink.split("?")[1] // 'generation=123445678912345&alt=media'
firstPartUrl = firstPartUrl.replace("https://storage.googleapis.com/download/storage", "https://firebasestorage.googleapis.com")
firstPartUrl = firstPartUrl.replace("v1", "v0")
firstPartUrl += "?" + secondPartUrl.split("&")[1]; // 'alt=media'
firstPartUrl += "&token=" + object.metadata.firebaseStorageDownloadTokens
return firstPartUrl
}
this is how your code might look like:
export const onAddSong = functions.storage.object().onFinalize((object) => {
console.log("object: ", object);
var url = mediaLinkToDownloadableUrl(object);
//do anything with url, like send via email or save it in your database in playlist table
//in my case I'm saving it in mongodb database
return new playlistModel({
name: storyName,
mp3Url: url,
ownerEmail: ownerEmail
})
.save() // I'm doing nothing on save complete
.catch(e => {
console.log(e) // log if error occur in database write
})
})
*I have tested this method on mp3 files, I'm sure it will work on all type of files but incase if it doesnt work for you simply go to firebase storage dashboard open any file and copy download url, and try to generate the same url in your code, and edit this answer too if possible

Copy a file on firebase storage?

Is it possible to copy an existing file on firebase storage without needing to uploading it again?
I need it for a published/working version setup of my app.
There is no method in the Firebase Storage API to make a copy of a file that you've already uploaded.
But Firebase Storage is built on top of Google Cloud Storage, which means that you can use the latter's API too. It looks like gsutil cp is what you're looking for. From the docs:
The gsutil cp command allows you to copy data between your local file system and the cloud, copy data within the cloud, and copy data between cloud storage providers.
Keep in mind that gsutil has full access to your storage bucket. So it is meant to be run on devices you fully trust (such as a server or your own development machine).
Here is an approach I ended up with for my project.
While it covers a broader case and copies all files under folder fromFolder to toFolder, it can be easily adopted to the case from the question (to copy only files one can pass delimiter = "/" - refer to the docs for more details)
const {Storage} = require('#google-cloud/storage');
module.exports = class StorageManager{
constructor() {
this.storage = new Storage();
this.bucket = this.storage.bucket(<bucket-name-here>)
}
listFiles(prefix, delimiter){
return this.bucket.getFiles({prefix, delimiter});
}
deleteFiles(prefix, delimiter){
return this.bucket.deleteFiles({prefix, delimiter, force: true});
}
copyFilesInFolder(fromFolder, toFolder){
return this.listFiles(fromFolder)
.then(([files]) => {
let promiseArray = files.map(file => {
let fileName = file.name
let destination = fileName.replace(fromFolder, toFolder)
console.log("fileName = ", fileName, ", destination = ", destination)
return file.copy(destination)
})
return Promise.all(promiseArray)
})
}
}
I needed to copy folders (including all it's descendants) in Google Cloud Storage. Here is the solution proposed by #vir-us, simplified and for NodeJS.
import type { Bucket } from '#google-cloud/storage';
/** Copy Cloud Storage resources from one folder to another. */
export const copyStorageFiles = async (input: {
bucket: Bucket;
fromFolder: string;
toFolder: string;
logger?: Console['info'];
}) => {
const { bucket, fromFolder, toFolder, logger = console.info } = input;
const [files] = await bucket.getFiles({ prefix: fromFolder });
const promiseArray = files.map((file) => {
const destination = file.name.replace(fromFolder, toFolder);
logger("fileName = ", file.name, ", destination = ", destination);
return file.copy(destination);
});
return Promise.all(promiseArray);
};

Resources