I have a firebase function to upload files to firebase storage, after upload I have to return the url (as Reset response) so that user can view the file
const bucket = admin.storage().bucket();
const [file, meta] = await bucket.upload(tempLocalFile, {
destination: uploadPath,
resumable: false,
public: true,
});
I have two options
1- const signedUrl = await file.getSignedUrl({ action: 'read', expires: '03-09-2491' });
2- meta.mediaLink
SignedUrl will be like https://storage.googleapis.com/web-scanner-dev.appspot.com/pwc%2Fwww.x.com%2F2019-11-17%2Fdesktop%2Fscreenshot-2019-11-17-1125.png?GoogleAccessId=firebase-gcloud%40scanner-dev.iam.gserviceaccount.com&Expires=16447035600&Signature=w49DJpGU9%2BnT7nlpCiJRgfAc98x4i2I%2FiP5UjQipZQGweXmTCl9n%2FnGWmPivkYHJNvkC7Ilgxfxc558%2F%2BuWWJ2pflsDY9HJ%2Bnm6TbwCrsmoVH56nuGZHJ7ggp9c3jSiGmQj3lOvxXfwMHXcWBtvcBaVj%2BH2H8uhxOtJoJOXj%2BOq3EC7XH8hamLY8dUbUkTRtaWPB9mlLUZ78soZ1mwI%2FY8DqLFwb75iob4zwwnDZe16yNnr4nApMDS7BYPxh4cAPSiokq30hPR8RUSNTn2GxpRom5ZiiI8dV4w%2BxYZ0DvdJxn%2FW83kqnjx6RSdZ%2B9S3P9yuND3qieAQ%3D%3D
and mediaLink will be like https://storage.googleapis.com/download/storage/v1/b/web-scanner-dev.appspot.com/o/pwc%2Fwww.x.com%2F2019-11-17%2Fdesktop%2Fscreenshot-2019-11-17-1125.png?generation=1574007908157173&alt=media
What is the pros and cons of each?
The mediaLink does not convey any access permissions on its own -- thus, the object itself will need to be publicly readable in order for end uers to make use of the link (or you will need to be authenticated as an account with read access to that bucket when you execute the link).
On the other hand, a URL returned by getSignedUrl will have a signature that allows access for as long as the URL hasn't expired. Thus, the link alone is sufficient (if temporary) permission to access the blob. Additionally, the URL that is generated maintains the permissions of the user who created it -- if that user loses access to the blob before the link would otherwise expire, the link will no longer function.
Related
Is it possible to download images to users local machines directly via Firebase functions? How to do it in case that:
Images are stored in Firebase storage.
Images are stored on other cloud storage providers (I can access them with url).
I don't want to download those images via url links so that I don't reveal the url the image is located on.
Is it possible to download images to users local machines directly via Firebase functions?
No, it's not possible. The client must reach out to the server in order to download content. The content can't be "pushed" to the client without its authorization. That would be a huge security hole for the client.
This is why download URLs exist - to give the client something to download via a normal HTTP request.
You can create a presigned URL using the Google APIs library. The Firebase bucket is just a regular GCS bucket. Something like this:
const admin = getFirebaseAdmin();
let bucket = admin.storage().bucket(firebaseConfig.storageBucket);
const f = bucket.file(location);
if (!(await f.exists())) {
throw logError(`No file found at specified location: ${location}`, functionName)
}
const url1 = await f.getSignedUrl({
action: 'read',
expires: new Date((new Date).getTime() + (24 * 60) * 60000) // expires in 24 hours
});
const url = url1[0];
return url;
Store references by name (users/1/profile.png). Then the URL needs to be generated all the time.
const url = await firebase
.storage()
.ref('users/1/profile.png')
.getDownloadURL()
Store references by URL. The access token could be revoked which would cause issues trying to generate a new one and update it in the database.
const url = await firebase
.storage()
.refFromURL(invalidURL)
.getDownloadURL()
Related to #1. Store by file name only so files can be moved without having to update database references.
const url = await firebase
.storage()
.ref(`users/${user.id}/${user.image}`)
.getDownloadURL()
The download URL and the reference path are two different things and I'd store each of them as appropriate (and sometimes both).
Store the Download URL when you want to directly serve the file from storage (e.g. an <img> tag).
Store the Reference Path when you need to keep a reference to the file to modify it later.
Calling getDownloadURL() does trigger a network request and so it's advisable to cache the result when possible to avoid unnecessary extra work/latency.
I'm using some API, that can give me file with 2 ways:
- I can get FILE in response
- I can get direct URL to the file to download
I decided to make request with first method, get FILE in response. I'm trying to save the file in firebase storage, from firebase cloud function:
exports.test = functions.region('europe-west1').https.onRequest(async (req, res) => {
const axios = require('axios').default
axios({
method: 'get',
url: `https://cloudpbx.beeline.ru/apis/portal/v2/records/06365a27-f8d1-4c51-bba3-a08802429964/9052948777%40mpbx.sip.beeline.ru/download`
}).then((resp) => {
console.log('resp', resp.data)
const bucket = admin.storage().bucket('cardbox-1.appspot.com/mcun/calls')
bucket.upload(resp.data)
})
})
In resp.data i have something like that:
��H�Xing��h $&)+-.179:>ACEGINRTW[_acegjmoqtwz|����������������������������������������������������PLAME3.100(,�$!��h�6���H��V]#mdE�E��"#���}�8kwT��&���>:P�2��>?�����8 ����������*���Z��XVC�(b�k�
D�G��
As i can see its file by itself, but how to handle it, and how upload it into firebase storage? Also, i'm not sure, that i'm correct creating a bucket and using storage api.
Please feel free to make request, you can get response without auth. That file must be mp3 type
Here is the logs with headers about what i'm getting from response:
HTTP/1.1 200 OK
Server: nginx/1.6.2
Date: Tue, 24 Mar 2020 16:19:55 GMT
Content-Type: application/octet-stream;charset=utf-8
Content-Length: 48744
Connection: close
Content-Disposition: attachment; filename=file.mp3
��H�Xing��h $&)+-.179:>ACEGINRTW[_acegjmoqtwz|����������������������������������������������������PLAME3.100(,�$!��h�6���H��V]#mdE�E��"#���}�8kwT��&���>:P�2��>?�����8 ����������*���Z��XVC�(b�k�
D�G��
C���al�ꝰ�z�KY�Q�]ЪF�8�^���W;,�LUteA�%�u�&��.���
0�q��Cڟ#���6��#�K�h��Itᠠ|[�q�{j'�+ �ʤS<F� Y�5D]?�����nlƦBC�S�Zx'�e�D�Fi_��Vl��4����H��E�z���SMVПQ�P� AT<���(�x�+3�ТB����e�ZC}t��Ї�#֗����2���Kve���.�ԏ�UH���u����ʘfbC��C� ��C�+ġ��̺��R#��|x5hq`Х3�?�N(���.5��� c+��`�Y�F8�d�B&53��¥�L�cx4��3��q�JHԓJ�43Ώ�(к�4r ��a �D
Dɍ0�
Well, it seems that there are a couple of tasks to do here :)
According to the definition of the upload method and their examples, using node js as client library (that makes total sense). You will be able to Upload a file to the bucket. But to be honest, I am not very clear about the usage of the pathString parameter since it points to a local path. Maybe the URL option that you mentioned would be great to test as a feature.
Just pasting the example from docs:
// Imports the Google Cloud client library
const {Storage} = require('#google-cloud/storage');
// Creates a client
const storage = new Storage();
async function uploadFile() {
// Uploads a local file to the bucket
await storage.bucket(bucketName).upload(filename, {
gzip: true,
metadata: {
Content-Type: 'audio/mpeg'
},
});
console.log(`${filename} uploaded to ${bucketName}.`);
}
uploadFile().catch(console.error);
Another way to accomplish this, is to make a POST request with the JSON API to the upload method of Cloud Storage. As I understand, the objects are saved as "RAW" in Cloud Storage.
If you are concerned about the conversion to MP3 file, I think this answer explains very well. Basically, if you want to perform the conversion in the Firebase Function, maybe is not the best scenario according to the size/time of the files.
Good Luck!
I want to get a downloadURL from a file inside my storage from within the onFinalize trigger. In a best case scenario, I want an URL as short as possible (so preferably not a signed one, but just one like the public one like it can be seen in the Firebase Storage UI). Keep in mind, that I am moving the file first, so I cannot access it directly from the onFinalize parameter.
I currently have the following solution:
await imageRef.move(newPath);
const newFile = defaultBucket.file(newPath);
const url = (await newFile.getSignedUrl({
action: 'read',
expires: '03-09-2491'
}))[0];
This approach has two flaws:
Apparently the signed URL is only valid for 3 days. This may be a known issue
The URL is very long and takes much space in my Firestore
I also saw an approach, where the URL is being reproduced from the bucket name and the token, but I did not manage to find the token in the metadata of the file.
I'm getting a 403 SignatureDoesNotMatch error when trying to load a url generated through:
file.getSignedUrl({
expires: moment()
.add(10, 'minutes')
.format()
})
I've done all the steps outlined in the example, including adding a service account token creator to the App Engine default service account to allow the creation of signed urls:
As an alternative approach to using admin via firebase-functions I tried downloading service account credentials service-account-credentials.json and creating a gcs storage object as suggested here as such:
const { Storage } = require('#google-cloud/storage');
const storage = new Storage({
keyFilename: 'service-account-credentials.json',
projectId: 'project-id',
});
storage.bucket('bucket-id').getFiles({prefix: 'path/to/dir'}).then(files => files.map(file => [same code as above]));
However this still generates SignatureDoesNotMatch urls.
I've followed the github issue related to the matter but I have not been able to find a viable solution. The solution listed by Firebase dev owner #mcdonamp in the issue references using iam.signBlobRequest but I don't know where iam is defined, I only see it here as a property of bucket, with no signBlobRequest method, and here as an HTTP API endpoint.
Seems that despite the doc's claim of Content-Type headers being optional, they are not. As suggested by this SO post and this github issue, adding contentType to the getSignedUrl options argument fixes the issue:
file.getSignedUrl({
action: 'read',
contentType: 'audio/wav',
expires: moment()
.add(10, 'minutes')
.format()
})
Make sure to also include the header when requesting the resource as well.