I'm building a Flutter app for which I use the same image in a different size. For performance reason, I would like to fetch this same image in different sizes depending on the display size on the client.
To do that, I would build a system allowing me to generate thumbnails of every uploaded images on firebase storage using cloud function and then use those thumbnails from the client.
On upload, the user would receive the public URL of the source image and could just append the desired size to the URL to fetch a specific thumbnail version (ex.https://google.api.com/mybucket/myprofilepicture.png -> https://google.api.com/mybucket/myprofilepicture#2x.png)
However, firebase cloud storage uses a token protection mechanism (ex. https://google.api.com/mybucket/myprofilepicture.png?alt=media&token=09cb2927-4706-4e36-95ae-2515c68b0d6e) more info here making the public URLs of newly uploaded files unpredictable from the client.
The official thumbnail generation cloud function example solve this by saving the generated URLs into cloud firestore and therefore, requiring another call to be made from the client to fetch the available thumbnails. https://github.com/firebase/functions-samples/tree/master/generate-thumbnail
Is there a way to generate predictable URLs for thumbnails to avoid the overhead of storing them in a separate DB?
Related
Right now, everytime I upload an image file to firebase storage, I get an url that looks like this:
https://firebasestorage.googleapis.com/v0/b/my-project.appspot.com/o/some-folder%2Ffile_name.jpg?alt=media
And I'm storing the full URL in my Firestore docs for the object that uses that image.
If I know that it will never change, I could save just the last part: /some-folder%2Ffile_name.jpg?alt=media
But I don't know if the first part could change for future uploads.
Since I'll be rebuilding the full URL on client, if the first part do change for future downloads, it will break my code.
Should I stick to storing the full URL? Or is it safe to assume that the first part of the Firebase Storage API will always be the same?
Firebase gives no written guarantee that nothing will change for that URL. The only "safe" thing to do is store the entire URL. Or, store the path and call getDownloadUrl every time you want to manufacture a new URL.
I have a scenario where a user can upload images to Firebase Storage, however I do not want them to be able to get a URL for these images (Copy Image Address). Instead I want to provide them with a blob.
When a user uploads the image client side, I get the download URL and store that in Firestore. When the user wants to see the image, I have a cloud function that downloads that image, and sends it to the user as a blob.
This works great for images the user uploads, however I also have a cloud function that is triggered automatically when an image is uploaded and generates a thumbnail.
How do I go about getting a permanent download URL from a Cloud Function/Node server for this generated image? I can get a signed one, but it's not what I need
You can't directly with Google Cloud Storage. The signed URL can't live more than 7 days
The longest expiration value is 604800 seconds (7days).
You can keep the link of the generated thumbnail image, but you have to either download it and serve it each time, or generate a signed url for using and displaying it, each time also. This second solution reduces the processing time and thus the cost.
If you want a permanent URL to access your data, you could make them public according to this documentation[1].
Just keep in mind that when accessing public data through the Google Cloud Platform Console, you must authenticate with Google. It can be accessed with any Google account, the account does not have to be associated with the project that contains the public data, nor must it be registered in the Cloud Storage service [2].
[1] https://cloud.google.com/storage/docs/access-control/making-data-public
[2] https://cloud.google.com/storage/docs/access-public-data
If you are using Django and django-storages[google] lib
Make the bucket public(not advisable for production)
Disable ACL and URL signing
Add these to the settings.py
GS_DEFAULT_ACL = None
GS_QUERYSTRING_AUTH = False
I'm using Google Storage to store profile pictures of my users. I have couple of thousands pictures.
Now the pictures are being saved in a bucket like so:
data/images/profiles/USER_ID.jpg
So the URL to an image:
https://storage.cloud.google.com/data/images/profiles/USER_ID.jpg
I don't want users being able to see someone else picture by just knowing their USER_ID, and still, It has to be the USER_ID for easier search from a developer's side.
I can't use Signed URL as my users do not have a google account, and the pictures from the storage are fetched from a Mobile Application.
Is there a way to keep the file names as they are in the storage, but simple hide the path+filename from the URL?
https://storage.cloud.google.com/fc720d5c05411b03e5e2a6692f8d7d61.jpg -> points to https://storage.cloud.google.com/data/images/profiles/USER_ID.jpg
Thank You
You have several options. Here are a few:
Have users request the URL for another user from the server, then have the server decide whether or not the user is allowed to see the image. If so, have the server (which does have a service account) generate a signed URL and pass it back to the user (or redirect to it). This way, although the user may know the user ID of another user and the URL of their image, they still can't see the image unless the server agrees that this is okay.
Use Firebase Storage to manage the images, which will still store them in GCS but will give you Firebase's auth support.
Proxy the images through your app, either an app engine app or something running in GCE or GKE. This lets you hide everything about the source of the image, including the user ID, but has the downside of requiring all of the data to pass through your service.
Reexamine your requirements. "Easier search on the developer's side" may not be as important as you think, and you need to way the benefit of that vs the cost of working around it.
Another option is Google Images API available on AppEngine. You can link your Cloud Storage objects with Google Images API and use benefits of this API - secure URLs, transform and resize images using URL parameters.
You only need to prepare servingURL for every image stored in GCS and persist this serving URL (for example in Google Datastore)
ImagesService imagesService = ImagesServiceFactory.getImagesService();
ServingUrlOptions suo = ServingUrlOptions.Builder
.withGoogleStorageFileName(gcsImageObjectPath)
.secureUrl(true);
String servingUrl = imagesService.getServingUrl(suo);
I have a situation where in Firebase Storage users store their avatar to /users/{uid}.jpg
If I then use the Storage API to get the download URL and display the image, it ends up being very slow to make the first request because the download URL is not cached anywhere.
So my solution is to get the DownloadURL when the user uploads the image and store that in Firebase allowing the client image provider to automatically cache the image which speeds up loads considerably.
However, there is one problem with this solution.
If a user replaces their avatar, the old link becomes broken instead of updated. I believe this is because a new token is generated each time something is uploaded for security reasons but these are of no benefit to me.
So my question is twofold:
1) How can I allow a user to upload an avatar to a path that is dedicated to them such as /users/{uid}.jpg, get a bare download URL that can be cached by the client, and have that URL remain the same even when the file changes at /users/{uid}.jpg
2) If this is not possible, what is the normal way to solve this issue?
Download URLs are opaque. The contents of the actual URL itself is an implementation detail of the system, and it's not supported to dig around in its contents. The URLs can't be dissected or composed.
You can use a storage trigger with Cloud Functions to automatically generate a signed URL whenever something changes in your storage bucket.
So instead of serving from a hard-coded URL, simply retrieve the URL from an updated value in the datastore (or any data storage system). Every time the user updates the avatar, simply store the new URL in the datastore and you can query for it when you need it.
The application I'm working on allows users to create businesses on the map, and upload associated images.
I use Cloud Functions to resize images for various screen resolutions and upload those back to GCS.
To make those images accessible to the public, I generate signed URL which gets saved in the associated entity in Real-time Database.
const [signedUrl] = await bucket.file(path).getSignedUrl({
action: "read",
expires: "01-01-2500",
})
Until today, URLs generated as per the code above would allow anyone to view the images. Then suddenly, all of the previously generated URLs became inaccessible and instead show the following error:
<Error>
<Code>SignatureDoesNotMatch</Code>
<Message>
The request signature we calculated does not match the signature you provided. Check your Google secret key and signing method.
</Message>
<StringToSign>
GET 16725225600 /project-name.appspot.com/placeImage%2F300w%2FUPfppRM1ZyjbwBNiakgzyQ%3D%3D.jpg
</StringToSign>
</Error>
I see the above message for all of the previously uploaded images. Newly uploaded images can be viewed just fine.
My code closely mimics this example provided by functions-samples, however I'm worried about taking this to production in case the issue becomes a recurring one.
What could be the source of this problem and is there any way to protect ourselves against it in the future?
It seems this problem is related to the use of v4 URLs.
The behaviour you are experiencing is explained in this comment by the owner of IAM in Google Cloud Functions:
Recently, GCS has already started supporting v4 URLs. For example, [gsutil] generates v4 URLs by default. These URLs have a max expiry of 7 days.