The application I'm working on allows users to create businesses on the map, and upload associated images.
I use Cloud Functions to resize images for various screen resolutions and upload those back to GCS.
To make those images accessible to the public, I generate signed URL which gets saved in the associated entity in Real-time Database.
const [signedUrl] = await bucket.file(path).getSignedUrl({
action: "read",
expires: "01-01-2500",
})
Until today, URLs generated as per the code above would allow anyone to view the images. Then suddenly, all of the previously generated URLs became inaccessible and instead show the following error:
<Error>
<Code>SignatureDoesNotMatch</Code>
<Message>
The request signature we calculated does not match the signature you provided. Check your Google secret key and signing method.
</Message>
<StringToSign>
GET 16725225600 /project-name.appspot.com/placeImage%2F300w%2FUPfppRM1ZyjbwBNiakgzyQ%3D%3D.jpg
</StringToSign>
</Error>
I see the above message for all of the previously uploaded images. Newly uploaded images can be viewed just fine.
My code closely mimics this example provided by functions-samples, however I'm worried about taking this to production in case the issue becomes a recurring one.
What could be the source of this problem and is there any way to protect ourselves against it in the future?
It seems this problem is related to the use of v4 URLs.
The behaviour you are experiencing is explained in this comment by the owner of IAM in Google Cloud Functions:
Recently, GCS has already started supporting v4 URLs. For example, [gsutil] generates v4 URLs by default. These URLs have a max expiry of 7 days.
Related
Is there any way to do some kind of authorization that allows only people who recently requested a page from Firebase Hosting to be able to send an HTTP POST request to a Firestore db and have it go through?
My page is basically an HTML form that posts data to a Firestore page, though it would be nice if at least one had to speak with the server beforehand, as people do not have to log in to post information.
[EDIT]
Requirements:
Serve a static HTML form (with some Javascript included)
The contents of the HTML form should be posted to a firestore database only if the client actually requested a page from the server within a reasonable timeframe
In general, some external code that did not recently request the page should not be able to post data to the database. This is just a minor restriction to mitigate any "attacks" being too easy.
No concept of user or login
All requests should be done through REST as including the Firebase SDK is way too large for this small of a project
With Firebase Hosting, there is no out-of-the-box logging mechanism that would allow detecting if a user has previously requested a page. You need a more "sophisticated" approach.
I can see two possible approaches. (There might be other ones!)
Approach #1 Use two Cloud Functions to:
Serve the page via Firebase Hosting, see Serve dynamic content and host microservices with Cloud Functions
Write to Firestore, after you have verified the user has previously requested a page.
More details:
For the first part, you will not actually serve dynamic content (I understand you plan to serve static pages through Hosting), but because this page is served through a Cloud Function, you will be able to save a unique token (e.g. a Firestore doc ID or any other UUID value) in, for example, Firestore, before sending back the page content.
Then, for the second part (writing to Firestore), the Cloud Function will first check that there is a document with the doc ID previously generated in the Firestore database, and if it is the case, will allow the write to the database (from the Cloud Function).
So, in this case, both Cloud Functions need to be HTTPS ones. You may be interested by this article which details the drawbacks of writing to Firestore through a CF.
Approach #2 Use Firestore security rules for the check before writing.
Do the same than the previous solution for serving the static pages;
Write directly to Firestore and implement a security rule that checks for the existence of a Firestore document with the doc ID saved in point 1. See the exists method.
I wanted to know how long lived the firebase storage URL's are.
I'm using firebase storage to host some static images. Currently using the file references to get the url's in app.
But would like to skip this step and just use the URL's instead. Does anyone know what if anything will cause first part of the URL to change?
(https://firebasestorage.googleapis.com/v0/b/)
Total URL
https://firebasestorage.googleapis.com/v0/b/{Project_ID}.appspot.com/o/{FILE_PATH}?alt=media&token={TOKEN}
The first part of the URL (https://firebasestorage.googleapis.com/v0/b/) will only change if the Firebase Storage API ever changes. Since this hasn't happened since its release in May 2016, and isn't planned to happen at any point at the moment, we can be certain it is a really infrequent occurrence.
The {Project_ID}.appspot.com/o/ will only change if you have a different project. For an existing project this will never change.
The {FILE_PATH} is the path to your file, so will only be different when referring to a different file.
And token={TOKEN} will only stop working if you revoke the token, as answer here: Firebase Storage getDownloadUrl's token validity
A download URL will last forever, or until its specific token is rejected from the Firebase console.
I have a scenario where a user can upload images to Firebase Storage, however I do not want them to be able to get a URL for these images (Copy Image Address). Instead I want to provide them with a blob.
When a user uploads the image client side, I get the download URL and store that in Firestore. When the user wants to see the image, I have a cloud function that downloads that image, and sends it to the user as a blob.
This works great for images the user uploads, however I also have a cloud function that is triggered automatically when an image is uploaded and generates a thumbnail.
How do I go about getting a permanent download URL from a Cloud Function/Node server for this generated image? I can get a signed one, but it's not what I need
You can't directly with Google Cloud Storage. The signed URL can't live more than 7 days
The longest expiration value is 604800 seconds (7days).
You can keep the link of the generated thumbnail image, but you have to either download it and serve it each time, or generate a signed url for using and displaying it, each time also. This second solution reduces the processing time and thus the cost.
If you want a permanent URL to access your data, you could make them public according to this documentation[1].
Just keep in mind that when accessing public data through the Google Cloud Platform Console, you must authenticate with Google. It can be accessed with any Google account, the account does not have to be associated with the project that contains the public data, nor must it be registered in the Cloud Storage service [2].
[1] https://cloud.google.com/storage/docs/access-control/making-data-public
[2] https://cloud.google.com/storage/docs/access-public-data
If you are using Django and django-storages[google] lib
Make the bucket public(not advisable for production)
Disable ACL and URL signing
Add these to the settings.py
GS_DEFAULT_ACL = None
GS_QUERYSTRING_AUTH = False
I'm using Google Storage to store profile pictures of my users. I have couple of thousands pictures.
Now the pictures are being saved in a bucket like so:
data/images/profiles/USER_ID.jpg
So the URL to an image:
https://storage.cloud.google.com/data/images/profiles/USER_ID.jpg
I don't want users being able to see someone else picture by just knowing their USER_ID, and still, It has to be the USER_ID for easier search from a developer's side.
I can't use Signed URL as my users do not have a google account, and the pictures from the storage are fetched from a Mobile Application.
Is there a way to keep the file names as they are in the storage, but simple hide the path+filename from the URL?
https://storage.cloud.google.com/fc720d5c05411b03e5e2a6692f8d7d61.jpg -> points to https://storage.cloud.google.com/data/images/profiles/USER_ID.jpg
Thank You
You have several options. Here are a few:
Have users request the URL for another user from the server, then have the server decide whether or not the user is allowed to see the image. If so, have the server (which does have a service account) generate a signed URL and pass it back to the user (or redirect to it). This way, although the user may know the user ID of another user and the URL of their image, they still can't see the image unless the server agrees that this is okay.
Use Firebase Storage to manage the images, which will still store them in GCS but will give you Firebase's auth support.
Proxy the images through your app, either an app engine app or something running in GCE or GKE. This lets you hide everything about the source of the image, including the user ID, but has the downside of requiring all of the data to pass through your service.
Reexamine your requirements. "Easier search on the developer's side" may not be as important as you think, and you need to way the benefit of that vs the cost of working around it.
Another option is Google Images API available on AppEngine. You can link your Cloud Storage objects with Google Images API and use benefits of this API - secure URLs, transform and resize images using URL parameters.
You only need to prepare servingURL for every image stored in GCS and persist this serving URL (for example in Google Datastore)
ImagesService imagesService = ImagesServiceFactory.getImagesService();
ServingUrlOptions suo = ServingUrlOptions.Builder
.withGoogleStorageFileName(gcsImageObjectPath)
.secureUrl(true);
String servingUrl = imagesService.getServingUrl(suo);
I have a situation where in Firebase Storage users store their avatar to /users/{uid}.jpg
If I then use the Storage API to get the download URL and display the image, it ends up being very slow to make the first request because the download URL is not cached anywhere.
So my solution is to get the DownloadURL when the user uploads the image and store that in Firebase allowing the client image provider to automatically cache the image which speeds up loads considerably.
However, there is one problem with this solution.
If a user replaces their avatar, the old link becomes broken instead of updated. I believe this is because a new token is generated each time something is uploaded for security reasons but these are of no benefit to me.
So my question is twofold:
1) How can I allow a user to upload an avatar to a path that is dedicated to them such as /users/{uid}.jpg, get a bare download URL that can be cached by the client, and have that URL remain the same even when the file changes at /users/{uid}.jpg
2) If this is not possible, what is the normal way to solve this issue?
Download URLs are opaque. The contents of the actual URL itself is an implementation detail of the system, and it's not supported to dig around in its contents. The URLs can't be dissected or composed.
You can use a storage trigger with Cloud Functions to automatically generate a signed URL whenever something changes in your storage bucket.
So instead of serving from a hard-coded URL, simply retrieve the URL from an updated value in the datastore (or any data storage system). Every time the user updates the avatar, simply store the new URL in the datastore and you can query for it when you need it.