In firebase, storage feature is comfortable. But I have two questions to use storage feature.
How to prevent users' unlimited file upload? e.g) if I can write /{uid}/{timestamp}/{fileName} path, user can upload till the storage is exploded.
How to trace legacy files and remove them? If a user upload chat message with image file and remove the image, file still exists in the storage. Is there any hook to manage this?
Thank you!
Good questions:
You can use Firebase Storage Security Rules to enforce arbitrary constraints for uploads. As long as the file has a piece of unique identifying information in the name or the metadata, you can tie that to a user and enforce time based access (among other things). A quick example:
match /files/{userId}/{fileName} {
// Only allow the authorized user to write to their files
allow write: if request.auth.uid == userId;
// Allow reads if the resource was created less than an hour ago
allow read: if resource.timeCreated < request.time + duration.value(60, "m");
}
You're asking for a garbage collection system :) The short answer is that it's best if you perform any file deletes along side any database object deletes on the client, but as we all know clients are unreliable. You can also set up a batch job (say every day) that reads the database data from a private backup, checks which blobs are no longer referenced, and deletes them. You can also wait for Google Cloud Functions, and perform actions (such as writing to the database on file upload) to sync this information. Lastly, you can use Object Lifecycle Management methods to clean up objects (delete, archive, etc.) in your bucket after a certain period of time.
Related
I am building web platform where user can upload photos and videos, I use firebase storage for it, but I come a cross one problem that I cant solve.
I want to manually check all of the files before making them public.
So my first idea was to make to separate folder, and when admin account approve the file, he can move it from /private/img.jpg to /public/img.jpg. But for some reason, firebase storage doesn't have function to move files.
So I come with new idea to use custom metadata, to block to read images that doesn't have metadata visibility set to 'public'. But for me, this solution doesn't seems to be the best... and more you aren't able to block listing file with specific metadata, so even when the file is private, everybody can see that the file exist, and the name, and this is potential risk for me.
So are there anybody, who can help find new solution?
Thank you :)
Approach #1:
You can use a Callable Cloud Function that you call from your app and which:
Checks that the user who is calling has the Admin role (use a specific Custom Claim);
If the user has the correct role, copies the file from the private "folder" to the public one (BTW see this answer on folders) by using the copy() method from the Google Cloud Storage Node.js SDK (which is the SDK to be used in a Cloud Function).
Approach #2:
Another totally different approach would be to take advantage of the new Cloud Storage Security Rules functionality which allows "building flexible permissions to protect access to storage files based on data within Firestore".
With this functionality, you would be able to manage access to the files through Documents in Firestore. For each file in Cloud Storage, create a document in Firestore which has the same name as the file and contains a boolean field named (for example) published. By default, it is false and if the Admin approves the file you change it to true.
The security rule would be along the following lines (untested):
service firebase.storage {
match /b/{bucket}/o {
match /.../{fileId} {
allow read: if
firestore.get(/databases/(default)/documents/files/$(fileId)).published
}
}
}
Don't forget to declare the correct Security Rules for the Firestore collection, in such a way only Admin users can approve files.
I am currently working on an iOS App that uses Cloud Firestore from Firebase.
I was wondering: what is the best way (cost, efficiency and security-wise) to upload some data to multiple Firestore documents simultaneously (or almost simultaneously)?
* The data I have to upload consists of the following: there are two users (User A is the user currently using the app, User B is the one whose profile is currently being seen by User A). If User A saves User B's profile, I must upload User B's UID to User A's Firestore Document. Then, I have to increase a counter in User A's Firestore Document. Finally, I must add User A's UID to User B's Firestore Document. - Note that with Firestore Document I mean either a document Field or a document Subcollection.
The choices are:
Upload everything from the client: seems the best method, cost-wise: it doesn't require extra Cloud Functions usage. I would create a Batch Operation and upload all the data from there*. The downside is that the client must be able to access multiple unrelated collections and documents.
Update one document from the client, then update everything else from Cloud Functions: this method is the best one efficiency and security-wise; the client only uploads data to the user's document*, without accessing unrelated collections and documents. Also, the client only has to upload a fraction of the data that it had to upload in the previous method, saving bandwidth and cellular data / WiFi usage. The downside is that the usage of Cloud Functions would increase, eventually resulting in more costs.
Update one document from the client, update the counter* from the client and then update everything else form Cloud Functions: this method is somewhat a hybrid between the first two, as I think that updating the counter from the client is more secure (Cloud Functions' .onWrite trigger may happen twice or more, increasing the counter multiple times?).
My first thought was to go with method 2, as it's far more secured and efficient, but I would like to have someone else's advice too, before "wasting" too much time coding something wrong.
I hope this isn't any kind of duplicate, as I couldn't find anything that answered my question with enough specificity.
Any advice would be much appreciated. Thank you.
I would follow the third approach: updating from the client the current user collections (the saved_profiles collection and the counter field), which are private and only accessible by this user (configure Firestore Security Rules) and updating the other user's collection (users_who_saved_my_profile) with a triggered Cloud Function. As these operations are not controlled by security rules, they can access any part of the database. This way no unnecessary permissions are granted to any user.
I know there are several questions regarding this (e.g. https://stackoverflow.com/a/52808572/3481904), but I still don't have a good solution for my case.
My application has Groups, which are created/removed dynamically, and members (users) can be added/removed at anytime.
Each Group has 0..N private files (Firebase Storage), saved in different paths (all having the prefix groups/{groupId}/...).
In Firestore Security Rules, I use get() & exists() to know if the signed-in-user is part of a group. But I cannot do this in the Firebase Storage Security Rules.
The 2 proposed solution are:
User Claims:
but the token needs to be refreshed (signing out/in, or renewing expired token) which is not acceptable for my use case, because users need to have access immediately once invited. Also, a user can be part of many groups, which can potentially grow over 1000 bytes.
File Metadata:
but Groups can have N files in different paths, so I will need to loop-list all files of a group, and set the userIds of the group-members in the metadata of each file, allowing access to it. This would be an action triggered by Firestore (a Firebase Function), when a member is added/removed.
I don't like this approach because:
needs to loop-list N files and set metadata for each one (not very performant)
To add new files, I think I would need to set create to public (as there is no metadata to check against yet), and then a Function would need to be triggered to add the userIds to the metadata
there might be some seconds of delay to give files access, which could cause problems in my case if the user opens the group page before that time, having a bad experience
So, my questions are:
Is there a better way?
If I only allow the client to get and create all files when authenticated (disallowing delete and list), would this be enough for security? I think that there might be a chance that malicious hackers can upload anything with an anonymous user, or potentially read all private group files if they know the path...
Thanks!
If custom claims don't work for you, there is really no "good" way to implement this. Your only real options are:
Make use of Cloud Functions in some way to mirror the relevant data from Firestore into Storage, placing Firestore document data into Storage object metadata to be checked by rules.
Route all access to Storage through a backend you control (could also be Cloud Functions) that performs all the relevant security checks. If you use Cloud Functions, this will not work for files whose content is greater than 10MB, as that's the limit for the size of the request and response with Cloud Functions.
Please file a feature request with Firebase support to be allow use of Firestore documents in Storage rules - it's a common request. https://support.google.com/firebase/contact/support
I had similar use case, here’s another way to go about it without using file metadata.
Create a private bucket
Upload files to this bucket via cloud function
2a. validate group stuff here then upload to above bucket.
2b. Generate a signed url for uploaded file
2c. Put this signed URL in Firestore where only the group members can read it (eg. /groups/id/urls)
In UI get the signed URL from firestore for given image id in a group and render the image.
Because we generate the signed URL and upload file together there will be no delay in using the image. (The upload might take longer but we can show spinner)
Also we generate the URL once so not incurring any B class operations or extra functions running every time we add new members to groups.
If you want to be more secure you could set expiry of signed urls quite short and rotate them periodically.
I want a group of users to access files stored in Cloud Storage, but I want to make sure they are authorized. Do the unique ids generated by Firestore create enough protection to make them unguessable?
I have my files stored using this structure in Firestore:
/projects/uidOfProject/files/uidOfFile
I made sure that only authorized users can view uidOfProject and uidOfFile using Firestore Rules.
I store that actual files in Storage here:
/projects/uidOfProject/files/uidOfFile
But, I cannot lock down this path to only the authenticated user id, because other users can access this project.
Is the fact that I have two unique ids enough to prevent a user who doesn't have access from finding these files? What are the odds of a user figuring out both the uidOfProject and uidOfFile and manipulating that file? Is there a more secure way of doing this? I know cloud functions could offer a solution, but at a cost of speed.
Do the unique ids generated by Firestore create enough protection to
make them unguessable?
Security through obscurity is NOT security. Good reference to read.
Unguessable, probably. However, due to the somewhat public nature of URLs, logfiles, information leaks, "hey check this out" favors, etc. objects that are not properly protected will be discovered.
If only users of the project can access the files, can they also list the files? If yes, curiosity might take place browsing to see what is there.
Trying to workout how I should be storing files each users uploaded files. The files need to be private so only the person who uploaded it can read/write.
My question is, should I be creating one bucket per userId and securing the bucket to that user, or am I supposed to dump everything in a single bucket and make use of the GCS ACL permissions on each file?
Putting each users files in their own bucket seems to make sense but just looking for some clarification around best practises.
In general, there is no need to create a new bucket for each user. That will not scale (in terms of effort) as you'll spend a lot of time administering all these buckets.
You should start with the documentation on Cloud Storage security rules. Especially the page on user based security. You use security rules to determine who can do what to the various files in storage. How you actually write those rules is going to depend on how you want to structure the files. Typically you use the Firebase Auth user id in the path of the files, and you use a wildcard in the rules to protect based on that uid.