I want a group of users to access files stored in Cloud Storage, but I want to make sure they are authorized. Do the unique ids generated by Firestore create enough protection to make them unguessable?
I have my files stored using this structure in Firestore:
/projects/uidOfProject/files/uidOfFile
I made sure that only authorized users can view uidOfProject and uidOfFile using Firestore Rules.
I store that actual files in Storage here:
/projects/uidOfProject/files/uidOfFile
But, I cannot lock down this path to only the authenticated user id, because other users can access this project.
Is the fact that I have two unique ids enough to prevent a user who doesn't have access from finding these files? What are the odds of a user figuring out both the uidOfProject and uidOfFile and manipulating that file? Is there a more secure way of doing this? I know cloud functions could offer a solution, but at a cost of speed.
Do the unique ids generated by Firestore create enough protection to
make them unguessable?
Security through obscurity is NOT security. Good reference to read.
Unguessable, probably. However, due to the somewhat public nature of URLs, logfiles, information leaks, "hey check this out" favors, etc. objects that are not properly protected will be discovered.
If only users of the project can access the files, can they also list the files? If yes, curiosity might take place browsing to see what is there.
Related
Short version
I'm using firestore to build an application where users can create courses, add resources to these courses, and then have other users sign up to their courses which would give them access to all of its resources.
I'm struggling to get the security rules set up so that only the course organiser or course members can access the resources.
My main problem is allowing a user to list all resources within a course they have signed up for, as list requests don't allow queries in the security rules which I need to check if that user is a member of that course.
What is the best approach that would allow hundreds (or potentially thousands) of course members to easily list all of a courses resources, while ensuring it's not possible for non-members to access it?
In the limitations section of the docs it specifically mentions moving the roles into a separate collection for large or complex groups, but doing so will require the security rules performing a query to check access which is not possible for list requests.
Full verison
I'm creating a prototype web application with clients directly accessing firestore. Eliminating the need for a backend to handle simple data access is refreshing, but I'm now really struggling to work out the correct data structure and approach for my use-case, particularly around implementing appropriate security rules.
I've worked extensively with relational databases, but am quite new to nosql databases and firestore specifically which doesn't help.
Concept
The main collections in my application are users, courses, and resources.
Users can create courses within which they create learning resources, other users can then sign up for these courses and therefore have access to all of the resources within them.
As some courses will be invite only it can not be possible for a user to get access to resources for a course they are not a member of.
The main requirements are:
A course organisor needs full read and write access to their courses and resources.
A course member needs full read access to their courses and resources.
Organisors and members both need to be able to easily list all of their courses, and for each course list all of their resources.
Organisors and members should not be able to retrieve information about courses or resources that they do not own or belong to.
My approach
To start with my approach is to heavily utilise sub-collections.
For the organisor perspective users have courses which have resources, and so I modelled it that way.
users -> courses -> resources
Security rules for the organisor were pretty straightforward here too, as the user id is part of the document path it can be easily checked with the user uid available (thanks firebase auth).
i.e
match /users/{userId}/courses/{courseId}/resources/{resourceId} {
allow read, write, update, delete: if (request.auth.uid == userId)
}
Giving members read-only access proved a little bit more difficult though. I needed somewhere appropriate to store that mapping.
I ended up trying to use a sub-collection of users to store that, so course memberships could be looked up with:
/users/{userId}/course_memberships/{courseId}
The theory being I could then write a security rule to grant read access to courses and resources that looked like:
match /users/{userId}/courses/{courseId}/resources/{resourceId} {
allow read: if exists(/databases/$(database)/documents/users/$(request.auth.uid)/course_memberships/${courseId})
}
However this only works when retrieving a single document, list requests won't evaluate a query in the security rules, therefore this set up doesn't allow a course member to view a list of all resources in a given course.
I can't work out what the correct approach here is, the only options I see are:
Create a separate collection hierarchy for each member that contains all of their courses and all of their resources, including key information used for listing their resources, and synchronise these using cloud functions. I understand that nosql is about storing denormalised data, but this feels like a lot of extra data, particularly when it's only necessary for security rules.
Add the uid of every member to an array on every resource (could
be in the thousands for each), which also feels like a lot of extra data and it would
be exposing those ids to all other members which seems wrong.
Is there a better approach here, or is firestore not well suited for this type of problem?
Hopefully this question hasn't been answered already, I did a search and couldn't find something covering quite the same scenario.
A read rule can be broken into get and list, and the latter can apply to queries.
Example:
allow list: if [condition];
More information can be found here.
I know there are several questions regarding this (e.g. https://stackoverflow.com/a/52808572/3481904), but I still don't have a good solution for my case.
My application has Groups, which are created/removed dynamically, and members (users) can be added/removed at anytime.
Each Group has 0..N private files (Firebase Storage), saved in different paths (all having the prefix groups/{groupId}/...).
In Firestore Security Rules, I use get() & exists() to know if the signed-in-user is part of a group. But I cannot do this in the Firebase Storage Security Rules.
The 2 proposed solution are:
User Claims:
but the token needs to be refreshed (signing out/in, or renewing expired token) which is not acceptable for my use case, because users need to have access immediately once invited. Also, a user can be part of many groups, which can potentially grow over 1000 bytes.
File Metadata:
but Groups can have N files in different paths, so I will need to loop-list all files of a group, and set the userIds of the group-members in the metadata of each file, allowing access to it. This would be an action triggered by Firestore (a Firebase Function), when a member is added/removed.
I don't like this approach because:
needs to loop-list N files and set metadata for each one (not very performant)
To add new files, I think I would need to set create to public (as there is no metadata to check against yet), and then a Function would need to be triggered to add the userIds to the metadata
there might be some seconds of delay to give files access, which could cause problems in my case if the user opens the group page before that time, having a bad experience
So, my questions are:
Is there a better way?
If I only allow the client to get and create all files when authenticated (disallowing delete and list), would this be enough for security? I think that there might be a chance that malicious hackers can upload anything with an anonymous user, or potentially read all private group files if they know the path...
Thanks!
If custom claims don't work for you, there is really no "good" way to implement this. Your only real options are:
Make use of Cloud Functions in some way to mirror the relevant data from Firestore into Storage, placing Firestore document data into Storage object metadata to be checked by rules.
Route all access to Storage through a backend you control (could also be Cloud Functions) that performs all the relevant security checks. If you use Cloud Functions, this will not work for files whose content is greater than 10MB, as that's the limit for the size of the request and response with Cloud Functions.
Please file a feature request with Firebase support to be allow use of Firestore documents in Storage rules - it's a common request. https://support.google.com/firebase/contact/support
I had similar use case, here’s another way to go about it without using file metadata.
Create a private bucket
Upload files to this bucket via cloud function
2a. validate group stuff here then upload to above bucket.
2b. Generate a signed url for uploaded file
2c. Put this signed URL in Firestore where only the group members can read it (eg. /groups/id/urls)
In UI get the signed URL from firestore for given image id in a group and render the image.
Because we generate the signed URL and upload file together there will be no delay in using the image. (The upload might take longer but we can show spinner)
Also we generate the URL once so not incurring any B class operations or extra functions running every time we add new members to groups.
If you want to be more secure you could set expiry of signed urls quite short and rotate them periodically.
I'm trying to determine the best way to reference a Firebase Storage (Google Cloud Storage) file in a direct-read database like Realtime Database or Cloud Firestore. Since a read operation to this database does not benefit from a backend that can issue tokens and cache image URLs, it is not clear to me what the most performant way is to store these references.
I have come up with a few options and none of them are a clear winner.
Store a path like /images/foo.jpg to the database, and use Storage Client SDK to generate a tokenized path with storage.bucket().getDownloadURL("/images/foo.jpg").
Pros: Secure & simple.
Cons: Network call for every single image you want to display hurts performance considerably.
Store a tokenized path like https://firebasestorage.googleapis.com/v0/b/storage-bucket-823743.appspot.com/o/images%2Ffoo.jpg?alt=media&token=c6da1e33-f3ff-41e2-a6f0-bdb475a2f6d9 with a super long TTL.
Pros: No extra fetch on the client.
Cons: long string stored in expensive RTDB. What if that token is revoked by mistake? The database is now broken.
Store a path like /images/foo.jpg to the Database and use public storage rules. Reconstruct into a custom static URL like https://firebasestorage.googleapis.com/v0/b/storage-bucket-823743.appspot.com/o/images%2Ffoo.jpg?alt=media
Pros: Tiny database use, no extra client fetch, explicit public access, no token to lose.
Cons: URL encoding is super flaky, Storage could change their URL format and we'd be out of luck.
So, those are the options I've come up with, and there may be more. Any suggestions for how to solve this issue? The issue is unique because the Firebase databases don't have the benefit of a custom server to handle a token/caching layer to resolve this problem.
There is no single "best way" to store these paths. It all depends on your use-case, your preferences, and the context in which you're implementing it.
I typically use :
If I need access to the files to be secured, I store the image path (like in your #1), and then use the Firebase SDK to access the file.
If I don't need access to the files to be secured, I store the image path and the download URL. This way I can find the image easily based on the path, and use the download URL in non-secured clients.
The con's you mention for these are simply not affecting me. I'd recommend you take a similar approach, and report back when the problem actually occurs.
Trying to workout how I should be storing files each users uploaded files. The files need to be private so only the person who uploaded it can read/write.
My question is, should I be creating one bucket per userId and securing the bucket to that user, or am I supposed to dump everything in a single bucket and make use of the GCS ACL permissions on each file?
Putting each users files in their own bucket seems to make sense but just looking for some clarification around best practises.
In general, there is no need to create a new bucket for each user. That will not scale (in terms of effort) as you'll spend a lot of time administering all these buckets.
You should start with the documentation on Cloud Storage security rules. Especially the page on user based security. You use security rules to determine who can do what to the various files in storage. How you actually write those rules is going to depend on how you want to structure the files. Typically you use the Firebase Auth user id in the path of the files, and you use a wildcard in the rules to protect based on that uid.
Let's say I'm developing app like Instagram: for iOS, Android and Web. I decided to use Google Firebase as it really seems to simplify the work.
The features user needs in the app are:
Authorization/Registration
Uploading photos
Searching for other people, following them and see their photos
I come from traditional "own-backend" development where I do need to setup a server, create database and finally write the API to let the frontend retrieve the data from the server. That's the reason why it's unclear to me how it all works in Firebase.
So the question is how can I create such app:
Should I create my own API with cloud functions? Or it's ok to work with the database directly from the client-side?
If I work with the database directly why do I need cloud functions? Should I use them?
Sorry for such silly questions, but it is really hard to get from scratch.
The main difference between Firebase and the traditional setup you describe is that with Firebase, as far as the app developer is concerned, the client has direct access to the database, without the need for an intermediate custom API layer. Firebase provides SDKs in various languages that you would typically use to fetch the data you need / commit data updates.
You also have admin SDKs that you can use server-side, but these are meant for you to run some custom business logic - such as analytics, caching in an external service, for exemple - not for you to implement a data fetching API layer.
This has 2 important consequences:
You must define security rules to control who is allowed to read/write at what paths in your database. These security rules are defined at the project level, and rely on the authenticated user (using Firebase Authentication). Typically, if you store the user profile at the path users/$userId, you would define a rule saying that this node can be written to only if the authenticated user has an id of $userId.
You must structure your data in a way that makes it easily readable - without the need for complex database operations such as JOINs that are not supported by Firebase (you do have some limited querying options tough).
These 2 points allow you to skip the 2 main roles of traditional APIs: validating access and fetching/formatting the data.
Cloud functions allow you to react to data changes. Let's say everytime a new user is created, you want to send him a Welcome email: you could define a cloud function sending this email everytime a new node is appended to the users path. They allow you to run the code you would typically run server-side when writes happen, so they can have a very broad range of use-cases: side-effects (such as sending an email), caching data in an external service, caching data within Firebase for easier reads, analytics, etc..
You don't really need a server, you can access the database directly from the client, as long as your users are authenticated and you have defined reasonable security rules on Firebase.
In your use case you could, for example, use cloud functions to create a thumbnail when someone uploads a photo (Firebase Cloud Functions has ImageMagick included for that), or to denormalize your data so your application is faster, or to generate logs. So, basically you can use them whenever you need to do some server side processing when something changes on your database or storage. But I find cloud functions hard to develop and debug, and there are alternatives such as creating a Node application that subscribes to real time changes in your data and processes it. The downside is that you need to host it outside Firebase.
My answer is definitely NOT complete or professional, but here are the reasons why I choose Cloud Functions
Performance
You mentioned that you're writing an instagram-like mobile device app, then I assume that people can comment on others' pictures, as well as view those comments. How would you like to download comments from database and display them on users' devices? I mean, there could be hundreds, maybe thousands of comments on 1 post, you'll need to paginate your results. Why not let the server do all the hard work, free up users' devices and wait for the results? This doesn't seem like a lot better, but let's face it, if your app is incredibly successful, you'll have millions of users, millions of comments that you need to deal with, server will do those hard jobs way better than a mobile phone.
Security
If your project is small, then it's true that you won't worry about performance, but what about security? If you do everything on client side, you're basically allowing every device to connect to your database, meaning that every device can read from/write into your database. Once a malicious user have found out your database url, all he has to do is to
firebase.database().ref(...).remove();
With 1 line of code, you'll lose all your data. Okay, if you say, then I'll just come up with some good security rules like the one below:
This means that for each post, only the owner of that post can make any changes to it or read from it, other people are forbidden to do anything. It's good, but not realistic. People are supposed to be able to comment on the post, that's modifying the post, this rule will not apply to the situation. But again, if you let everybody read/write, it's not safe again. Then, why not just make .read and .write false, like this:
It's 100% safe, because nobody can do anything about anything in your database. Then, you write an API to do all the operations to your database. API limits the operations that can be done to your database. And you have experience in writing APIs, I'm sure you can do something to make your API strong in terms of security, for example, if a user wants to delete a post that he created, in your deletePost API, you're supposed to authenticate the user first. This way, 'nobody' can cause any damage to your database.