I have seen a firebase api in which you can create firebase projects and it occurred to me for example, when a user wants to counter a SaaS, make it easier for me to create a project and connect it to their SaaS but I have the following questions:
If there are supposed to be limits to creating firebase projects in a google account, will there be a time when databases can no longer be created for new users?
If the above is true, how can this be solved?
I have seen that you can ask for more projects, but how many can I have?
I know that with firestore, I can model data and only in a database have all the information, but for example, each user may have special requirements in their system and also provide security and information saving operations that would be impossible if all information is in a single database.
Thanks for the help.
EDIT
"How many projects can you have as a developer?" Yes, that's what I mean, having all the user bases in a single firebase google account. For example, on a web page, the user wants to pay for the application, with firebase admin and google cloud functions, I can automatically create a project and have all the databases in a google account. And what I want to know is if you can have multiple databases. I have seen that you can ask google to give you permission to host several projects but, for example, can you have more than 100 projects or even 1000? (I may never have reached that number, but in that case, I would like to know the limits that can be reached).
Edit 2
This first structure I have all the documents in a "universal" collection and I identify them with an id to know the user who used it.
But the second structure seems to me more organized and clean, in addition to the fact that users at no time need to share information among others. I have read that having nested collections is not good but over the years and the progress that firestore has had, this is no longer or was a problem only that the limitation is that you can only have up to 100 collections anidades but I never think to reach that quantity.
Inside of list, have all products for that user, because inside of document, only can have 1 mb of data and download 1 mb and is not the best option.
in the firestore documentation I see that it does not reveal other problems, it only says that it is difficult to delete the subcollection, but I do not allow users to do that and if I need to delete the subcollections, with the Google Cloud functions i can do it.
So, the second structure is more intuitive for me, but is the best option for that? or actually firestore is not good for this strcucutre?
Related
Context: I am total Google Cloud begginer and I have just convinced my company headers to use Firestore Realtime Database for pushing transaction status to our mobile application. We have around 4 millions users that will use significantly our application for small money transfers. Now-a-days we use the concept of polling from Android/IOS to our Microservice endpoints and it will replaced by Firebase SDK imported to our Mobile app which will listen/observe to our Firestore Collection following few Firestore Rules. Since all money transfer will be confirmed/denied in short time (from few seconds to 1 or 2 minutes) the idea of replacing polling by a real reactive approach straigh from Firestore sounded and is already ongoing coding.
The issue: Firstly I don't what to compare solutions. It is just my reality: the prodution support operators must look after our internal Dashboard. Isn't allowed to them look at Google Dashboard Console (please accept this for this question). I need get on demand metrics of our FIrestore. It is nothing to do with Google pricing. It is just our demand: they want to see metrics like:
how many users listening at the same time now
how many users took some exception during connection
is there any user holding connection for more than X minute
when was the connection pick this morning
any exception of any type surrounding our Firestore database
I read Code Samples carefully follow the sample step-by-step trying to figure out some idea if there is some API providing the answers I am looking for.
So, my straight question is: is there such type of Google API providing metrics about my Firestore Database? Maybe following the same idea we found in Performance Monitor which works on Mobile side also some similar aproach on Firestore side.
*** Edited
Future readers may find worth read also about a way to get Firestore metrics info striagh from curl/postman
A couple of things: You mentioned both Firestore and Realtime Database; just wanted to make sure that you are aware that those are two different databases offered under the Firebase umbrella.
how many users listening at the same time now
is there any user holding connection for more than X minute
Yes, there's a dashboard: https://support.google.com/firebase/answer/6317517?hl=en. Including lots of options, like users active in the last 30 mins.
how many users took some exception during connection
any exception of any type surrounding our Firestore database
Yes, you can track errors and other logging via Stack Driver logging. These can give you reports on your cloud functions.
https://cloud.google.com/functions/docs/monitoring
Where can I find Stackdriver in Firebase console?
when was the connection pick this morning
For this one, I'm not sure if you mean A. when did somebody log on in the morning, or B. what was the time that there was the peak \ most usage. If B see 1. If A,
Real-time database has the concept of presence, which lets you know if a user is currently logged in or not. See examples here from the official documentation:
https://firebase.google.com/docs/firestore/solutions/presence
and this post
How to make user presence mechanism using Firebase?
Also applies to your
is there any user holding connection for more than X minute
..............
Edit in response to comments: I believe you are experiencing the XY problem https://meta.stackexchange.com/questions/66377/what-is-the-xy-problem where you are focused on a particular solution, even though your problem has other solutions. User metrics, database events, and errors are all accessible through both dashboards and cloud functions. You can cURL cloud functions if you wish, or set up cron functions to auto report, or set up database trigger functions to log errors. So, while the exact way you want this to work may not exist, you just need to connect existing tools to get the result you want.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 2 years ago.
Improve this question
Is there any situation where it makes sense to use both realtime and firestore in conjunction? What situations lend themselves more favorably to firebase realtime vs firestore, or a combination? I keep reading about horror stories of people getting hit with huge costs is there anyway to test before hand.
For context I am looking to work with an auction based market place of over 50,000 products. The idea is to be able to filter those products as needed, create, modify and delete bids for those products, favorite items and retrieve Users bids. From what I was reading the general suggestion (to keep cost low) for market places using firebase seems to suggest storing products in realtime db and the user objects, sales etc. in firestore. Kinds of queries I will need are find products with the lowest/highest bids, most favorited items, as well as fetching users current and purchases.
When would it be optimal to store in realtime vs firestore, from a cost perspective?
My current logic is to store the product objects in realtime since they will be referenced more frequently. Alternatively I am thinking it makes sense to store the user info, their bids, and purchases in one document in a firestore as that would incur just one read cost, and for a highly active user could result in a large amount of data to be transferred. Where I am confused comes with things like viewing the previous sales of a given product vs getting a user's previous sales, should sales be stored in realtime (as their own object or embedded in the product object) or firestore (embedded in the User doc) or both?
Looking at your app that you plan to make, let have a short talk regarding it.
A bidding app, first someone wants to sell their stuff so they post it in your app. Then every single user of your app may see it start bidding on it. Now as I don't know how your app is going to work but here's my assumption you will store the data of bidders and the bids they make in firebase realtime database.
This will involve lots of read, write operations. Now Firestore does offer you 20K operations/day, but if you cross the limit it will barely cost you $0.18/100K writes and $0.06/100K reads. Now the choice entirely depends on scale of your app. If your app has large number of audience, go for Realtime-Database. You can download upto 10GB of data per month for free and a dollar per GB beyond that. But this has a catch, if you stick to the spark plan, you can have only 100 simultaneous connections to the database so I doubt the performance if you have large number of users. It can go upto 200K using Blaze plan and that too per database. So if you create another database you will have more. I will personally suggest create multiple databases as per the region or any parameter to spread the traffic. [Again it's upto how many people use your app]
In my opinion, you should use the Firebase Realtime database your app. [Make sure you utilize the firebase storage as well for storing large photos of the things on sale].
Lastly, use firestore when you have less number of operations but are larger in size. Use firebase realtime database when you have many small tasks like updating the highest bid value or number of users currently bidding for a particular thing, use Realtime DB.
In my opinion, go for realtime database. I too use it for some game stuff like to store user stats and update it as the user progresses. This involves lots of read/write/update/delete operations so I stick with realtime-database.
When to use Firestore along with the real-time database?
As you have mentioned user profile, I will suggest use Firestore to store those credentials. Because user's won't generally update their profile so this won't cost much writes. Also the bidders would be much interest in bidding rather than watching others profiles. So even if if a few users check other's profile. This won't cost you much reads. But even if your app is designed in such a way that bidder must check seller's profile once, then firestore will definitely help you reduce usage of realtime database's [GB Downloaded] quota.
Every time someone queries data from your realtime database, you consume some part of the 10 GB of free download limit.
Also as I have mentioned the simultaneous connections to the database, if you host user profile data in Firestore then firestore will take care of profile visits so that bidders get faster response from your application. Just make sure you utilise all the free quotas from firebase storage, firestore and the realtime database and make sure your app is designed in such a way that it spreads traffic evenly between all services. Use the cloud functions on your back-end, and don't make your application [.apk] too heavy on client side as the app needs a lot to code.
So the conclusion, use firestore to store data which won't be accessed frequently like the user credentials and whatever stuff they have on sell. Use realtime database to store bidding data. Oh and yes, if you also want to store some stats like how many purchases has someone made or some information that changes too frequently put that in firebase realtime database. You can simply create a child node users/${username} and keep the frequent stuff in realtime database. This won't cost you much storage but take of that download limit. Shouldn't be expensive much especially talking of your app is going to address 50000 products XD.
I am looking to work with an auction based market place of over 50,000 products.
If you have comparatively less number of users, realtime database is sufficient but who knows when there may be a huge rise in your app users. So it's better to spread the data in both Firestore and Real-time database as mentioned above.
Just a caution: This is what I faced, then searched over stackoverflow and found this. Firestore counts READS even if you are just scrolling over the data tab in firestore. So make sure you don't just get surfing over there. I made 2 writes and was just looking at how the data is being stored and I already got 27 reads ...
I have a question about the best way to structure my Firebase database for my iOS app. The basic structure is that there are users and posts. The app will open and the user who is logged in will see all the posts that he/she has uploaded to the database (I may update this down the road so that users can see posts of users they follow). Is it better to have each post have a userId on it? Or is it better to have the an array of posts associated to each user?
If you can shard/partition the data so that you don't need to query a potentially very long list of that, that is always preferable with the realtime database.
So in your case, if you know that you want to show the user a list of their own posts as a starting point, it is a good idea to model that list in your database. You'd typically call this the user's feed or wall, depending on what your social network of choice is.
If you then later want to also show the posts of users they follow, you might want to store those posts in the user's wall. This type of data duplication may seem unnatural at first, but is quite common in NoSQL databases. In fact, this is the exact model that Firebase uses in its class FireFeed example.
To learn more about NoSQL data modeling, see:
NoSQL data modeling.
Firebase for SQL developers.
Getting to know Cloud Firestore, which is about Firebase's newer Firestore database, but has great tips that apply to the Realtime Database too.
Let's say I'm developing app like Instagram: for iOS, Android and Web. I decided to use Google Firebase as it really seems to simplify the work.
The features user needs in the app are:
Authorization/Registration
Uploading photos
Searching for other people, following them and see their photos
I come from traditional "own-backend" development where I do need to setup a server, create database and finally write the API to let the frontend retrieve the data from the server. That's the reason why it's unclear to me how it all works in Firebase.
So the question is how can I create such app:
Should I create my own API with cloud functions? Or it's ok to work with the database directly from the client-side?
If I work with the database directly why do I need cloud functions? Should I use them?
Sorry for such silly questions, but it is really hard to get from scratch.
The main difference between Firebase and the traditional setup you describe is that with Firebase, as far as the app developer is concerned, the client has direct access to the database, without the need for an intermediate custom API layer. Firebase provides SDKs in various languages that you would typically use to fetch the data you need / commit data updates.
You also have admin SDKs that you can use server-side, but these are meant for you to run some custom business logic - such as analytics, caching in an external service, for exemple - not for you to implement a data fetching API layer.
This has 2 important consequences:
You must define security rules to control who is allowed to read/write at what paths in your database. These security rules are defined at the project level, and rely on the authenticated user (using Firebase Authentication). Typically, if you store the user profile at the path users/$userId, you would define a rule saying that this node can be written to only if the authenticated user has an id of $userId.
You must structure your data in a way that makes it easily readable - without the need for complex database operations such as JOINs that are not supported by Firebase (you do have some limited querying options tough).
These 2 points allow you to skip the 2 main roles of traditional APIs: validating access and fetching/formatting the data.
Cloud functions allow you to react to data changes. Let's say everytime a new user is created, you want to send him a Welcome email: you could define a cloud function sending this email everytime a new node is appended to the users path. They allow you to run the code you would typically run server-side when writes happen, so they can have a very broad range of use-cases: side-effects (such as sending an email), caching data in an external service, caching data within Firebase for easier reads, analytics, etc..
You don't really need a server, you can access the database directly from the client, as long as your users are authenticated and you have defined reasonable security rules on Firebase.
In your use case you could, for example, use cloud functions to create a thumbnail when someone uploads a photo (Firebase Cloud Functions has ImageMagick included for that), or to denormalize your data so your application is faster, or to generate logs. So, basically you can use them whenever you need to do some server side processing when something changes on your database or storage. But I find cloud functions hard to develop and debug, and there are alternatives such as creating a Node application that subscribes to real time changes in your data and processes it. The downside is that you need to host it outside Firebase.
My answer is definitely NOT complete or professional, but here are the reasons why I choose Cloud Functions
Performance
You mentioned that you're writing an instagram-like mobile device app, then I assume that people can comment on others' pictures, as well as view those comments. How would you like to download comments from database and display them on users' devices? I mean, there could be hundreds, maybe thousands of comments on 1 post, you'll need to paginate your results. Why not let the server do all the hard work, free up users' devices and wait for the results? This doesn't seem like a lot better, but let's face it, if your app is incredibly successful, you'll have millions of users, millions of comments that you need to deal with, server will do those hard jobs way better than a mobile phone.
Security
If your project is small, then it's true that you won't worry about performance, but what about security? If you do everything on client side, you're basically allowing every device to connect to your database, meaning that every device can read from/write into your database. Once a malicious user have found out your database url, all he has to do is to
firebase.database().ref(...).remove();
With 1 line of code, you'll lose all your data. Okay, if you say, then I'll just come up with some good security rules like the one below:
This means that for each post, only the owner of that post can make any changes to it or read from it, other people are forbidden to do anything. It's good, but not realistic. People are supposed to be able to comment on the post, that's modifying the post, this rule will not apply to the situation. But again, if you let everybody read/write, it's not safe again. Then, why not just make .read and .write false, like this:
It's 100% safe, because nobody can do anything about anything in your database. Then, you write an API to do all the operations to your database. API limits the operations that can be done to your database. And you have experience in writing APIs, I'm sure you can do something to make your API strong in terms of security, for example, if a user wants to delete a post that he created, in your deletePost API, you're supposed to authenticate the user first. This way, 'nobody' can cause any damage to your database.
Is there a way to protect the database from deletion? I mean it's very easy to click on the "x" next to the root node. This would destroy the whole app and cause an enourmous mess to deal with.
How to deal with this fragility?
EDIT:
Let's assume I have two firebase accounts: one for testing and one for the launched app. I regularly log in and out to use the other one. On the test account I delete whole nodes on a regular basis. An activated password protection would avoid a very expensive confusion of the two accounts.
If you give a user edit access to the Firebase Console of your project, the user is assumed to be an administrator of the database. This means they can perform any write operation to the database they want and are not tied to your security rules.
As a developer you probably often use this fact to make changes to your data structure while developing the app. For application administrators, you should probably create a custom administrative dashboard, where they can only perform the actions that your code allows.
There is no way to remove specific permissions, such as limiting the amount of data they can remove. It could be a useful feature request, so I suggest posting it here. But at the moment: if you don't trust users to be careful enough with your data, you should not give them access to the console.
As Travis said: setting up backups may be a good way to counter some of this anxiety.