I'm starting a project with Cloud Firestore and I added my first collections and documents.
In order to be sure to use the right tool, I tried to search if there were limitations with Cloud Firestore.
I saw there were some limitations in bandwidth, number of commits etc.. but I didn't find (or didn't understand) if there was a limitation with the size of the database (number of collections / documents).
Is there a limitation? If yes, does it exist plans/bundles to extend those limitations ?
Best Regards,
Cloud Firestore scales effortlessly. It will store as much data as you're willing to put into it (and pay for). Practically speaking, volume of data is not a concern.
Related
I am wondering why aggregation (like SUM) is not built into Firestore when other NoSQL databases like MongoDB has them. Is it inherent to the design of firebase? Do you think it can be added soon?
This is a good question, actually.
Firestore was built for some certain use cases, that are not the use cases MongoDB is built for. MongoDB can be used for a lot of use cases, even those covered by Firestore.
Basically, the main idea behind the dev team was that they wanted to build a document database, easy to use, managed and lightweight. This led to making it without features like shardings, or aggregations, and so on, but still, the dev team knew some of them would be useful.
So they decided to leverage the possibilities offered by a cloud platform, and built it to support everything the platform (Firebase) could offer: Firebase Functions.
So, in the end, the answer is:
no, Firestore will never support aggregation functions. Or at least, that's not in the plans, as for now;
it is still possible to obtain a SUM, using a Firebase Function that will trigger everytime you perform a write operation, so you can update the SUM value. You'll need to store the value somewhere else in your firestore database, but that's a pretty good solution, and it is even documented here as an example. Only thing you have to remind is: the sum value will be "eventually consistent". It means that there could be instants when the persisted value is different from the real value, because the trigger is yet to fire, or because the function that will update the value has yet to finish, but this is the way Google designed Firestore and Firebase, so it's a good practice and pattern we can use.
There is also a third option when it comes to aggregations queries in Firestore (other than counters and client side aggregations)
You could alos also mirror your database to some OLAP capable database and do your aggregations there. Either build the sync mechanism yourself by listening to your data or use dedicated services.
Currently there are two Firebase Extensions you can install that do exacly that for you. "Firestore Big Query Stream" and "Firea.io". Both automatically mirror your data to another database and then allow for queries over that database.
This will allow you to use much more powerfull query languages (SQL / MQL) over your data.
(full disclosure I am one of the founders of Firea.io)
Despite not performing too many read/write operations - 170r, 4w to my Firestore database, I appear to have a huge number of API calls - 15,093 (See image below). The cause of this high calls to read/write ratio should be accounted to my application's use of network streams. My question is, should this be considered as a billable metric, or should I not worry if this runs into the millions so long as read/write are within limits (theoretically, I've never seen this happen on my own account).
I'm inclined to believe that I needn't worry about this metric, as I can't seem to find it under either the Firebase or GCP quotas page.
It may also be considered that I use the google maps and directions API from GCP, although this aren't nearly used as much as Firestore.
Thanks.
According to this doc, you are charged for the following:
The number of documents you read, write, and delete.
The amount of storage that your database uses, including overhead
for metadata and indexes.
The amount of network bandwidth that you use.
Unfortunately when using the amazing Firebase Realtime Database (ie, traditional Firebase), and the Cloud Functions thereof
There's no concept really of lockup available, other than the base transaction concept. (Which is awesome as far as it goes.) For example you can't do a say read, delete, insert.
We haven't user the new Firestore in a project yet; I'm wondering if it solves that particular problem?
This would make it tremendously useful for things like, well almost anything really, transactional game currencies, logic, etc.
Is this an advantage of Firestore?
Transactions in Firestore are more flexible than those in Realtime Database. With Realtime Database transactions, you had to choose a single location in that transaction, and you could only modify children under that location. All clients has to be using transactions to safely modify that transaction.
With Firestore transactions, you can transact using any arbitrary set of documents across any set of collections in your database, and you have atomicity on changes made to those documents. You're not obliged to choose just one collection or just one document.
There is no such thing as a "lock" in either product. Locks are not provided because they're difficult to manage correctly (avoiding deadlock) while also being scalable to millions of concurrent writers.
I would like to use Firebase Firestore for my next project but I need some help understanding the limitations of what is possible with Firestore queries.
Basically, I would like to perform queries over collections with large amounts of documents but upon doing research I have come across conflicting information.
This video (from the Firebase team): https://youtu.be/W3xIOQu0h1w?t=11m50s states that you can perform a query over a collection with "billions" of documents and maintain the same level of performance compared to a query over a collection with a few documents.
Then, I came across this github issue where the poster states that queries are taking too long and demand alot from the system. A member of the Firebase team answers by stating that performing a query over a collection containing 35k documents is beyond the "performance envelope".
So can someone point me in the right direction concerning Firestore queries and its limitations.
Please let me know if I was not clear in any part of this post.
That GitHub issue you linked to is specifically talking about offline searches. This means the Firestore backend service is not available, so the search is performed on the client. This is a very different situation than when the client is online and the service can perform the query in a massively scalable way. (Client apps are never massively scalable.)
Google just released Cloud Firestore, their new Document Database for apps.
I have been reading the documentation but I don't see a lot of differences between Firestore and Firebase DB.
The main point is that Firestore uses documents and collections which allow the easy use of querying compared to Firebase, which is a traditional noSQL database with a JSON base.
I would like to know a bit more about their differences, or usages, or whether Firestore just came to replace Firebase DB?
I wrote an entire blog post all about this very question, and I recommend you check it out (or the official documentation) for a more complete answer.
But if you want the quick(-ish) summary, here it is:
Better querying and more structured data -- While the Realtime Database is just a giant JSON tree, Cloud Firestore is a little more structured. All your data consists of documents (which are basically key-value stores) and collections (which are collections of documents). Documents will also frequently point to subcollections, which contain other documents, which themselves can contain other documents, and so on.
This structured data helps you out in two ways. First, all queries are shallow, meaning that you can request a document without grabbing all the data underneath. This means you can keep your data stored hierarchically in a way that makes more sense to you without having to worry about keeping your database shallow. Second, you have more powerful queries. For instance, you can now query across multiple fields without having to create those "combo" fields that combine (and denormalize) data from other parts of your database. In some cases, Cloud Firestore will just run those queries directly, and in other cases, it will automatically create and maintain indexes for you.
Designed to Scale -- Cloud Firestore will be able to scale better than the Realtime Database. It's important to note that your queries scale to the size of your result set, not your data set. So searching will remain fast no matter how large your data set might become.
Easier manual fetching of data -- Like the Realtime Database, you can set up listeners in Cloud Firestore to stream in changes in real-time. But if you don't want that kind of behavior, and just want a simple "fetch my data" call, Cloud Firestore has that as well, and it's built in as a primary use case. (They're much better than the once calls in Realtime Database-land)
Multi region support -- This basically means more reliability, as your data is shared across multiple data centers at once. But you still have strong consistency, meaning you can always make a query and be assured that you're getting the latest version of your data.
Different pricing model -- While the Realtime Database primarily charges based on storage or network bandwidth, Cloud Firestore primarily charges based on the number of operations you perform. Will this be better, or worse? It depends on your app.
For powering a news app, turn-based multiplayer game, or something like your own version of Stack Overflow, Cloud Firestore will probably look pretty favorable from a pricing standpoint. For something like a real-time group drawing app where you're sending across multiple updates a second to multiple people, it probably will be more expensive than the Realtime Database.
Why you still might want the to use the Realtime Database -- It comes down to a few reasons.
That whole "it'll probably be cheaper for apps that make lots of frequent updates" thing I mentioned previously,
It's been around for a long time and has been battle tested by thousands of apps,
It's got better latency and when you need something with reliably low latency for a real-timey feel, the Realtime Database might work better.
For most new apps, we recommend you check out Cloud Firestore. But if you have an app that's already on the Realtime Database, I don't really recommend switching just for the sake of switching, unless you have a compelling reason to do so.
Reasons to choose Cloud Firestore over Realtime Database
It is an improved version
Firebase database was enough for basic applications. But it was not powerful enough to handle complex requirements. That is why Cloud Firestore is introduced. Here are some major changes.
The basic file structure is improved.
Offline support for the web client.
Supports more advanced querying.
Write and transaction operations are atomic.
Reliability and performance improvements
Scaling will be automatic.
Will be more secure.
Pricing
In Cloud Firestore, rates have lowered even though it charges primarily on operations performed in your database along with bandwidth and storage. You can set a daily spending limit too. Here is the complete details about billing.
Future plans of Google
When they discovered the flaws with Real-time Database, they created another product rather than improving the old one. Even though there are no reliable details revealing their current standings on Real-time Database, it is the time to start thinking that it is likely to be abandoned.
Suggest link from google as well :
Firebase Real-time Database vs FireStore
Extracted from google docs, a small sumamry here:
FireBase Real Time DB is JSON based NO SQL DB, meant for mobile apps, regional, and used typically to store and sync data between users/devices in realtime / extremely low latency.
FireStore is JSON 'like' NOSQL DB meant for high concurrency, global, easily auto scaling persistence, designed for any clients (not only mobile apps) with typical use cases such as asset tracking, real time analytics, building retail product catalogs, social user profile, gaming leaderboards, chat based applications etc.
Cloud Firestore is Firebase's database for mobile app
development. It builds on the successes of the Realtime Database with
a new, more intuitive data model. Cloud Firestore also features
richer, faster queries and scales further than the Realtime Database.
Realtime Database is Firebase's original database. It's an efficient,
low-latency solution for mobile apps that require synced states
across clients in realtime.
To choose between Firebase Realtime database and Cloud firestore based on your application requirements, read official documentation here.