I have an application that was initially just using a local json server, but I started migrating to Firebase/Firestore recently. With my json server, I had an endpoint that retrieved just an object with the user's custom settings. Those settings will never need to be an array, it will always just be an object. How do I handle that in Firestore where every document is supposed to belong to a collection? Do I just create a collection that will always have 1 object inside? That seems kind of hacky, but I can't think of another solution and I am struggling to figure out how to search for the answer.
EDIT:
I have a lot more data than just this object. Most of my data are arrays that can get very long.
Just create a collection that has one document. You are absolutely obliged to put a document in a collection.
My opinion: if all you have is one object to store, Cloud Firestore is massive overkill for that problem. It's easy to put together, but you are going to pay for document reads that might be cheaper using something a little more straightforward than a massively scalable, cloud hosted, NoSQL, realtime database.
Related
I'm working on a calendar app and I'm currently fetching the data from firestore to populate the calendar. Eventually, there will be a lot of data to be fetched and I'm trying to understand the caching system of Firestore but can't get behind it.
What I ideally want to achieve:
Always use cached data and only update those documents, which are new, edited or deleted.
How would I achieve that?
If you want to force the SDK to read from the cache, you can specify source options when calling get().
If you find yourself doing this everywhere though, it might make sense to consider using another database than Firestore as that is primarily an online, cloud-hosted database that continues to work while you're temporary offline.
My users can create documents (let's say tasks) in a subcollection with a bunch of security rules checking for authentication, permissions and data validity. They can even select multiple tasks and copy them in the same collection.
Now, a regular user will likely create at most a hundred tasks at once, but what if someone with bad intentions manage to obtain my database credentials, authenticate and try to create a huge number of valid documents programmatically? This will result in Firestore scaling without problems and an unexpected surprise in my Firebase billing.
This is my first concern, but I'm also thinking about the possibility to limit a collection size for other reasons, and it would be at the same time a solution for the problem described.
I read about techniques to count documents in a collection described in the Firestore documentation, but I did not found a solution.
Keeping a counter on a doc field updated with a transaction in a cloud function would be inefficient in my case. Distributed counters increase the complexity of my data model a bit, and also I would not know how to properly read those counters in security rules for every task creation, and even if that would be an efficient solution.
Does anyone has suggestions?
I believe the way for a person to gain read/write access to your database would be to either to hack Google servers, in which case no one is safe and it doesn't really matter what you do, or to guess the exact name of your collections and documents.
As for the latter case, what I have done in my project is that for each collection and document I have used the name I wanted plus random 10-char Strings (including all kinds of chars and numbers. For example Users-x5NfaS1jCb) which kind of serve as independent, separate passwords every step of the way. This, at least, makes it difficult to guess the name of the collections and documents.
(Just like mentioned in the question) If using authentication does not cause any complications for you project, you can use it to further raise the security of your database by limiting access to users authenticating through your app only.
I guess (have never tried it) you can make use of Firebase Functions to limit the number of documents available in any given collection based on the criteria you want. This function will be invoked every time an event in created in the database.
If by "obtain my database credentials", you mean finding the username and password to your Firebase account, well it doesn't really matter what you do again. If they know what they are doing, they can take so many advantages that this particular issue will be the least of your problems.
All in all, if you ask me, your database is safe unless either someone guesses your collection and document names, or gains access to your Firebase account.
These are the only things I can think of for now. I'll try to update my answer later.
I need to get a user profile document, which then needs to access two other documents in separate collections, before it returns. At the moment I have implemented this client side but it takes a while. Should I/Can I run this using Cloud Functions, so that I just call one GET and retrieve everything in one go, rather than calling separate get functions sequentially from within my app?
The database retrieval from separate collections would take a similar amount of time whether it's done from the client or Cloud Function.
Collection queries should be very fast on your indexed fields, so probably your problem is the way you are handling asynchronicity. Are you waiting for the result from the first collection before starting the second query? You could dispatch both queries at the same time to cut your waiting time.
You can store all your documents in Firebase Storage and then concatenate the references from the files and download all the documents at the same time, plus you can access them quicker because you can store them into your SD card or internal storage.
Then, if the documents need to be rewritten there is not problem because if you download again from the storage it will auto replace them and the user will still have access to the documents. I tell you this because I'm doing something similar and it's working great!
Edit: As Sujil says, first make an authentication between the user and the database structure with Firebase, so only people logged in or authenticated in your app can read/write files.
I'm planning a web application that requires user auth, plus the ability to display data for the users that is stored in a database. No interaction between the users is needed (yet), however the users should be able create objects and query their "own" objects. For example I list 10 book names (10 book objects), and User A should be able to pick a book and create a new object, call it userNoteObject that contains the name of a choosen book and a short note (that he/she writes).
With a basic pseudo code one book object would look like this:
bookObj = {"id": 1, "name": "book name"}
And the user's note object would be something like this:
userNoteObject = {
"id": 1,
"book_name": "random book name",
"owner_userid": "a1b2c3d",
"note": "some random string"
}
With MySQL I would create three tables, one for the users and one for the userNoteObject-s and another for the bookObj-s. Everytime an user saves a note, I would add it to the table that lists the saved notes. Then I can simply query the notes that belongs to X user based on the user's owner_userid. It's a quite simple functionality.
After reading about the possibilities I've made a decision to go with Firebase Auth (because in the future I might need Android and iOS compatibility) + Google Cloud Datastore or Firebase Realtime Database. However I'm a little bit scared about the Realtime Database of Firebase since I've never worked any DB like it. I also like to be able to modify records manually with something like PhpMyAdmin and I assume Cloud Datastore has a visual interface like that.
I'm familiar with JSON handling and creating JSON files, however the JSON based database is strange for me at the moment. Therefore I'm thinking about that maybe the other option would be a better choice. It's very important that I don't need realtime db features. I would load X number of entries into the table that holds the bookObj-s and sometimes update them. I assume when the user creates an userNoteObject it would be saved quickly with both and after deleting an userNoteObject I could refresh the page close to realtime with Datastore. But the table that holds the book objects must be able to store millions of entries easily.
So the important things:
One db table should be able to handle millions of records easily
Easy as possible querying
Visual interface for the DB (if it's possible)
I don't need realtime features like dynamic game score display/saving
Other info:
I would like to use Angular.js
I'm familiar with Python if it can help in something
So my question is that which database would be better for my needs? At the moment I say Datastore, but I'm totally new with these services so I'm not really against the Realtime Database, but Datastore looks more suitable since it has a visual interface. However I'm also not sure that how would work Datastore with Firebase. If there is a third option like combining both, Realtime Database for the objects save by the user and the static objects for Datastore for example, I would love to hear about it too. My overall goal is to be able to write and query the db easy and fast as it's possible and easily use it with Firebase auth.
UPDATE: I just discovered Firebase's Cloud Firestore, so if it can be more useful I could use it.
If you are going to use Firebase I would recommend you use Cloud Firestore instead of either Cloud Datastore or Firebase realtime database. You get the benefits of a real-time database plus a true document based JSON data store. The one downside is that you don`t have a UI to interact with the data. Datastore has one but its not as robust as say PHPMyAdmin. And since these are NoSQL datastores SQL support is pretty limited.
If you really want a true relational back-end you could try Cloud SQL which is basically MySQL running on Google Servers.
For the Firestore console/UI, see https://firebase.google.com/docs/firestore/using-console. Is that the kind of thing you're looking for?
I am considering using Firebase for an application that should people to use full-text search over a collection of a few thousand objects. I like the idea of delivering a client-only application (not having to worry about hosting the data), but I am not sure how to handle search. The data will be static, so the indexing itself is not a big deal.
I assume I will need some additional service that runs queries and returns Firebase object handles. I can spin up such a service at some fixed location, but then I have to worry about its availability ad scalability. Although I don't expect too much traffic for this app, it can peak at a couple of thousand concurrent users.
Architectural thoughts?
Long-term, Firebase may have more advanced querying, so hopefully it'll support this sort of thing directly without you having to do anything special. Until then, you have a few options:
Write server code to handle the searching. The easiest way would be to run some server code responsible for the indexing/searching, as you mentioned. Firebase has a Node.JS client, so that would be an easy way to interface the service into Firebase. All of the data transfer could still happen through Firebase, but you would write a Node.JS service that watches for client "search requests" at some designated location in Firebase and then "responds" by writing the result set back into Firebase, for the client to consume.
Store the index in Firebase with clients automatically updating it. If you want to get really clever, you could try implementing a server-less scheme where clients automatically index their data as they write it... So the index for the full-text search would be stored in Firebase, and when a client writes a new item to the collection, it would be responsible for also updating the index appropriately. And to do a search, the client would directly consume the index to build the result set. This actually makes a lot of sense for simple cases where you want to index one field of a complex object stored in Firebase, but for full-text-search, this would probably be pretty gnarly. :-)
Store the index in Firebase with server code updating it. You could try a hybrid approach where the index is stored in Firebase and is used directly by clients to do searches, but rather than have clients update the index, you'd have server code that updates the index whenever new items are added to the collection. This way, clients could still search for data when your server is down. They just might get stale results until your server catches up on the indexing.
Until Firebase has more advanced querying, #1 is probably your best bet if you're willing to run a little server code. :-)
Google's current method to do full text search seems to be syncing with either Algolia or BigQuery with Cloud Functions for Firebase.
Here's Firebase's Algolia Full-text search integration example, and their BigQuery integration example that could be extended to support full search.