I am working on a personal project to recreate the news feed of Facebook. So what I am trying to do is to recreate the scenario where when the user goes to the news feed, the user gets posts of everyone he follows only. Is there any way to run a query like that using the Firebase real-time database using an of "followings".
I can successfully generate single users posts in the android studio app using snapshot and recycler view.
If you're asking whether you can get posts from multiple userUID values with a single query, that is not possible.
If you're asking whether you can pass a list of postUID values to retrieve, that is also not possible.
In both cases the solution is to execute a separate query/read operation for each of the values, and merge the results in your application code. This is not nearly as slow as you may think, since Firebase pipelines the requests over a single web socket connection - which is quite efficient. For more on this, see Speed up fetching posts for my social network app by using query instead of observing a single event repeatedly
Related
I need to get a user profile document, which then needs to access two other documents in separate collections, before it returns. At the moment I have implemented this client side but it takes a while. Should I/Can I run this using Cloud Functions, so that I just call one GET and retrieve everything in one go, rather than calling separate get functions sequentially from within my app?
The database retrieval from separate collections would take a similar amount of time whether it's done from the client or Cloud Function.
Collection queries should be very fast on your indexed fields, so probably your problem is the way you are handling asynchronicity. Are you waiting for the result from the first collection before starting the second query? You could dispatch both queries at the same time to cut your waiting time.
You can store all your documents in Firebase Storage and then concatenate the references from the files and download all the documents at the same time, plus you can access them quicker because you can store them into your SD card or internal storage.
Then, if the documents need to be rewritten there is not problem because if you download again from the storage it will auto replace them and the user will still have access to the documents. I tell you this because I'm doing something similar and it's working great!
Edit: As Sujil says, first make an authentication between the user and the database structure with Firebase, so only people logged in or authenticated in your app can read/write files.
Let's say I'm developing app like Instagram: for iOS, Android and Web. I decided to use Google Firebase as it really seems to simplify the work.
The features user needs in the app are:
Authorization/Registration
Uploading photos
Searching for other people, following them and see their photos
I come from traditional "own-backend" development where I do need to setup a server, create database and finally write the API to let the frontend retrieve the data from the server. That's the reason why it's unclear to me how it all works in Firebase.
So the question is how can I create such app:
Should I create my own API with cloud functions? Or it's ok to work with the database directly from the client-side?
If I work with the database directly why do I need cloud functions? Should I use them?
Sorry for such silly questions, but it is really hard to get from scratch.
The main difference between Firebase and the traditional setup you describe is that with Firebase, as far as the app developer is concerned, the client has direct access to the database, without the need for an intermediate custom API layer. Firebase provides SDKs in various languages that you would typically use to fetch the data you need / commit data updates.
You also have admin SDKs that you can use server-side, but these are meant for you to run some custom business logic - such as analytics, caching in an external service, for exemple - not for you to implement a data fetching API layer.
This has 2 important consequences:
You must define security rules to control who is allowed to read/write at what paths in your database. These security rules are defined at the project level, and rely on the authenticated user (using Firebase Authentication). Typically, if you store the user profile at the path users/$userId, you would define a rule saying that this node can be written to only if the authenticated user has an id of $userId.
You must structure your data in a way that makes it easily readable - without the need for complex database operations such as JOINs that are not supported by Firebase (you do have some limited querying options tough).
These 2 points allow you to skip the 2 main roles of traditional APIs: validating access and fetching/formatting the data.
Cloud functions allow you to react to data changes. Let's say everytime a new user is created, you want to send him a Welcome email: you could define a cloud function sending this email everytime a new node is appended to the users path. They allow you to run the code you would typically run server-side when writes happen, so they can have a very broad range of use-cases: side-effects (such as sending an email), caching data in an external service, caching data within Firebase for easier reads, analytics, etc..
You don't really need a server, you can access the database directly from the client, as long as your users are authenticated and you have defined reasonable security rules on Firebase.
In your use case you could, for example, use cloud functions to create a thumbnail when someone uploads a photo (Firebase Cloud Functions has ImageMagick included for that), or to denormalize your data so your application is faster, or to generate logs. So, basically you can use them whenever you need to do some server side processing when something changes on your database or storage. But I find cloud functions hard to develop and debug, and there are alternatives such as creating a Node application that subscribes to real time changes in your data and processes it. The downside is that you need to host it outside Firebase.
My answer is definitely NOT complete or professional, but here are the reasons why I choose Cloud Functions
Performance
You mentioned that you're writing an instagram-like mobile device app, then I assume that people can comment on others' pictures, as well as view those comments. How would you like to download comments from database and display them on users' devices? I mean, there could be hundreds, maybe thousands of comments on 1 post, you'll need to paginate your results. Why not let the server do all the hard work, free up users' devices and wait for the results? This doesn't seem like a lot better, but let's face it, if your app is incredibly successful, you'll have millions of users, millions of comments that you need to deal with, server will do those hard jobs way better than a mobile phone.
Security
If your project is small, then it's true that you won't worry about performance, but what about security? If you do everything on client side, you're basically allowing every device to connect to your database, meaning that every device can read from/write into your database. Once a malicious user have found out your database url, all he has to do is to
firebase.database().ref(...).remove();
With 1 line of code, you'll lose all your data. Okay, if you say, then I'll just come up with some good security rules like the one below:
This means that for each post, only the owner of that post can make any changes to it or read from it, other people are forbidden to do anything. It's good, but not realistic. People are supposed to be able to comment on the post, that's modifying the post, this rule will not apply to the situation. But again, if you let everybody read/write, it's not safe again. Then, why not just make .read and .write false, like this:
It's 100% safe, because nobody can do anything about anything in your database. Then, you write an API to do all the operations to your database. API limits the operations that can be done to your database. And you have experience in writing APIs, I'm sure you can do something to make your API strong in terms of security, for example, if a user wants to delete a post that he created, in your deletePost API, you're supposed to authenticate the user first. This way, 'nobody' can cause any damage to your database.
I'm planning a web application that requires user auth, plus the ability to display data for the users that is stored in a database. No interaction between the users is needed (yet), however the users should be able create objects and query their "own" objects. For example I list 10 book names (10 book objects), and User A should be able to pick a book and create a new object, call it userNoteObject that contains the name of a choosen book and a short note (that he/she writes).
With a basic pseudo code one book object would look like this:
bookObj = {"id": 1, "name": "book name"}
And the user's note object would be something like this:
userNoteObject = {
"id": 1,
"book_name": "random book name",
"owner_userid": "a1b2c3d",
"note": "some random string"
}
With MySQL I would create three tables, one for the users and one for the userNoteObject-s and another for the bookObj-s. Everytime an user saves a note, I would add it to the table that lists the saved notes. Then I can simply query the notes that belongs to X user based on the user's owner_userid. It's a quite simple functionality.
After reading about the possibilities I've made a decision to go with Firebase Auth (because in the future I might need Android and iOS compatibility) + Google Cloud Datastore or Firebase Realtime Database. However I'm a little bit scared about the Realtime Database of Firebase since I've never worked any DB like it. I also like to be able to modify records manually with something like PhpMyAdmin and I assume Cloud Datastore has a visual interface like that.
I'm familiar with JSON handling and creating JSON files, however the JSON based database is strange for me at the moment. Therefore I'm thinking about that maybe the other option would be a better choice. It's very important that I don't need realtime db features. I would load X number of entries into the table that holds the bookObj-s and sometimes update them. I assume when the user creates an userNoteObject it would be saved quickly with both and after deleting an userNoteObject I could refresh the page close to realtime with Datastore. But the table that holds the book objects must be able to store millions of entries easily.
So the important things:
One db table should be able to handle millions of records easily
Easy as possible querying
Visual interface for the DB (if it's possible)
I don't need realtime features like dynamic game score display/saving
Other info:
I would like to use Angular.js
I'm familiar with Python if it can help in something
So my question is that which database would be better for my needs? At the moment I say Datastore, but I'm totally new with these services so I'm not really against the Realtime Database, but Datastore looks more suitable since it has a visual interface. However I'm also not sure that how would work Datastore with Firebase. If there is a third option like combining both, Realtime Database for the objects save by the user and the static objects for Datastore for example, I would love to hear about it too. My overall goal is to be able to write and query the db easy and fast as it's possible and easily use it with Firebase auth.
UPDATE: I just discovered Firebase's Cloud Firestore, so if it can be more useful I could use it.
If you are going to use Firebase I would recommend you use Cloud Firestore instead of either Cloud Datastore or Firebase realtime database. You get the benefits of a real-time database plus a true document based JSON data store. The one downside is that you don`t have a UI to interact with the data. Datastore has one but its not as robust as say PHPMyAdmin. And since these are NoSQL datastores SQL support is pretty limited.
If you really want a true relational back-end you could try Cloud SQL which is basically MySQL running on Google Servers.
For the Firestore console/UI, see https://firebase.google.com/docs/firestore/using-console. Is that the kind of thing you're looking for?
I need to create a typical find friends feature in my mobile app that's using Firebase. The user would upload a list of hashed contact emails or phone numbers from their address book and the server would return a list of usernames that are already using the application. The typical user would have around 500 contacts in their address book.
This would be pretty straightforward to set up using a traditional SQL or Mongo database but in Firebase this would be difficult because I don't see any WHERE IN clauses with Firebase Query and it seems like it would be very inefficient using a Firebase Database for this. Even if I created a specific HashedPhoneNumbers collection with the hash being the id, it still seems like a monster query. Is there a way to make this query run efficiently in Firebase?
i.e. SELECT username from Users WHERE phoneHash IN [list of 500 phone hashes]
Alternatively, if I were to use Google Cloud DataStore looks like it supports chaining a bunch of AND email_hash = XXX filters together, but I don't know how efficient that would be if the filter list is 500 filters chained together.
Yes it does. For that i recomand you to see: The Firebase Database For SQL Developers and NoSQL Data Modeling Techniques.
It's it does not support this kind of query but there method with which you can achieve the same thing. Explained in the above tutorial.
Yes it does. For this i recomand you to use Cloud Storage For Firebase.
As a conclusion, i kindly recomand you using Firebase.
I am considering using Firebase for an application that should people to use full-text search over a collection of a few thousand objects. I like the idea of delivering a client-only application (not having to worry about hosting the data), but I am not sure how to handle search. The data will be static, so the indexing itself is not a big deal.
I assume I will need some additional service that runs queries and returns Firebase object handles. I can spin up such a service at some fixed location, but then I have to worry about its availability ad scalability. Although I don't expect too much traffic for this app, it can peak at a couple of thousand concurrent users.
Architectural thoughts?
Long-term, Firebase may have more advanced querying, so hopefully it'll support this sort of thing directly without you having to do anything special. Until then, you have a few options:
Write server code to handle the searching. The easiest way would be to run some server code responsible for the indexing/searching, as you mentioned. Firebase has a Node.JS client, so that would be an easy way to interface the service into Firebase. All of the data transfer could still happen through Firebase, but you would write a Node.JS service that watches for client "search requests" at some designated location in Firebase and then "responds" by writing the result set back into Firebase, for the client to consume.
Store the index in Firebase with clients automatically updating it. If you want to get really clever, you could try implementing a server-less scheme where clients automatically index their data as they write it... So the index for the full-text search would be stored in Firebase, and when a client writes a new item to the collection, it would be responsible for also updating the index appropriately. And to do a search, the client would directly consume the index to build the result set. This actually makes a lot of sense for simple cases where you want to index one field of a complex object stored in Firebase, but for full-text-search, this would probably be pretty gnarly. :-)
Store the index in Firebase with server code updating it. You could try a hybrid approach where the index is stored in Firebase and is used directly by clients to do searches, but rather than have clients update the index, you'd have server code that updates the index whenever new items are added to the collection. This way, clients could still search for data when your server is down. They just might get stale results until your server catches up on the indexing.
Until Firebase has more advanced querying, #1 is probably your best bet if you're willing to run a little server code. :-)
Google's current method to do full text search seems to be syncing with either Algolia or BigQuery with Cloud Functions for Firebase.
Here's Firebase's Algolia Full-text search integration example, and their BigQuery integration example that could be extended to support full search.