Our Firebase server currently uses a one device per topic. However as users of the App are increasing the overhead of sending many topic sends is starting to be significant. What are our options for grouping sends, given that the targets dataset changes on every send.
We started to look at device groups, but these will be unsupported if we are forced to move to HTTP V1. We had considered just adding a group of users to a topic, but managing the lifespan of the topic could become an issue. We would have to create a new one on every send, working out at what point we could tear this subscription down with impacting any messages which have not been received may be problematic.
And suggestions welcome.
The usual approach for this is to either use topics, or implementing your own topic-like system. In the latter you'd store the relation between the Instance ID and your grouping logic in a database, and then use the batch-sending feature of FCM to deliver to up to 1000 devices at a time.
Related
When we are talking about individual private push notifications per user (rather than per device), there are couple of ways to send them via Firebase:
Collect registration tokens, associate them with the user on the trusted server and send notifications to all registered tokens.
Assign a topic per user, e.g. topics/user-id and let the app subscribe to it upon successful login / unsubscribe on logout.
However, Firebase documentation recommends to use approach no. 1, saying "For fast, secure delivery to single devices or small groups of devices, target messages to registration tokens, not topics."
One concern is the latency (referring to "fast") issue. Is it really that significant in practice comparing registration tokens vs topics-based messaging?
More serious concern is security.
What does Firebase documentation mean exactly by "... secure delivery..." as opposed to topics?
In our case one device will likely be used by multiple users logging in and out. That means registration tokens will have to be disassociated from the user in the backend when user logs out. If something goes wrong and registration token is not disassociated, device will continue getting previous person's notifications.
Is this the trade-off I have to accept or am I missing some other option?
There are a few questions in there. Let's see if I can cover them main ones.
If you need to deliver messages to multiple tokens, the two approaches you're considering are:
Subscribe the app installs/tokens to a specific topic, that you then deliver the message to.
Keep your own registry of tokens for the user, and do the fan-out of user-to-tokens in your own code.
The biggest difference is in where the fan-out of a user to their tokens happens. When you use topics it is done by Firebase on Google's servers, while in the second case you do it yourself. There is no guaranteed performance difference between these two, but in the latter case you have more control. So you spend more effort (writing your own code for something Firebase can do for you), and in turn gain more control (which may or may not translate into better performance)..
The second question is around the security of topics. The documentation contains that note because topics often have a much simpler structure than tokens. For example, if you have a topic-per-user, you will often use the UID as the topic ID. And since you may be sharing that UID in other places, it is possible that other users may know a user's UID. And since subscribing to a topic only requires that you know your own token and the topic ID, that means that any user can subscribe to another user's topic.
We have 1M+ devices registered. Is there a way to limit how quickly the messages get delivered? Obviously it's real hard to scale if 1M+ notifications at the exact same time cause a massive spike of traffic to your backend. Would be great if instead of all the messages getting delivered immediately to all devices, you could make it only send X messages per second.
The best way to control the delivery of those message is actually by calling FCM with the token IDs yourself, preferably with the batched delivery feature from the legacy API (look for the registration_ids parameter there). You can scale this up to as many calls to the API as you need to deliver your message to all devices.
Using topics is also possible, but you lose control of the delivery performance since the fan-out happens in a process you don't control.
Alternatively: consider sending a data message that contains a timestamp on when it should be displayed. That way you separate the delivery time from the display time, removing the critical path (but of course introducing other considerations).
I've read quite a few posts (including the firebase.com website) on Firebase connections. The website says that one connection is equivalent to approximately 1400 visiting users per month. And this makes sense to me given a scenario where the client makes a quick connection to the Firebase server, pulls down some data, and then closes the connection. However, if I'm using angular bindings (via angularfire), wouldn't each client visit (in the event the user stays on the site for a period of time) be a connection? In this example having 100 users (each of which is making use of firebase angular bindings) connecting to the site at the same time would be 100 connections. If I opted not to use angular bindings, that number could be (in a theoretical sense) 0 if all the clients already made their requests for data and were just idling.
Do I understand this properly?
AngularFire is built on top of Firebase's regular JavaScript/Web SDK. The connection count is fundamentally the same between them: if a 100 users are using your application at the same time and you are synchronizing data for each of them, you will have 100 concurrent connections at that time.
The statement that one concurrent connection is the equivalent of about 1400 visits per month is based on the extensive experience that the Firebase people have with how long the average connection lasts. As Andrew Lee stated in this answer: most developers vastly over-estimate the number of concurrent connections they will have.
As said: AngularFire fundamentally behaves the same as Firebase's JavaScript API (because it is built on top of that). Both libraries keep an open connection for a user, so that they can synchronize any changes that occur between the connected users. You can manually drop such a connection by calling goOffLine and then re-instate it with goOnline. Whether that is a good approach is largely dependent on the type of application you're building.
Two examples:
There recently was someone who was building a word game. He used Firebase to store the final score for each game. In his case explicitly managing the connections makes sense, because the connection is only needed for a relatively short time when compared to the time the application is active.
The "hello world" for Firebase programming is a chat application. In such an application it doesn't make a lot of sense to manage the connections yourself. So briefly connect every 15 seconds and then disconnect again. If you do this, you're essentially reverting to polling for updates. Doing so will lose you one of the bigger benefits of using Firebase: it automatically synchronizes data to connected clients.
So only you can decide whether explicit connection management is best for you application. I'd recommend starting without it (it's simpler) and first testing your application on a smaller scale to see how actual usage holds up to your expectation.
The Firebase chat application seems to suggest that all clients will receive the messages sent to a given FireBase url.
Now, one way of ensuring that users only receive messages sent to a specific channel is to filter messages at the client, but this would mean that all messages will be propagated to all clients and the client would do the filtering.
Is there any way to establish channels at the Firebase server - or does this mean that one would need to create separate firebases for separate channels, which would mean that if one wanted user-specific push-messages, it might require creating one channel/firebase for each user.
So, what is the suggested solution?
The Firebase Data Structure makes this quite easy, actually! The demo app puts the chat messages in the root of the Firebase (i.e. https://samplechat.firebaseio.com/), but you could just as easily use separate locations within your Firebase for separate chats, e.g. /chat1, /chat2, etc. Or better yet, you could have a /chats/ location with an arbitrary number of chats underneath, each named uniquely (possibly using push()).
Then a user could receive and push messages to a particular chat by referencing it directly (e.g. https://samplechat.firebaseio.com/chats/chat-id/) and then they won't get any data for any other chats.
Could you validate my approach for using Firebase to manage a user notification system?
Basically I want to have user specific channels as well as more general channels which hold notifications. These notifications would appear on an intranet if the user has not viewed them before.
The idea being a server side action will update Firebase endpoints using the REST API either for a specific user or broadcast to everyone. The specific user messages I can easily mark as read and therefore not show them again, its the general broadcast I am struggling slightly with.
I could add a flag(user ID) to the general broadcast to indicate its read but I am concerned about performance as the client would have to check historic broadcast messages for the existence of this flag. I could add a user id to create a new endpoint which should be quicker.
e.g. /notification/general/ - contains the message, this triggers the client which then checks to see if /users/USERID/MessageID exists if it doesnt display the message and create this end point.
Is there something I am missing or is that the best approach?
Are the messages always consumed in-order? If so you could have each client remember the ID of the last message it read in each public channel. You could then use "startAt" on the queue to limit it to only new messages.
If they're not consumed in order, then you'll need some way of storing data about which ones were read and which ones weren't. Perhaps you can have each message get sent out to everyone's personal queue, and then have each user remove read messages.
Since there are already undividual user messages, why not just deliver the broadcasts to everyone individually (think email) rather than trying to store a single copy and figure out who read it.
To reduce bulk, you could store the message content separately, and simply store the ids in a user's queue. Then when they are viewed, you flag them per-user without any additional complexity.
At 100k of users receiving 100 messages a day including broadcasts, with a standard firebase ID (around 20 characters), that comes out to 210,000,000 characters a year (i.e. nothing for a database, and probably still far less than the actual bulk of storing the message body), assuming they never expire and get deleted.