I have a server that needs to receive real time updates from Firebase, for multiple users, where each user grants Oauth access to his Firebase data to my app.
My server is implemented using Firebase REST Streaming, based on Server Sent Events.
I need to know if there is a way to multiplex Firebase data pertaining to multiple users on a single stream.
I would like to be able to set up the stream with Oauth tokens pertaining to multiple users, and to subsequently receive real time updates pertaining to the multiple users on the same stream.
Otherwise, it seems that I need to maintain a separate stream per Oauth token, which seems to be non-scalable.
I think Twitter have a Site Streams feature like what I am looking for in their API, implemented via an envelope that indicates the user the message is targetted to.
Does Firebase support anything similar?
A single Firebase REST call will only monitor a single node. E.g.
curl 'https://samplechat.firebaseio-demo.com/users/jack/name.json'
You can control what data is returned from under that node with the orderBy, startAt,endAtandlimitTo...` parameters. E.g.
curl 'https://samplechat.firebaseio-demo.com/users/.json?orderBy="name"&startAt="Jack"'
There is no way to have a single REST request return data from different nodes/nodesets. So unless you find a way to gather all data you want to return under single node, where it can be returned by a single set of query parameters (orderBy, etc), you will have to execute multiple REST requests to get your data.
Note that the SDKs that Firebase provides internally use a web-socket protocol, so are not impacted by this limitation. If an SDK is available for your server-side language (e.g. node.js, Java), you could solve it by using that one.
Related
Seems like majority of solutions here require the developer to manually save each user's push token in addition to Firebase saving these tokens as well, then iterate through these and send out notifications. This is not really desirable since I would be storing push tokens in two separate places and need to make sure their lifecycle management is synchronized, which is error prone 😢Is there a better way?
Some options I considered:
Give each user a person/unique topic, then map out a geolocation per topic; then filter through these locations, aggregate these topics into batches of 500 and send a push. This is probably the best option so far 🤷♂️ but seems like a silly use of topics API.
Bucket users in cities, with each city corresponding to a topic; then find all cities near a location, and send a push to those topics. This is doable, but lots of complex city mapping code without much flexibility (vs. considering a simple radius).
Manually store push token → user → geolocation; filter out the ones you want, then send a push to the token specifically. This is problematic for many reasons (multiple device management, push token lifecycle management, data duplication, etc.)
Ideally, there is a way to send out a push notification to a list of user IDs, without even touching push tokens, is there a way to do this?
The only way to associate a device token to a user ID is to store that mapping yourself. You should also assume that one user could have multiple devices. FCM doesn't have a concept of a "user". It only knows device tokens. Your app has to bring the concept of a user, then map that to tokens in code.
Note that any messaging solution that requires topics is not "secure". Anyone can effectively receive messages from any topic.
I've looked at a few places, Including this post and the firebase panel
Is there no way to use these api's to secure these endpoints using an api key you create per client who uses your cloud functions?
I'm able to block every one putting a restriction on the Browser key, but I would like to create a new api key, and use that as a way to authenticate my endpoint for various clients.
Creating a new api key, and using that as a parameter in my query doesn't work (don't now if I'm doing anything wrong)
Is there a way to do this?
Option 1: handle authentication within the function
https://github.com/firebase/functions-samples/tree/master/authorized-https-endpoint
Adapt above to use clients/keys stored in firestore
Option 2: Use an an API Gateway
Google Cloud Endpoints (no direct support for functions yet, need to implement a proxy)
Apigee (higher cost, perhaps more than you need)
Azure API Management (lower entry cost + easy to implement as a facade for services hosted outside Azure)
there are more..
The above gateways are probably best for your use case in that the first two would let you keep everything within Google, albeit with more complexity/cost -- hopefully Endpoints will get support for functions soon. Azure would mean having part of your architecture outside Google, but looks like an easy way to achieve what your after (api key per client for your google cloud / firebase functions)
Here's a good walkthrough of implementing Azure API Management:
https://koukia.ca/a-microservices-implementation-journey-part-4-9c19a16385e9
Not to achieve what you are after, as far as firebase and GCP is concerned your clients is your specific business problem.
One way you could tackle this (with the little information that is provided);
You need somewhere to store a list of clients + their API key (I would use firestore)
For the endpoints you want to secure with a client-specific API key you can include a check to confirm the header exists and also exists in your firestore client record.
Considerations:
Depending on your expected traffic loads and the the number of firestore reads you'll be adding, you might want to double check this kind of solution will work for your budget.
Is the API-key type solution the only option you must go for? You Could probably get pretty far using the https://github.com/firebase/firebaseui-web and doing user checks in your function with no extra DB read required. If you go down this path most of the user signup/ emails / account creation logic is ready to go.
https://firebase.google.com/docs/auth/web/password-auth#before_you_begin
Curious to see what some other firebase users suggest.
I'm trying to get started implementing Web Push in one of my apps. In the examples I have found, the client's endpoint URL is generally stored in memory with a comment saying something like:
In production you would store this in your database...
Since only registered users of my app can/will get push notifications, my plan was to store the endpoint URL in the user's meta data in my database. So far, so good.
The problem comes when I want to allow the same user to receive notifications on multiple devices. In theory, I will just add a new endpoint to the database for each device the user subscribes with. However, in testing I have noticed that endpoints change with each subscription/unsubscription on the same device. So, if a user subscribes/unsubscribes several times in a row on the same device, I wind up with several endpoints saved for that user (all but one of which are bad).
From what I have read, there is no reliable way to be notified when a user unsubscribes or an endpoint is otherwise invalidated. So, how can I tell if I should remove an old endpoint before adding a new one?
What's to stop a user from effectively mounting a denial of service attack by filling my db with endpoints through repeated subscription/unsubscription?
That's more meant as a joke (I can obvioulsy limit the total endpoints for a given user), but the problem I see is that when it comes time to send a notification, I will blast notification services with hundreds of notifications for invalid endpoints.
I want the subscribe logic on my server to be:
Check if we already have an endpoint saved for this user/device combo
If not add it, if yes, update it
The problem is that I can't figure out how to reliably do #1.
I will just add a new endpoint to the database for each device the user subscribes with
The best approach is to have a table like this:
endpoint | user_id
add an unique constraint (or a primary key) on the endpoint: you don't want to associate the same browser to multiple users, because it's a mess (if an endpoint is already present but it has a different user_id, just update the user_id associated to it)
user_id is a foreign key that points to your users table
if a user subscribes/unsubscribes several times in a row on the same device, I wind up with several endpoints saved for that user (all but one of which are bad).
Yes, unfortunately the push API has a wild unsubscription mechanism and you have to deal with it.
The endpoints can expire or can be invalid (or even malicious, like android.chromlum.info). You need to detect failures (using the HTTP status code, timeouts, etc.) when you try to send the push message from your application server. Then, for some kind of failures (permanent failures, like expiration) you need to delete the endpoint.
What's to stop a user from effectively mounting a denial of service attack by filling my db with endpoints through repeated subscription/unsubscription?
As I described above, you need to properly delete the invalid endpoints, once you realize that they are expired or invalid. Basically they will produce at most one invalid request. Moreover, if you have high throughput, it takes only a few seconds for your server to make requests for thousands of endpoints.
My suggestions are based on a lot of experiments and thinking done when I was developing Pushpad.
Another way is to have a keep alive field on you server and have your service worker update it whenever it receives a push notification. Then regularly purge endpoints which haven't been responded to recently.
I am about to start working on the back-end for a mobile app (initially iOS/Android, later also website) and I am thinking whether Realm could fulfill all my needs.
The basic idea is that there are two types of users - customers and service-providers. The customers send requests to the server once in a while and are subscribed (real-time) for any event that might occur in relation to this request in the future. Each service-provider is listening for specific requests from all customers and is the one who is going to trigger various events (send data) for each of those requests.
From the Realm docs, it is obvious that the real-time data sync is not going to be a problem. The thing I am concerned about is how to model the scenario (customer/service-provider) in the Realm 'world'. Based on what I read, it is preferred to have one realm per user. Therefore, I suppose the user will register and will be given a realm. Then whenever he makes a request, it is going to be stored in his realm. Now the question is how to model the service-provider. There are going to be various service-providers each responding (triggering various kinds of events up to one hour after request) to different kinds of requests. (Each user can send any request and therefore be served by any service-provider.)
I read a bit about that Realm supports data sharing among different realms which could be a partial solution for this problem, however I was not able to find if this 'sharing' could share only particular requests. (Meaning each service-provider will get only requests intended for him.)
My question is whether this scenario is doable using Realm?
This sounds like a perfect fit for Realm's server-side event-handling. Put simply, Realm offers the ability through our Node SDK to listen for changes across Realms on the server.
So in your example, where each mobile user would have their own Realm, the URL for this would be /~/myRealm in which the tilde represents the Realm user ID. The Node SDK event handling API allows you to register a JS function that will run in response to changes represented by a Regex pattern for Realm URLs. In this case you could use: ^/([0-9a-f]+)/myRealm so that any time any user's myRealm updated, the server could perform some logic.
In this manner, the server via the Node SDK is really a "super-user" or service-provider as you describe. When an event fires, the JS function that runs is provided the Realm that was updated and a list of indexes pertaining to the objects in the Realm that were inserted, deleted, or modified. You can then perform any logic in JS, such as using the changed data to call out to another API or opening the Realm in question or any other and writing changes which will get pushed back out to the respective clients.
The full server-side event handling is part of Realm Professional Edition, but we recently released another way to interact with this called Realm Functions. This provides the ability through the server's dashboard to create the same JS functions that will run in response to changes across Realms. The developer edition support 3 functions so you can try it out immediately!
The Firebase chat application seems to suggest that all clients will receive the messages sent to a given FireBase url.
Now, one way of ensuring that users only receive messages sent to a specific channel is to filter messages at the client, but this would mean that all messages will be propagated to all clients and the client would do the filtering.
Is there any way to establish channels at the Firebase server - or does this mean that one would need to create separate firebases for separate channels, which would mean that if one wanted user-specific push-messages, it might require creating one channel/firebase for each user.
So, what is the suggested solution?
The Firebase Data Structure makes this quite easy, actually! The demo app puts the chat messages in the root of the Firebase (i.e. https://samplechat.firebaseio.com/), but you could just as easily use separate locations within your Firebase for separate chats, e.g. /chat1, /chat2, etc. Or better yet, you could have a /chats/ location with an arbitrary number of chats underneath, each named uniquely (possibly using push()).
Then a user could receive and push messages to a particular chat by referencing it directly (e.g. https://samplechat.firebaseio.com/chats/chat-id/) and then they won't get any data for any other chats.