I am using Firebase to store users with their last scanned latitude and longitude.
An entry looks like this:
"Bdhwu37Jdmd28DmenHahd221" : {
"country_code" : "at",
"firstname" : "John",
"gender" : "m",
"lat" : 11.2549387,
"lon" : 17.3419559
}
Whenever a user presses a specific "search" button, I want my Firebase function to fetch the people nearest to the person who sent the request.
Since Firebase only allows for querying after one field, I decided to add the country_code, to kind of have some range-restrictions and query for that field. But it is still super slow when I load every user of a specific country and then check for the smallest distance between a given user and all the other users in the same country.
Already with 5 users, the function takes like 40 seconds to achieve the results.
I have also read about compound Indexes, but I would need to somehow combine the latitude and the longitude and query for both fields.
Is there any way to either get a second and third query involved here (e.g. search for the same country_code, and then for a similar longitude and latitude) or do I have to solve this inside my server code ?
The Firebase Database can only query by a single property. So the way to filter on latitude and longitude values is to combine them into a single property. That combined property must retain the filtering traits you want for numeric values, such as the ability to filter for a range.
While this at first may seem impossible, it actually has been done in the form of Geohashes. A few of its traits:
It is a hierarchical spatial data structure which subdivides space into buckets of grid shape
So: Geohashes divide space into a grid of buckets, each bucket identified by a string.
Geohashes offer properties like arbitrary precision and the possibility of gradually removing characters from the end of the code to reduce its size (and gradually lose precision).
The longer the string, the larger the area that the bucket covers
As a consequence of the gradual precision degradation, nearby places will often (but not always) present similar prefixes. The longer a shared prefix is, the closer the two places are.
Strings starting with the same characters are close to each other.
Combining these traits and you can see why these Geohashes are so appealing for use with the Firebase Database: they combine the latitude and longitude of a location into a single string, where strings that are lexicographically close to each other point to locations that are physically close to each other. Magic!
Firebase provides a library called Geofire, which uses Geohashes to implement a Geolocation system on top of its Realtime Database. The library is available for JavaScript, Java and Objective-C/Swift.
To learn more about Geofire, check out:
this blog post introducing Geofire 2
the demo app that used to show local busses moving on a map . The app doesn't work anymore (the data isn't being updated), but the code is still available.
this video and documentation on how to implement geoqueries on Cloud Firestore.
Related
Firebase RTDB costs are, in contrast to Firestore, calculated by download size (and not by operation count).
When you get data, like in this pseudo code database.child(path1).child(path2).getData(), then it also gets the keys for every value (key-value pairs). And also each single path, including path1 and path2. The keys in the key-value pairs are actually also just paths.
Are these paths taken into the calculation of the download size for the pricing?
I'm aiming at this:
Suppose you load 1 million key-value pairs of this kind: Every value is a boolean. But each key is a 200-character long string.
Would it be cheaper if the keys would be 10-character long strings instead?
Since the path names are sent from the server to the client, there is a cost associated with having longer path names. And I've definitely seen projects where they went with single-character key names to reduce that cost.
Whether it is worth choosing shorter path names purely for the cost savings, is something only you can determine for yourself.
Also consider if you app really should be reading a million key value pairs. Are you really going to show all of those to the user? In general: store the data close to what show on the screen to the user, and then only load data that you're showing in the current screen.
I'm developing a Location Tracking App with Android Studio and need to store coordinates in a normalized manner (One Variable in One Field). Can I modify the location data created by GeoFire in Firebase? I would like to have a single field called Latitude and another one called Longitude. In the picture above you can see that the coordinates are both stored below one field called ' l ', I would like to modify this, or copy them as separate childs of the current userID in the node Location. Would greatly appreciate any help, thanks!
There's nothing built into Geofire for you control the property names. But since Geofire is open-source, you can modify it to use the property names you want.
Just be aware that Geofire uses these shorter names to limit the bandwidth it uses, so your users see a bandwidth increase.
I'm building an app that tracks the user's location and updates Firebase. I've read the documentation about structure data but still have a few questions.
I'm considering structuring the data in one of two ways, but can't determine which one.
users
$id
-position
-other attr
vs:
user_position
$id
users
$id
-other attr.
In what scenario would the first design work best, second?
If you only keep one position per user (as seems to be the case by the fact that you use singular user_position), there is no useful difference between the two structures. A user's position in that case is just another attribute, just one that happens to have two value (lat and lon).
But if you want to keep multiple positions per user, then your first structure is mixing entity types: users and user_positions. This is an anti-pattern when it comes to Firebase Database.
The two most common reasons are:
Say you want to show a list of user names (or any specific, single-value attribute). With the first structure you will also need to read the list of all positions of all users, just to get the list of names. With the second structure, you just read the user's attributes. If that is still much more data than you need, consider also keeping a list of /user_names for optimal read performance.
Many developers end up wanting different access rules for the user positions and the other user attributes. In the first structure that is only possible by pushing the read permission from the top /users down to lower in the tree. In the second structure, you can just give separate permissions to /users and /user_positions.
I am currently in the evaluation process for a database that should serve as a backend for a mobile application.
Right now I am looking at Firebase, and for now I like it really much.
It is a requirement to have the possibility to fetch the
most popular items
at a certain location
(possibly in the future: additionally for a certain time range that would be an attribute of the item)
from the database.
So naturally I stumbled upon GeoFire that provides location based query possibilities for Firebase.
Unfortunately - at least as far as I understood - there is no possibility to order the results by an attribute other than the distance. (correct me if I am wrong).
So what do I do if I am not interested in the distance (I only want to have items in a certain radius, no matter how far from the center) but in the popularity factor (e.g. for the sake of simplicity a simple number that symbolizes popularity)?
IMPORTANT:
Filtering/Sorting on the client-side is not an option (or at least the least preferred one), as the result set could potentially grow to an infinite amount.
First version of the application will be for android, so the Firebase Java Client Library would be used in the first step.
Are there possibilities to solve this or is Firebase out of the race and not the right candidate for the job?
There is no way to add an extra condition to the server-side query of Geofire.
The query model of the Firebase database allows filtering only on a single property. Geofire already performs a seemingly impossible feat of filtering on both latitude and longitude. It does this through the magic of Geohashes, which combine latitude and longitude into a single string.
Strictly speaking you could find a way to extend the magic of Geohashes to add a third number into the mix. But while possible, I doubt it's feasible for most of us.
I'm trying to figure out how to query with filter with Geofire.
Suppose I have restaurants with different category. and I want to add that category to my query. How do I go about this?
One way I have now is querying the key with Geofire, run the for loop through each key and get the restaurant, and insert the appropriate restaurant to the array.
These seems so inefficient. Is there any other way to go about this?
Ideally I will have the filtered results, and only load each item when they're about to be shown.
Cheers!
Firebase queries can only filter by one condition. Geofire already does quite some "magic" to allow it to filter on both longitude and latitude. Adding another property to that equation might be possible, but is well beyond what Geofire handles by default. See GeoFire: How to add extra conditions within the query?
If you only ever want to access one category at a time, you can put the restaurants in a top-level node per category and point Geofire to one category.
/category1
item1
g: "pns0h0mf2u"
l: [-53.435719, 140.808716]
item2
g: "u417k3dwub"
l: [56.83069, 1.94822]
/category2
item3
g: "8m3rz3s480"
l: [30.902225, -166.66809]
/items
item1: ...
item2: ...
item3: ...
In the above example, we have two categories: category1 with 2 items and category2 with just 1 item. For each item, we see the data that Geofire uses: a geohash and the longitude and latitude. We also keep a single list with the other properties of these 3 items.
But more commonly, you simply do the extra filtering in client-side code. If you're worried about the performance of that: measure it, share the code, JSON data and measurements.
This is an old question, but I've seen it in a few places on the web, so I thought I might share one trick I've used.
The Problem
If you have a large collection in your database, maybe containing hundreds of thousands of keys, for example, it might not be feasible to grab them all. If you're trying to filter results based on location in addition to other criteria, you're stuck with something like:
Execute the location query
Loop through each returned geofire key and grab the corresponding data in the database
Check each returned piece of data to see if it matches the other criteria
Unfortunately, that's a lot of network requests, which is quite slow.
More concretely, let's say we want to get all users within e.g. 100 miles of a particular location that are male and between ages 20 and 25. If there are 10,000 users within 100 miles, that means 10,000 network requests to grab the user data and compare their gender and age.
The Workaround:
You can store the data you need for your comparisons in the geofire key itself, separated by a delimiter. Then, you can just split the keys returned by the geofire query to get access to the data. You still have to filter through them, but it's much faster than sending hundreds or thousands of requests.
For instance, you could use the format:
UserID*gender*age, which might look something like facebook:1234567*male*24. The important points are
Separate data points by a delimiter
Use a valid character for the delimiter -- "It can include any unicode characters except for . $ # [ ] / and ASCII control characters 0-31 and 127.)"
Use a character that is not going to be found elsewhere in your database - I used *, but that might not work for you. Do not use any characters from -0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ_abcdefghijklmnopqrstuvwxyz, since those are fair-game for keys generated by firebase's push()
Choose a consistent order for the data - in this case, UserID first, then gender, then age.
You can store up to 768 bytes of data in firebase keys, which goes a long way.
Hope this helps!