I have a dataset of objects, e.g. cars. I want to make a system where users are presented an object, and can decide whether or not they like the car. But, I want to show them each car only once.
- allCars
- car1
- car2
...
- car348237
- carsLiked
- user1
- carsLiked
- car123
- car234
- carsNotLiked
- car321
- user2
- carsLiked
- carsNotLiked
Given some user, e.g. user1, how can I make a selection from allCars, WHITOUT cars that the user has already seen? In SQL I would do something like " WHERE carId NOT IN (car123, car234, car321) "
Any idea how I can do this in firebase (without filtering on clientside, I know how to do that)...? Any structure possible, using some kind of index.? I struggled for sometime but didn't find a solution.
Denormalization is a key.
I would replicate a set of all cars in each user's object and than I would delete the car object already displayed to user.
cars:{
CAR_AUDO_ID: {
//car object
}
users:{
user1:{
car_selection:{
CAR_AUTO_ID: true //reference to above
...
},
cars_liked:{
},
cars_disliked:{
}
}
coming from SQL it might sound like a lot of replication, but that the way to go with firebase.
in case you have something like 10K+ cars, of course the above would be overkill. If users are presented a random car than I would focus on random number generator where I would store only already picked numbers. In that case the best would be to use priority ordered list and your key would be something like increment counter.
Related
i have collection which contains the whole users products , and to avoid the casts of reading all docs to get the length of the collection , i increase one value +1 to every field once it's doc being created , ok now everything is ok
here i am using this method to chick if doc if exist or not so i can avoid the duplicate of counter in case offline mode
removeProduct1(){
FirebaseFirestore.instance.collection("product").doc('product1')
.get(const GetOptions(source: Source.server)).then((value1) {
if(value1.exists) {
FirebaseFirestore.instance.collection("product").doc('product1').delete();
FirebaseFirestore.instance.collection("product").doc('productCount').update({
'productCount': FieldValue.increment(-1),
});
}
}).onError((error, stackTrace) {
ScaffoldMessenger.of(context).showSnackBar(snackBar,);
});
}
also everything is ok , but i noticed if two user pressed the button for this method at the same time so its gonna be two value decrease -2
maybe someone will think its rarely when two user hit the button at the same time , well this is only simple sample of what i have for real instead , i have page which contains too many users which is not be rarely for being two user hit the button
EDIT FOR MOER EXPLATION
i have collection called Friends which contains User IDs of friends together
for example user1 his id is ABC and user2 his id is DEF
so id becomes final into the Friends collection like this ABCDEF by following function
and also this following function increase counter +1 for ever one of them
makeFriends(){
// here i make sure first if the friends Ids are not exists To avoid suspiciously increasing the counter in offline mode and its work very well
FirebaseFirestore.instance.collection("Friends").doc(user1Id+user2Id).get().then((value) {
if(!value.exists){
// here to add ids to the friends collection
FirebaseFirestore.instance.collection("Friends").doc(user1Id+user2Id).set({
'test':''
});
//here to increase counter in other colection which is users for both user
//user1
FirebaseFirestore.instance.collection("users").doc(user1Id).update({
'friendsCounter':FieldValue.increment(1)
});
//user2
FirebaseFirestore.instance.collection("users").doc(user2Id).update({
'friendsCounter':FieldValue.increment(1)
});
}
});
}
Okay now everything going fine and they became a friends with one value increased and their Ids became unit into Friends collection .. and here image for better explanation
image 1
image
here i have function responsible for deleting friend with decrease one value from FriendsCount Field for both of them
deleteFriends(){
// here i make sure first if the friends Ids are exists To
// avoid suspiciously decrease the counter in offline mode and its work very well
FirebaseFirestore.instance.collection("Friends").doc(user1Id+user2Id).get().then((value) {
if(value.exists){
// here to remove ids from the friends collection
FirebaseFirestore.instance.collection("Friends").doc(user1Id+user2Id).delete();
//here to decrease counter for both user
//user1
FirebaseFirestore.instance.collection("users").doc(user1Id).update({
'friendsCounter':FieldValue.increment(-1)
});
//user2
FirebaseFirestore.instance.collection("users").doc(user2Id).update({
'friendsCounter':FieldValue.increment(-1)
});
}
});
}
ok now also everything going fine .. lets say user1 deleted his friend user 2 , by the previous functionthe Friends id which is ABCDEFhas been deleted so next time if user 2 want to do the same opretion it will fail because Friends ids are not exist anymore ..
the problem IS :
lets say these two users are still friends and
these two user visited each others profiles and hit the button at the same time which is call the pervious function (deleteFriend) so they gonna get the same return value which is (yes exists) then will be double decrease value for both of them instead of one as required .
means if their friendsCount was 1 both of them when they was a friend so tee value will be -1 instead 0 because there two operation was proceed
and this happen only if these two user hit the button at the same moment , but if the user 1 hit the button before user 2 then user 2 want to do same operation so he ganna fail because friendId was deleted by user1 operation as a condition statement in previous function ..
if(value.exists) i thought it too fast to delete friendIds before it give same result to other user if they together call the function at the same time
of course i can prevent the button of calling the delete function in case one of these user is already deleted other but I didn't see this as a strong solution since i can't grantee traffic jam or any reason else like bad users behavior .. etc , also basically i can't do it because delete button must be available as long as they still a friends
Note: this also happen in addFriends buttom call too if two users hit it at the same time , double value +2 instead one ..
i need any way to avoid server to give same result if there was two user query same doc at the same time , or any good solution
If you need to both read and write a document to update it, you should use a transaction to prevent the race condition that you now have when multiple users perform the operation around the same time.
Here you definitely want to use a transaction, because if the delete fails for whatever reason, the decrement operation should also fail (and vice versa).
The FlutterFire documentation on transactions also has a good explanation and example code to get started.
im working on Smalltalk, i have "tweets" collection, each tweet have "user" and i want to count the tweets by user in a dictionary.
So i have to add a counter in each key of the dictionary, and have to add keys for the first time i get the user.
I was thinking in this code for the add, i want to know if is human redeable and efficient or is there a better way to do it
tweetCountUserFrom: tweet in: aDictionary
| user |
user := tweet user.
aDictionary at: user ifAbsentPut: 0.
aDictionary at: user put: (result at: user) + 1
Your code is clear enough (even though it likely includes a mistake as the result variable should have been aDictionary instead). Anyway, the intention is clear and does (or is expected to do) the same a well-known object does, which is Bag. So, let's see how the code would have been with an instance of Bag (it is always a good idea to become increasingly familiar with the Collection hierarchy of Smalltalk). Here is how you could arrive at the same result
tweetCountUserFrom: tweet in: aBag
aBag add: tweet user
That's it!
When it is time for you to see how many tweets a user has authored, you will only need to evaluate
aBag occurrencesOf: user
Internally, aBag will hold a Dictionary and will maintain it the way you intended in your code. However, by using an existing object you will keep your code simpler, shorter and easier to read and maintain. You will also decrease the probability of failure as Bags have been kicking around for more than 40 years now, so there is little chance for them to decieve you.
I have a problem doing complex queries in Firestore database. I have read the documentation countless times so I know where the limitations are, but I wonder whether there is a way to structure the data so it supports my use cases. Let me explain what the use cases first:
I have a list of jobs, and users and I want to able to list/filter jobs according to some criteria and to list/filter users according to some criteria.
Job
JOB ID
- job type (1 of predefined values)
- salary (any number value)
- location (any value)
- long
- lat
- rating (1 - 5)
- views (any number value)
- timeAdded (any timestamp value)
- etc.
User
User ID
- experiences (0, 1 or more of predefined values)
- experience1
- jobCategory
- jobName
- timeEmployed
- experience2
- etc
- languages (0, 1 or more of predefined values)
- language1
- languageName
- proficency
- language2
- etc.
- location (any value)
- long
- lat
- rating (1 - 5)
- views (any number value)
- timeLastActive (any timestamp value)
- etc.
Filtering by field which can only have one value is fine. Even when I add sorting by "timeAdded" or a ragne filter.
1) The problem is when I introduce second range filter, such as jobs with "salary" higher then 15 bucks and at the same time "rating" higher then 4. Is there a way to get around this problem when I have N range filters?
2) Second problem is that I cannot use logical OR. Lets say, filter jobs, where "jobCategory" is Bartender or Server.
3) Another problem is to filter by fields, which can have more then 1 value, e.g. User can speak more than one language. Then if I want to filter users which speak English, it is not possible. Not to mention to filter users who speak e.g. English OR French.
I know I can model the data the way that I use the language as the name of the field, like -english = true, but when I introduce range filter to this, I need to create a Firestore index, which is very inconvenient since I can have around 20 languages and around 50 job types at the same time, and I would have to create indexes all the combinations together with different range filters.. is this assumption correct?
4) How would I filter jobs which up to 20 km from certain position? Radius and the position is on the user to choose.
5) What if I want to filter by all those fields at the same time? E.g. filter certain "jobCategory", location and radius, "salary" higher then something and "rating" higher then something, and sort it all by "timeAdded".
Is this possible with Firestore / Realtime database, can I model the data in some way to support this, or do I have to look for an alternative DB solution? I really like the real-time aspect of it. It will come handy when it is time to implement chat feature to the app. Is it solvable with Cloud functions? I am trying to avoid doing multiple requests, merging them together and sending that to client, since there can be any combination of filters.
If not doable with Firebase, do you know of any alternatives similar to Firestore with better querying options? I really hope I am just missing something :)
Thank you!
I've moved from parse-server to firebase for my new project, but reached a point in the project where I beginning to think it was a bad idea.
Basically, I'm making an app where people can post information about concerts going on in their town.
My first challenge was to filter the events, so a user only get events in his/her own town. I did this by structure the data after cities:
{
concerts: {
"New york": {
...,
...
},
"Chicago": {
...,
...
}
}
}
Then I figure I need another filter for the type of concert, e.g rock, pop, etc. So I though I did another restructure. However, there probably need to be 5-10 more filters, and it will become very hard to structure the database in a good way.
I though about multiple query, but this wasn't allowed:
firebase.database().ref("concerts")
.orderByChild("type").equalTo("rock")
.orderByChild("length").equalTo("2")
.orderByChild("artist").equalTo("beatles")
I thought about fetching everything from the server, and then filter the result in the client. I see however two problems with this:
There might be a ton of unnecessarily data being downloaded.
Some concerts will be locked only to certain users (e.g users who have gone to at least 10 other concerts), and there might be a security aspect of pulling home these concerts to user not being allowed to see them.
I thought about combining filters to create query keys, like this this, but with over 10 filters, it will become to complex.
Is there a solution to this or should I forget about firebase for this use case?
Thanks in advance
Incredibly complex queries can be crafted in Firebase. The data needs to be stored in a structure that lends itself to being queried and most importantly, don't be afraid of duplicate data.
For example, lets assume we have an app that enables a user to select a concert for a particular year and month, a specific city, and in a particular genre.
There are 3 parameters
year_month
city
genre
The UI first queries the user to select a City
Austin
then the UI asks to select a year and month
201704
then a genre
Rock
Your Firebase structure looks like this
concerts
concert_00
city: Memphis
year_month: 201706
genre: Country
city_date_genre: Memphis_201606_Country
concert_01
city: Austin
year_month: 201704
genre: Rock
city_date_genre: Austin_201704_Rock
concert_02
city: Seattle
year_month: 201705
genre: Disco
city_date_genre: Seattle_201705_Disco
Your UI has already polled the user for the query info and with that, build a query string
Austin_201704_Rock
and then query the 'city_date_genre' node for that string and you have your data.
What if the user wanted to know all of the concerts in Austin for April 2017
queryStartingAt("Austin_201704").queryEndingAt("Austin_201704")
You could easily expand on this by adding another query node and changing the order of the data
concerts
concert_00
city: Memphis
year_month: 201706
genre: Country
city_date_genre: Memphis_201606_Country
city_genre_date: Memphis_Country_201606
And depending on which order the user selects their data, you could query the associated node.
Adding additional nodes is a tiny amount of data and allows for very open ended queries for the data you need.
I see this is an old post, but I'd like to take this opportunity to point others running into a similar Firebase issues to AceBase, which is is a free and open source alternative to the Firebase realtime database. The lack of proper querying and indexing options in Firebase was one of the reasons AceBase was built. Using AceBase would enable you to query your data like so:
const snapshots = await db.ref('concerts')
.query()
.filter('city', '==', 'New York')
.filter('date', 'between', [today, nextWeek]) // today & nextWeek being Dates
.filter('genre', 'in', ['rock', 'blues', 'country'])
.get();
Because AceBase supports indexing, adding 1 or more indexes to the the queried fields will make those queries run incredibly fast, even with millions of records. It supports simple indexes, but also FullText and Geo indexes, so you could also query your data with a location and keywords:
.filter('location', 'geo:nearby', { lat: 40.730610, long: -73.935242, radius: 10000 }) // New York center with 10km radius
.filter('title', 'fulltext:contains', '"John Mayer" OR "Kenny Wayne Shepherd"')
If you want to limit results to allow paging, simply add skip and limit: .skip(80).limit(20)
Additionally, if you'd want to make the query deliver realtime results so any newly added concert will immediately notify your app - simply adding event listeners will upgrade it to a realtime query:
const results = await db.ref('concerts')
.filter('location', 'geo:nearby', { lat: 40.730610, long: -73.935242, radius: 10000 })
.on('add', addConcert)
.on('remove', removeConcert)
.get();
function addConcert(match) {
results.push(match.snapshot);
updateUI();
}
function removeConcert(match) {
const index = results.findIndex(r => r.ref.path === match.ref.path);
results.splice(index, 1);
updateUI();
}
If you want to know more about AceBase, check it out at npm: https://www.npmjs.com/package/acebase. AceBase is free and its entire source code is available on GitHub. Have fun!
Is it possible to do wildcard queries on Firebase? For example:
https://foo.firebaseio.com/person.json?orderBy="name"&equalTo="Lun*"
I know it's been a while but I thought that others might be interested. You can "fake" a wildcard search for things like foo* (so basically you can search for values beginning with a specified string).
For iOS & Swift it would look like this:
dbReference.child("person").queryOrdered(byChild: "name").queryStarting(atValue: "foo").queryEnding(atValue: "foo\u{f8ff}").observe(.childAdded) { (snapshot: FIRDataSnapshot) in
print("\(snapshot.key) - \(String(describing: snapshot.value))")
}
What this does is using a start and end values for name property where the end key is equal to the start + a very high code point in the Unicode range. Because it is after most regular characters in Unicode, the query matches all values that start with foo.
No. But kinda.
You cannot do a wildcard query, however, you can structure your data that will allow for this.
For example, say we want to find matches for users whose name starts with Ler
Here's our structure
users
uid_0
name: "Leroy"
Store the decomposed data in another node: Remember, disk space is cheap.
decomposed
uid_0
L: true
Le: true
Ler: true
Lero: true
Leroy: true
then perform a query on the decomposed node for the value of true for children equal to Ler
ref.queryOrderedByChild("Ler").queryEqualToValue(true).observeEventType(.ChildAdded,
withBlock: { snapshot in
print(snapshot.key)
})
And the snapshot.key will be uid_0
You can do something like this.
Make sure you order your search field alphabetically.
Then you search for all names (starting at Lun and ending at Luo) where the last letter 'o' is calculated with the initial last letter 'n' + 1.
I guess you see the general idea here.
So it should return anything between 'Lun*' and stop at the first entry of 'Luo*'
https://foo.firebaseio.com/person.json?orderBy="name"&startAt="Lun"&endAt="Luo"
Since Firebase doesn't support wildcard searching I decided to go with Apigee BaaS for our company.