How to filter records by dateTime using firebase realtime database - firebase

In my firebase Realtime Database, every document saved has a datetime variable.
Example, dateTime:"2021-04-11T17:32:50.833523"
I have the following URL which fetches all documents. However, I now want to fetch only today's documents. I know that there are two options to achieving this. First one is to fetch all the documents from DB and then filter only today's records. Second option is to tweak the URL and put the filter while fetching records from the DB.
First I would like to know which approach yields faster results?
If it's the second approach, then I would like to know how to achieve it.
Here's the function I am using:
Future<void> fetchAllOrders() async {
final url ='https://fhgi-92211.firebaseio.com/orders.json?auth=$authToken';
}

According to the docs, something like this should work:
'https://fhgi-92211.firebaseio.com/orders.json?auth=$authToken&orderBy="dateTime"&startAt="2021-04-11"'
https://firebase.google.com/docs/database/rest/retrieve-data

Best solution - is n°2 : using & tweaking your URL - to do so I suggest :
First, tweaking your model by adding your date data in a simpler 2021-04-11 kinda of format
It will allow you much easier filtering
You can do so like this before writing your data to your RealTime Database :
import 'package:intl/intl.dart';
DateTime today = DateTime.now();
String simpleFormatToday = DateFormat('yyyy-MM-dd').format(today);
// returns a simple 2021-04-11 format of today's date
If your really need for some reason the full dateTime value you can write both dateTime:"2021-04-11T17:32:50.833523" & simpleDate:"$simpleFormatToday" to your object
Next,
To retrieve your data - use your function & tweak your URL to use Range Querying to retrieve only documents having simpleDate value equal to today - like so :
Future<void> fetchAllOrders() async {
final url ='https://fhgi-92211.firebaseio.com/orders.json?auth=$authToken&orderBy="simpleDate"&equalTo="$simpleFormatToday"';
}

First Approach
It will be useful if you want to get all the data and need to filter the dates several times from user input.
For example: If you have 100 records that you fetched using first approach i.e all you have is 100 records in your db and now user will make some filter on fetched records like getting 20-30 then 40-79, something like that. In this case making query using second approach wont be appropriate because of two reasons
Querying everytime on firebase would count read/write towards billing.
Managing data locally is more fast and reliable rather than making request to network frequently.
Second Approach
It will be useful in areas where you have to show specific time interval records in the one go and that's it, after fetching the records you dont need more of them to pull out from DB.
The Query will mainly revolve around what you want
equalsTo : It will give you only those records which will have exact date that you have passed as params.
orderBy : It will only work with equals to and will give you dates on bases of applied sorted filter (ex: by createdAt)
Future<void> fetchAllOrders() async {
//It will get today's date
var todaysDate = DateTime.now();
//From today's 12AM
var filterStartDate = DateTime(todaysDate.year,todaysDate.month,todaysDate.day,0,0,0);
//To second days' 11:59PM
var filterEndDate = DateTime(todaysDate.year,todaysDate.month,todaysDate.day,23,59,59);
final url ='https://fhgi-92211.firebaseio.com/orders.json?auth=$authToken&startAt="${filterStartDate.toIso8601String()}&endAt="${filterEndDate.toIso8601String()}';
}

Related

Using a date as Firestore Document Id?

I have a diary App where I want to have a single document for each day. My question is if it would be a good practise to use an ISO 8601 datetime String as a document Id. Because every day can only have one document it would be an unique Id.
Or should I store the date in the document and use a random document Id? What would be a better choice?
One querie I want to do is check if there is an document with a specific date (for example today). If so, I want to fetch this document, otherwhise If there is no document for today I want to create one.
Using the value that you want to be unique as the document ID is an easy (and in many cases the only) way to ensure uniqueness. As long as the limit on throughput for such sequential keys is acceptable for your use-case (which seems to be the case here), it is a good, simple method of ensuring uniqueness.
It seems good, however firebase does not recommend this. This is because using incremental document id names is not recommended. This could be Customer1, Customer2, ... CustomerN, or in your case dates. 10-31-21, 11-1-21, 11-2-21, ... is incremental.
Also, note their second suggestion: Avoid using / forward slashes in document IDs.
Or should I store the date in the document and use a random document Id? What would be a better choice?
I would chose the latter. Use a random doc id and then store the date inside.
With this being said, best practices and exactly that, best practices. They are not rules. In your case, if using the date as the document id makes it easier and requires less reads, it may be the best choice.
A solution for your use case might be to add a date field and query on that date, where you still would have a unique ID per document.
Query:
const analyticsCol = await x.ref
.collection('dishes')
.where('date', '>=', yesterdayDate)
.where('date', '<=', new Date()) // today
.limit(1)
.get();
const getYesterdayDate = () => {
const yesterdayDate = new Date();
yesterdayDate.setDate(yesterdayDate.getDate() - 1);
yesterdayDate.setTime(
yesterdayDate.getTime() -
yesterdayDate.getHours() * 3600 * 1000 -
yesterdayDate.getMinutes() * 60 * 1000
);
return yesterdayDate;
};
Hope it helps.

How to use composite query in firestore?

I have a question about firestore query.
I'm working react native project with firestore
I made index to retrieve data which has been voted in a week.
So, my code is like below.
db.getData
.where('updatedAt' '<=' now - week)
.orderBy('count', 'desc')
I know it's not allowed in firestore.
Any idea of this?
If I need to restructure DB, then please show me how it looks.
It depends on the type of updatedAt data. Is is a date object or a number or string timestamp? I prefer to store updates with plain Date.now() timestamps (as number type). So, if updatedAt is a number it's a simple comparison. Tip, ...just name it updated rather than updatedAt, so you have created and updated fields.

How to query one field then order by another one in Firebase cloud Firestore?

I'm struggling to make a (not so) complex query in firebase cloud firestore.
I simply need to get all docs where the id field == a given id.
Then order the results by their date and limit to 10 results.
So this is my current situation
db.firestore().collection('comments')
.where("postId",'==',idOfThePost)
.orderBy('date','asc').limit(10).get().then( snapshot => {
//Nothing happens and the request wasn't even been executed
})
I can get the result only if i don't use the orderBy query but i have to process this sorting for the needs of my application.
Someone has an idea to help me to fix this ?
thanks
You can do this by creating an Index in the firestore.
The first field of the index should be the equality field and the second field of the index should be the order by field.
Given your problem, you would need the following index:
first field: postId, 'asc'
second field: date, 'asc'
Please check the doc. It says
However, if you have a filter with a range comparison (<, <=, >, >=), your first ordering must be on the same field
you can try this code
db.firestore().collection('comments')
.where("postId",'==',idOfThePost)
.orderBy('postId')
.orderBy('date','asc').limit(10).get().then( snapshot => {
.....
})
My Workaround
If you're googling this you've probably realized it can't be done traditionally. Depending on your problem though there may be some viable workarounds, I just finished creating this one.
Scenario
We have an app that has posts that appear in a feed (kind of like Reddit), each post has an algorithmic score 'score' and we needed a way to get the 'top posts' from 12-24 hours ago. Trying to query sorted by 'score' where timestamp uses > and < to build the 12-24 hour ago range fails since Firebase doesn't allow multiple conditional querying or single conditional querying with an descending sort on another field.
Solution
What we ended up doing is using a second field that was an array since you can compound queries for array-contains and descending. At the time a post was made we knew the current hour, suppose it was hour 10000 since the server epoch (i.e. floor(serverTime/60.0/60.0)). We would create an array called arrayOfHoursWhenPostIsTwelveToTwentyFourHoursOld and in that array we would populate the following values:
int hourOffset = 12;
while (hourOffset <= 24) {
[arrayOfHoursWhenPostIsTwelveToTwentyFourHoursOld addObject:#(currentHour+hourOffset)];
hourOffset++;
}
Then, when making the post we would store that array under the field hoursWhenPostIsTwelveToTwentyFourHoursOld
THEN, if it had been, say, 13 hours since the post was made (the post was made at hour 10000) then the current hour would be 10013, so we could use the array-contains query to see if our array contained the value 10013 while also sorting by algorithm score at the same time
Like so:
FIRFirestore *firestore = [Server sharedFirestore];
FIRCollectionReference *collection = [firestore collectionWithPath:#"Posts"];
FIRQuery *query = [collection queryOrderedByField:#"postsAlgorithmScore" descending:YES];
query = [query queryWhereField:#"hoursWhenPostIsTwelveToTwentyFourHoursOld" arrayContains:#(currentHour)];
query = [query queryLimitedTo:numberToLoad];
Almost Done
The above code will not run properly at first since it is using a compound index query, so we had to create a compound index query in firebase, the easiest way to do this is just run the query then look at the error in the logs and firebase SDK will generate a link for you that you can navigate to and it will auto-generate the compound index for your database for you, otherwise you can navigate to firebase>database>index>compound>new and build it yourself using hoursWhenTwelveToTwentyFourHoursOld: Arrays, score: Descending
Enjoy!
same here, it is weird why can't. below is another sample. can't get the result. Hoping firebase can reply about this and update the document.
dbFireStore.collection('room').where('user_id.'+global.obj_user.user_id,'==',true).orderBy('last_update').get().then((qs)=>{
console.log(qs);
});
using other work-around solution is javascript array and array.sort()
I ran into the same issue yesterday on Android. The Callback was just not called. Today I suddenly got an error message. FAILED_PRECONDITION: The query requires an index. It even contains a URL in the error message to generate that index with one click.
It seems that if you want to use orderBy on your data, you need to create an index for that field. The index also needs to be in the correct order (DESC, ASC).
As per firestore document,
If you attempt a compound query with a range clause that doesn't map to an existing index, you receive an error. The error message includes a direct link to create the missing index in the Firebase console.
So just click that link you get in Logcat, it will be redirected to create index page, just create index. It will take some time. after enabling composite index, you will get the result as your requested query.
Stumbled across this looking for help when i found that using the orderBy function didnt work and the documentation still says it does not support it. A bit weird and unclear to be honest, because it does support it so long as you index your Firestore database. For example, this query now works fine for me having set up indexing:
const q = query(docRef, where("category", "==", 'Main'), orderBy('title', 'asc')
Indexing in Firestore
Console Log that even gives you the url to automatically create the index if you try and run with the above command.
Maybe I am missing something, or a later version of Firebase (I am using v9) simply does support it.

Firebase - Structuring Data For Efficient Indexing

I've read almost everywhere about structuring one's Firebase Database for efficient querying, but I am still a little confused between two alternatives that I have.
For example, let's say I want to get all of a user's "maxBenchPressSessions" from the past 7 days or so.
I'm stuck between picking between these two structures:
In the first array, I use the user's id as an attribute to index on whether true or false. In the second, I use userId as the attribute NAME whose value would be the user's id.
Is one faster than the other, or would they be indexed a relatively same manner? I kind of new to database design, so I want to make sure that I'm following correct practices.
PROGRESS
I have come up with a solution that will both flatten my database AND allow me to add a ListenerForSingleValueEvent using orderBy ONLY once, but only when I want to check if a user has a session saved for a specific day.
I can have each maxBenchPressSession object have a key in the format of userId_dateString. However, if I want to get all the user's sessions from the last 7 days, I don't know how to do it in one query.
Any ideas?
I recommend to watch the video. It is told about the structuring of the data very well.
References to the playlist on the firebase 3
Firebase 3.0: Data Modelling
Firebase 3.0: Node Client
As I understand the principle firebase to use it effectively. Should be as small as possible to query the data and it does not matter how many requests.
But you will approach such a request. We'll have to add another field to the database "negativeDate".
This field allows you to get the last seven entries. Here's a video -
https://www.youtube.com/watch?v=nMR_JPfL4qg&feature=youtu.be&t=4m36s
.limitToLast(7) - 7 entries
.orderByChild('negativeDate') - sort by date
Example of a request:
const ref = firebase.database().ref('maxBenchPressSession');
ref.orderByChild('negativeDate').limitToLast(7).on('value', function(snap){ })
Then add the user, and it puts all of its sessions.
const ref = firebase.database().ref('maxBenchPressSession/' + userId);
ref.orderByChild('negativeDate').limitToLast(7).on('value', function(snap){ })

How to load first 50 objects in firebase, stop the loading and then filter results?

My Firebase database is more than 800mb large and with more than 100.000 objects (news articles).
What I want to do is to fetch just the first 50 objects (most recent) and then to sort the objects got from the result of the first query according to child parameters.
So, for example, when the page is loaded, I need angularfire / firebase to load just first 50 objects and to stop loading the rest of objects in database. Then, I want to filter out just these 50 objects (articles) based on node category music.
So far, my first query seems to be fine (but if there is better way to ask firebase to load X objects and to stop, I would appreciate). But, the second part, I can’t figure it out because firebase throw an error.
The error is:
Query: Can't combine startAt(), endAt(), and limit(). Use limitToFirst() or limitToLast() instead
Here is my sample code:
var myArticlesRef = new Firebase(FIREBASE_URL + 'articles/');
var latestArticlesRef = myArticlesRef.limitToFirst(20); // is this the recommended way to ask firebase to stop
var latestArticlesOrder = latestArticlesRef.orderByChild('category').equalTo(‘Music’); // <- how to do something similar?
var latestArticlesInfo = $firebaseArray(latestArticlesOrder);
$scope.latestArticles = latestArticlesInfo;
console.log($scope.latestArticles);
This should work:
var query = myArticlesRef.orderByChild('category').equalTo(‘Music’).limitToFirst(20);
So you're asking Firebase to return the first 20 articles in the Music category.
While it is common to think of queries like this when coming from a relational/SQL mindset, I recommend that you consider this alternative data structure:
articles_by_category
music
article1: { }
article2: { }
article3: { }
...
technology
article4: { }
...
So instead of storing the articles in one big list, store them by category. That way to access the articles about music, you only have to do:
var query = ref.child('articles_by_category').child('music').limitToFirst(20);
With this approach the database doesn't have to execute any query and it can scale to a much higher number of users.
This is something you'll see regularly in a NoSQL world: you end up modeling your data for the way your application wants to query it. For a great introduction, see this article on NoSQL data modeling. Also read the Firebase documentation on data modeling and indexes.

Resources