I have a diary App where I want to have a single document for each day. My question is if it would be a good practise to use an ISO 8601 datetime String as a document Id. Because every day can only have one document it would be an unique Id.
Or should I store the date in the document and use a random document Id? What would be a better choice?
One querie I want to do is check if there is an document with a specific date (for example today). If so, I want to fetch this document, otherwhise If there is no document for today I want to create one.
Using the value that you want to be unique as the document ID is an easy (and in many cases the only) way to ensure uniqueness. As long as the limit on throughput for such sequential keys is acceptable for your use-case (which seems to be the case here), it is a good, simple method of ensuring uniqueness.
It seems good, however firebase does not recommend this. This is because using incremental document id names is not recommended. This could be Customer1, Customer2, ... CustomerN, or in your case dates. 10-31-21, 11-1-21, 11-2-21, ... is incremental.
Also, note their second suggestion: Avoid using / forward slashes in document IDs.
Or should I store the date in the document and use a random document Id? What would be a better choice?
I would chose the latter. Use a random doc id and then store the date inside.
With this being said, best practices and exactly that, best practices. They are not rules. In your case, if using the date as the document id makes it easier and requires less reads, it may be the best choice.
A solution for your use case might be to add a date field and query on that date, where you still would have a unique ID per document.
Query:
const analyticsCol = await x.ref
.collection('dishes')
.where('date', '>=', yesterdayDate)
.where('date', '<=', new Date()) // today
.limit(1)
.get();
const getYesterdayDate = () => {
const yesterdayDate = new Date();
yesterdayDate.setDate(yesterdayDate.getDate() - 1);
yesterdayDate.setTime(
yesterdayDate.getTime() -
yesterdayDate.getHours() * 3600 * 1000 -
yesterdayDate.getMinutes() * 60 * 1000
);
return yesterdayDate;
};
Hope it helps.
Related
In my firebase Realtime Database, every document saved has a datetime variable.
Example, dateTime:"2021-04-11T17:32:50.833523"
I have the following URL which fetches all documents. However, I now want to fetch only today's documents. I know that there are two options to achieving this. First one is to fetch all the documents from DB and then filter only today's records. Second option is to tweak the URL and put the filter while fetching records from the DB.
First I would like to know which approach yields faster results?
If it's the second approach, then I would like to know how to achieve it.
Here's the function I am using:
Future<void> fetchAllOrders() async {
final url ='https://fhgi-92211.firebaseio.com/orders.json?auth=$authToken';
}
According to the docs, something like this should work:
'https://fhgi-92211.firebaseio.com/orders.json?auth=$authToken&orderBy="dateTime"&startAt="2021-04-11"'
https://firebase.google.com/docs/database/rest/retrieve-data
Best solution - is n°2 : using & tweaking your URL - to do so I suggest :
First, tweaking your model by adding your date data in a simpler 2021-04-11 kinda of format
It will allow you much easier filtering
You can do so like this before writing your data to your RealTime Database :
import 'package:intl/intl.dart';
DateTime today = DateTime.now();
String simpleFormatToday = DateFormat('yyyy-MM-dd').format(today);
// returns a simple 2021-04-11 format of today's date
If your really need for some reason the full dateTime value you can write both dateTime:"2021-04-11T17:32:50.833523" & simpleDate:"$simpleFormatToday" to your object
Next,
To retrieve your data - use your function & tweak your URL to use Range Querying to retrieve only documents having simpleDate value equal to today - like so :
Future<void> fetchAllOrders() async {
final url ='https://fhgi-92211.firebaseio.com/orders.json?auth=$authToken&orderBy="simpleDate"&equalTo="$simpleFormatToday"';
}
First Approach
It will be useful if you want to get all the data and need to filter the dates several times from user input.
For example: If you have 100 records that you fetched using first approach i.e all you have is 100 records in your db and now user will make some filter on fetched records like getting 20-30 then 40-79, something like that. In this case making query using second approach wont be appropriate because of two reasons
Querying everytime on firebase would count read/write towards billing.
Managing data locally is more fast and reliable rather than making request to network frequently.
Second Approach
It will be useful in areas where you have to show specific time interval records in the one go and that's it, after fetching the records you dont need more of them to pull out from DB.
The Query will mainly revolve around what you want
equalsTo : It will give you only those records which will have exact date that you have passed as params.
orderBy : It will only work with equals to and will give you dates on bases of applied sorted filter (ex: by createdAt)
Future<void> fetchAllOrders() async {
//It will get today's date
var todaysDate = DateTime.now();
//From today's 12AM
var filterStartDate = DateTime(todaysDate.year,todaysDate.month,todaysDate.day,0,0,0);
//To second days' 11:59PM
var filterEndDate = DateTime(todaysDate.year,todaysDate.month,todaysDate.day,23,59,59);
final url ='https://fhgi-92211.firebaseio.com/orders.json?auth=$authToken&startAt="${filterStartDate.toIso8601String()}&endAt="${filterEndDate.toIso8601String()}';
}
I'm creating a small game where people are able to join a room using a six-digit pin. Every room is represented by a document in a Firestore collection, where the room pin is the id of a room document.
My initial idea was to randomly generate a six-digit pin and check if a document with that id exists. If it does, create a new document with the generated pin, if not, generate a new pin and check if that id is free. This method will work, however, with a bit of bad luck it might cause a lot of unnecessary requests to the database.
My question is, therefore: is it possible to specify a format of the autogenerated id's? Or, if it is not possible, is there a way to fetch all documents id's to locally check whether or not the id exists?
You cannot specify a format for the auto-generated IDs but you can check if a room with same ID exists. If yes, then try new ID else create a room with same ID.
async function addRoom(roomId) {
const roomRef = admin.firestore().collection("rooms").doc(roomId)
if ((await roomRef.get()).exists) {
return addRoom(generatePin())
} else {
await roomRef.set({ ... })
}
return `${roomId} created`
}
function generatePin() {
return String(Math.floor(Math.random() * 999999) + 100000)
}
return addRoom(generatePin())
.then(() => console.log("Room created"))
.catch((e) => console.error(e))
PS: This might end up in some recursive state so I'd recommend using Firestore's IDs or uuid-int is you need numerical IDs only.
There is no way for you to specify the format of automatic IDs generated by the Firestore SDKs. They are purely random, and have an amount of entropy that statistically guarantees there won't be collisions (the chance where two clients generate the same ID is infinitesimally small), and minimizes the chance of hotspots (created when writing lots of documents or index values to the same area on a disk).
You can generate whatever auto-ID format you want however. You'll just have to accept a higher chance of collisions, as you already did, and the fact that you may experience hotspots when the documents are in the same area on a disk.
I have a collection where the documents are uniquely identified by a date, and I want to get the n most recent documents. My first thought was to use the date as a document ID, and then my query would sort by ID in descending order. Something like .orderBy(FieldPath.documentId, descending: true).limit(n). This does not work, because it requires an index, which can't be created because __name__ only indexes are not supported.
My next attempt was to use .limitToLast(n) with the default sort, which is documented here.
By default, Cloud Firestore retrieves all documents that satisfy the query in ascending order by document ID
According to that snippet from the docs, .limitToLast(n) should work. However, because I didn't specify a sort, it says I can't limit the results. To fix this, I tried .orderBy(FieldPath.documentId).limitToLast(n), which should be equivalent. This, for some reason, gives me an error saying I need an index. I can't create it for the same reason I couldn't create the previous one, but I don't think I should need to because they must already have an index like that in order to implement the default ordering.
Should I just give up and copy the document ID into the document as a field, so I can sort that way? I know it should be easy from an algorithms perspective to do what I'm trying to do, but I haven't been able to figure out how to do it using the API. Am I missing something?
Edit: I didn't realize this was important, but I'm using the flutterfire firestore library.
A few points. It is ALWAYS a good practice to use random, well distributed documentId's in firestore for scale and efficiency. Related to that, there is effectively NO WAY to query by documentId - and in the few circumstances you can use it (especially for a range, which is possible but VERY tricky, as it requires inequalities, and you can only do inequalities on one field). IF there's a reason to search on an ID, yes it is PERFECTLY appropriate to store in the document as well - in fact, my wrapper library always does this.
the correct notation, btw, would be FieldPath.documentId() (method, not constant) - alternatively, __name__ - but I believe this only works in Queries. The reason it requested a new index is without the () it assumed you had a field named FieldPath with a subfield named documentid.
Further: FieldPath.documentId() does NOT generate the documentId at the server - it generates the FULL PATH to the document - see Firestore collection group query on documentId for a more complete explanation.
So net:
=> documentId's should be as random as possible within a collection; it's generally best to let Firestore generate them for you.
=> a valid exception is when you have ONE AND ONLY ONE sub-document under another - for example, every "user" document might have one and only one "forms of Id" document as a subcollection. It is valid to use the SAME ID as the parent document in this exceptional case.
=> anything you want to query should be a FIELD in a document,and generally simple fields.
=> WORD TO THE WISE: Firestore "arrays" are ABSOLUTELY NOT ARRAYS. They are ORDERED LISTS, generally in the order they were added to the array. The SDK presents them to the CLIENT as arrays, but Firestore it self does not STORE them as ACTUAL ARRAYS - THE NUMBER YOU SEE IN THE CONSOLE is the order, not an index. matching elements in an array (arrayContains, e.g.) requires matching the WHOLE element - if you store an ordered list of objects, you CANNOT query the "array" on sub-elements.
From what I've found:
FieldPath.documentId does not match on the documentId, but on the refPath (which it gets automatically if passed a document reference).
As such, since the documents are to be sorted by timestamp, it would be more ideal to create a timestamp fieldvalue for createdAt rather than a human-readable string which is prone to string length sorting over the value of the string.
From there, you can simply sort by date and limit to last. You can keep the document ID's as you intend.
I want to query by date and status. I created an index for both the fields, but when querying it throws an error:
Cannot have inequality filters on multiple properties: [created_at, status]
const docQuery = db.collection('documents')
.where('created_at', '<=', new Date()) // ex. created_at later than 2 weeks ago
.where('status', '!=', 'processed') // works when == is used
.limit(10);
'status' is a string and can be multiple things.
I read about query limitations, https://firebase.google.com/docs/firestore/query-data/queries but that is such a simple thing for any database to do, what is a good solution for such a problem? Loading entire records is not an option.
The issue isn't that it's a "simple" thing to do. The issue is that it's an unscalable thing to do. Firestore is fast and cheap because of the limitations it places on queries. Limiting queries to a single inequality/range filter allows it to scale massively without requiring arbitrarily large amounts of memory to perform lots of data transformations. It can simply stream results directly to the client without loading them all into enough memory to hold all of the results. While it's not necessary to understand how this works, it's necessary to accept the limitations if you want to use Firestore effectively.
For your particular case, you could make your data more scalable to query by changing to your "status" field from a single string field to a set of boolean fields. For each one of the possible status values, you could instead have a boolean field that indicates the current status. So, for the query you show here, if you had a field called "isProcessed" with a boolean value true/false, you could find the first 10 unprocessed documents created before the current date like this:
const docQuery = db.collection('documents')
.where('created_at', '<=', new Date())
.where('processed', '==', false)
.limit(10);
Yes, this means you have to keep each field up to date when there is a change in status, but you now have a schema that's possible to query at massive scale.
I have a question about firestore query.
I'm working react native project with firestore
I made index to retrieve data which has been voted in a week.
So, my code is like below.
db.getData
.where('updatedAt' '<=' now - week)
.orderBy('count', 'desc')
I know it's not allowed in firestore.
Any idea of this?
If I need to restructure DB, then please show me how it looks.
It depends on the type of updatedAt data. Is is a date object or a number or string timestamp? I prefer to store updates with plain Date.now() timestamps (as number type). So, if updatedAt is a number it's a simple comparison. Tip, ...just name it updated rather than updatedAt, so you have created and updated fields.