In the new Cloud Firestore you can set a server timestamp on my message object using the following line during creation of the object:
showtime: firebase.firestore.FieldValue.serverTimestamp()
This is useful for messages from any user as it does not rely in each users local clock.
The challenge I am having is in querying and setting the showtime field afterwards. What I want to do is look up any message that has a showtime that is early than 'now'. In my application many messages are pushed into the future. I only want the ones with a showtime before today to be returned and displayed.
Here is my query:
var query = firebase.firestore()
.collection('messages')
.where('owner','==',userID)
.where('showtime','<',timenow)
.orderBy('showtime', 'desc')
.limit(25);
The challenge is that I do not know how to get the current time (on the server) to use in the query. There is a now() call on the type Timestamp in Firebase, but I am unsure how to call it AND I am not sure based on some other questions here whether the Firebase timestamp matches the Cloud Firestore timestamp!
So the question is: How do I set a variable called timenow to be the current time on the server and use it in a query to pull 25 messages before now? (in Javascript on the client and also extra credit for in a function on the server)
A quick follow on question is then how I update the showtime timestamp to be one week later than now?
Hope we have some Firebase / Cloud Firestore mavens on Stackoverflow!
** Choosing an answer below with the caveat of a feature request: A call in Firebase to the server to request a current timestamp so that the client code can work off one standard clock. **
Expanding on the excellent answer by Alex Dunlop, I was in a similar situation and it made sense once I realized that Firestore operates according to two design principles (or at least, that is my analysis)
The emphasis is on simple queries that stream live updates to you. To enable this, queries are by design limited in complexity. This takes load of the Firestore database server and allows it to stream (effectively re-run) queries fast.
Because of the limited query language and few 'server side' operators (FieldValue is one of the few), Google can optimize the queries mercilessly.
There are many more complex operations that developers require. Instead of implementing them in Firestore, developers are asked to spin up a cloud function which makes them responsible (in code and cost) for that additional complexity.
If you come from a conventional DB design like Postgres, the lack of query expressiveness and server-side operations is stunning. If you think of the Firestore use case and the principles, it all makes sense.
Try Firestore security rules with data validation:
match /messages/{msgId} {
// optionally: request.time + someBuffer
allow read: if request.time > resource.data.showtime;
}
I don't think you want to trust the client (since you mentioned you want to block client access to future showtimes). Besides changing their clock, they could just edit the javascript.
A buffer might be necessary to not invalidate the query if there is some discrepancy between client provided Date.now() and Firestore rules request.time, sepecifically if the request.time happens to be earlier than client date, then the query would have documents falling outside valid range and fail.
Checking the read timestamp on an empty snapshot seems to work:
const serverTime = (await db.collection('nonexistent').get()).readTime;
Agreed with one of the other comments that this extra call could be expensive, but this should work in cases where an accurate "not before" server time is important for correctness.
You are right serverTimestamp() is exactly for getting a timestamp on the server and not relying on the users local clock. One thing to note is that generally sending a message and getting the timestamp from a users local clock is going to be okay as a message timestamp is not extremely time sensitive. Sure you would like to know when the message is sent but if it is within 1-2 seconds not a problem in most cases.
If you are doing a query on the client side your query should not be based on the server time it should be based on the client side. As it is a client query not a server query.
Here is a client query that you are after.
const currentTime = new Date();
const query = firebase.firestore()
.collection('messages')
.where('owner','==',userID)
.where('showtime','<',currentTime)
.orderBy('showtime', 'desc')
.limit(25);
This query will get 25 messages with a 'showtime' after the current time on the client.
Now if the messages need to be extremely time sensitive and you do absolutely need the messages to based off the server timestamp I recommend that instead of doing a query on the client like above you set up a cloud function api.
Have a look at the firebase docs for calling cloud functions directly if you haven't before.
Here is what you would want your cloud function to look like:
import * as functions from 'firebase-functions';
import * as admin from 'firebase-admin';
admin.initializeApp()
exports.getUserMessages = functions.https.onCall((data, context) => {
const uid = context.auth.uid;
const firestore = admin.firestore();
firestore.collection('messages')
.where('owner', '==', uid)
.where('showtime', '<', new Date())
.orderBy('showtime', 'desc')
.limit(25)
.get()
.then(snapshot => {
return snapshot;
});
});
This will get the messages based of the server timestamp. One thing to note is UNLESS you need this to be extremely time sensitive this is not a good idea. As this call has to do an extra unnecessary call every time you call it. Because you are doing a call to the cloud function and then you are doing a query to the firestore database.
I would recommend that instead of doing it based on the server time you do it based on client timestamp. 99 times out of 100 the time difference between client and server is not worth the extra double calls you are doing, especially when you think about scaling everything up when you get more users.
Hope that answered your question :)
Related
I'm relying on Firebase Firestore offline capabilities, so I'm not using await on my queries as stated on the Access Data Offline Firebase doc. I'm expecting that when I write something I'll get an immediate reflection on my read stream, however, I'm only getting an update when the server/remote has been updated. Basically:
Update something in the DB. Note, I'm not using await
_db.doc(parentDoc).collection(DocInnerCollection).doc(childDoc).update({
"name": value,
});
I expect my listeners to be updated immediately. Note I've set the includeMetadataChanges to true as stated in the above doc.
_db.doc(parentDoc)
.collection(DocInnerCollection)
.orderBy('start_date', 'desc')
.limitToLast(1)
.snapshots(includeMetadataChanges: true)
.map((snapshot) {
print(snapshot.metadata.isFromCache)
});
However, I get no such update and instead I only get an update when the server has been updated.
You're requesting only one document with .limitToLast(1), yet are not providing a sort order for your query. This essentially means that you'll get a random document from your collection, and the chances of that being the newly updated document are close to zero.
If you want the latest (not just last) document, you need some ordering criteria to determine what latest means. Typically you'd do this by:
Adding a lastUpdated field to your documents, and setting that to firebase.firestore.FieldValue.serverTimestamp().
Ordering your query on that timestamp with orderBy('lastUpdated', 'desc').
And then limiting to the first result with limit(1).
Is it possible to filter a Firestore query using the server's timestamp? When trying to perform a query such as:
firebase.app.firestore()
.collection('posts')
.where('timestamp', '<=', firebase.firestore.FieldValue.serverTimestamp())
The following error is thrown.
FirebaseError: Function Query.where() called with invalid data. FieldValue.serverTimestamp() can only be used with update() and set()
I know that I can use new Date() instead of FieldValue.serverTimestamp(), but it would be nice if there was a way to query using the trusted server timestamp. My use case is a collection of discount codes with expiry dates, where the expiry dates are saved as Firestore timestamps. If a user changes their system time this query can potentially return invalid (expired) results if I rely on a client-side date for filtering. I'm using Firebase functions to actually process the discount (where it is easy to validate the expiry using the server timestamp), so users with incorrect system times would still not be able to actually use the discount code. Nonetheless it would be nice to guarantee the time in such a query to never show an expired code in the first place.
FieldValue.serverTimestamp() is a flag, not a real value - it instructs FireeStore to use the current server time as a value during a write operation. It is NOT a true "value" that you can use as a substitute for "now".
What you are thinking of (and want to use) is firebase.firestore.Timestamp.now(), (sp?) https://firebase.google.com/docs/reference/js/firebase.firestore.Timestamp , which uses Epoch time (compensating for timezone). Changing their local clock will NOT defeat this - as you may have noticed elsewhere, most browsers and services (including Firestore) need tot local clock to be fairly accurate to maintain operations. If your user/hacker sufficiently hacks their local clock to try to get around your rules, their Firestore service won't be working anyway.
I've been thinking about a problem similar to yours.
Late reply, but I'll write a solution in hopes that it will help
As you know, "firebase.firestore.FieldValue.serverTimestamp()" cannot be used for querying, and it is very uncertain to query with the device's built-in Timestamp. (e.g. things like currentTimestamp(), Date() )
In this case, the solution may be to use the Security Rule in Firestore.
In my case, I used that method to restrict users.
The "suspesnsions_user" collection is used to limit membership due to fraudulent use of users. (User's membership must be suspended before releasedTimestamp.)
match /suspensions_user/{userId} {
allow read : if request.auth.uid == userId && resource.data.releasedTimestamp > request.time
}
If the Security Rule is used as follows, if the user can access the document located in "suspensions_user/{userId}" through the relevant Security Rule, it can be determined that he is in the status of loss of membership.
Conversely, if the user cannot access the document for reasons such as permission denied , it can be determined that he has normal membership.
As you said in the question, accurate validation is possible if you apply the corresponding Security Rule to a document such as "/coupons/{id}" to validate access to coupons.
Here is what I do to keep my date comparisons accurate:
let timestamp = new Date().toISOString();
// "2021-03-01T11:10:51.392Z"
If you do this in your Firebase Function (or in the client), you'll always get UTC time, in ISO format (ISO 8601).
This is great because you can do simple String comparisons of ISO 8601 date strings.
Does anyone know how to limit an array so new items get pushed in and old ones are discarded in the same write?
I'm guessing this isn't possible but it sure would be handy.
// * Store notification
// Users collection
const usersCollection = db.collection('users').doc(uid).collection('notifications').doc();
// Write this notification to the database as well
await usersCollection.update({
count: admin.firestore.FieldValue.increment,
notifications: admin.firestore.FieldValue.arrayUnion({
'symbol': symbol,
'companyname': companyname,
'change': priceDifference,
'changeDirection': directionOperatorHandler,
'updatedPrice': symbolLatestPrice,
'timestamp': currentTimestamp,
})
});
Written in Typescript
Alternatively, I was thinking of running a scheduled cloud function every week to go through and trim down the arrays based on the timestamp.
The reason I'm using an array to store my notifications is because I'm expecting a lot of writes.
There is no simple configuration for this. Your code should implement your requirements by:
Reading the document
Modifying the array in memory
Checking that the size is within limits
Writing the document back
I'm trying to update the same document which triggered an onUpdate cloud function, with a read value from the same collection.
This is in a kind of chat app made in Flutter, where the previous response to an inquiry is replicated to the document now being updated, for easier showing in the app.
The code does work, however when a user quickly responds to two separate inquiries, they both read the same latest response thus setting the same previousResponse. This must be down to the asynchronous nature of flutter and/or the cloud function, but I can't figure out where to await or if there's a better way to make the function, so it is never triggering the onUpdate for the same user, until a previous trigger is finished.
Last part also sound a bit like a bad idea.
So far I tried sticking the read/update in a transaction, however that only seems to work for the single function call, and not when they're asynchronous.
Also figured I could fix it, by reading the previous response in a transaction on the client, however firebase doesn't allow reading from a collection in a transaction, when not using the server API.
async function setPreviousResponseToInquiry(
senderUid: string,
recipientUid: string,
inquiryId: string) {
return admin.firestore().collection('/inquiries')
.where('recipientUid', '==', recipientUid)
.where('senderUid', '==', senderUid)
.where('responded', '==', true)
.orderBy('modified', 'desc')
.limit(2)
.get().then(snapshot => {
if (!snapshot.empty &&
snapshot.docs.length >= 2) {
return admin.firestore()
.doc(`/inquiries/${inquiryId}`)
.get().then(snap => {
return snap.ref.update({
previousResponse: snapshot.docs[1].data().response
})
})
}
})
}
I see three possible solutions:
Use a transaction on the server, which ensures that the update you write must be based on the version of the data you read. If the value you write depends on the data that trigger the Cloud Function, you may need to re-read that data as part of the transaction.
Don't use Cloud Functions, but run all updates from the client. This allows you to use transactions to prevent the race condition.
If it's no possible to use a transaction, you may have to include a custom version number in both the upstream data (the data that triggers the write), and the fanned out data that you're updating. You can then use security rules to ensure that the downstream data can only be written if its version matches the current upstream data.
I'd consider/try them in the above order, as they gradually get more involved.
My use case is that I want to keep aggregating my firebase user count in the database for quick and easy access. For that, I have a cloud function listening on user.onCreate and it simply increments a field in a document using the atomic FieldValue.increment.
Here is the code:
exports.createProfile = functions.auth.user().onCreate(async user => {
return Promise.all([
addProfileToDatabase(),
function() {
db.collection('someCollection').doc(docId).update({
count: admin.firestore.FieldValue.increment(1)
)}
}
])
})
Issue: the count in the database becomes more than the number of authenticated users shown in the Authentication tab of Firebase. I regularly reset it to the correct number and then it slowly increases again.
I have read about the write throttling on a document, but that should instead result in lower count if at all. But why is that the count in the database always overshoots the actual count?
Without seeing your code, the only thing I can imagine is that your function isn't idempotent. It's possible that functions may be invoked more than once per triggering event. This would be an explanation why your count exceeds the expectation.
Read more about Cloud Functions idempotency in the documentation and also this video.