How to set timeout for firestore operations like read, write and update?. Because When connection is not there.. it's not triggering OnCompleteListener
There are no configurable timeouts for Firestore reads and writes. The Firestore SDK will automatically retry database operations with the assumption that connectivity will return shortly.
If you want to force some code to run after some amount of time has elapsed, you will have to implement your own timer.
If you want to only use locally cached values rather than require a round trip with the server, you can specify a source of "cache" in the query as well. For example, in Android, use get(Source) instead of the normal get().
Firestore Get method have a parameter to define timeout (float):
.get(timeout=500000)
Here is API documentation
Related
I have noticed a delay between the time some data is written to Firestore and when it is available to be read via a query. In my tests I see this can go up to 30sec.
Is this documented anywhere?
Are there ways to decrease this delay?
Is there a way to know the server timestamp corresponding to the data being returned? Or to have any indication about this delay in the data being returned from Firestore?
(say some data is written to the server at 1:00 - the document is created server-side at that time, I query it at 1:01, but due to the delay it returns the data as it was at 0:58 server-side, the timestamp would be 0:58)
Here I am not speaking about anything with high load, my tests were just about writing and then reading 1 small document.
Firestore will have some delays, even I have noticed it. It's better to use the Realtime Database as it lives up to the name, the time lag is minimal, less than a second in most cases !
If you use Firestore on a native device offline first is enabled by default. That means data is first written to a cache on your device and then synced to the backend. On Web you can enable that to. To listen for those changes when a write is saved to the backend you would need to enable includeMetadataChanges on your realtime listeners:
db.collection("cities").doc("SF")
.onSnapshot({
// Listen for document metadata changes
includeMetadataChanges: true
}, (doc) => {
// ...
});
That way you will get a notice when the data is written to the backend. You can read more about it here.
The delay should be only between different devices. If you listen to the changes on your device where you write the data it will be shown immidiately.
Also something you should take care of. If you have offline enabled you should not await for the writes to finish. Just use them as if they are synchronous (even on web with enabled offline feature). I had that before in my code and it looked like the database was slow. By removing the await the code was running much faster.
I have a Firebase realtime database structure that looks like this:
rooms/
room_id/
user_1/
name: 'hey',
connected: true
connected is basically a Boolean indicating as to whether the user is connected or not and will be set to false using onDisconnect() that Firebase provides.
Now my question is - If I trigger a cloud function every time theconnected property of a user changes , can I run a setTimeout() for 45 seconds . If the connected property is still false, at the end of the setTimeout() (for which I read that particular connected value from the db ) then I delete the node of the user (like the user_1 node above).
Will ths setTimeout() pose a problem if there are many triggers fired simultaneously?
In short, Cloud Functions have a maximum time they can run.
If your timeout makes it's callback after that time limit expired, the function will already have been terminated.
Consider using a very simple and efficient way for calling scheduled code in Cloud Functions called a cron-job.
Ref
If you use setTimeout() to delay the execution of a function for 45 seconds, that's probably not enough time to cause a problem. The default timeout for a function is 1 minute, and if you exceed that, the function will be forced to terminate immediately. If you are concerned about this, you should simply increase the timeout.
Bear in mind that you are paying for the entire time that a function executes, so even if you pause the function, you will be billed for that time. If you want a more efficient way of delaying some work for later, consider using Cloud Tasks to schedule that.
For my understanding your functionality is intend to monitoring the users that connected and is on going connect to the Firebase mealtime database from the cloud function. Is that correct?
For monitoring the Firebase Realtime database, GCP provided the tool to monitor the DB performance and usage.
Or simply you just want to keep the connection a live ?
if the request to the Firebase Realtime DB is RESTful requests like GET and PUT, it only keep the connection per request, but is is High requests, it still cost more.
Normally, we suggest the client to use the native SDKs for your app's platform, instead of the REST API. The SDKs maintain open connections, reducing the SSL encryption costs and database load that can add up with the REST API.
However, If you do use the REST API, consider using an HTTP keep-alive to maintain an open connection or use server-sent events and set keep-alive, which can reduce costs from SSL handshakes.
I wonder if firebase listeners incur more reads than 'normal', snapshot gets from firestore database. I assume a listener has to be constantly checking the DB to see if something new is in there and therefore that would add up to the reads on my app.
A get() is not necessarily more expensive than a snapshot listener. The difference between a get and a listen boils down to the fact that get will read a document exactly once, and a listen will read at least once, which each change to that document incurring another read, for as long as the listener is active.
A listener will only cause more billing for reads if the listener is left active, and the document changes over the time while it's active.
You could instead poll the document with several gets over time, but each one of those gets will cost a document read, even if the document is unchanged. So you will have to determine for yourself what the best way is to get updates to a document, or even if you want updates at all.
You have two ways to retrieve data from Firestore:
Promise
Listener
When you make use of promise with get() method, you are sending lazy request to server. While listener method opens a connection channel that retrieve information in real time on every change. (It doesn't mean such a process making request every time)
When you access to data using the Listener strategy, you are opening a communication channel that, of course, consumes connection resources. But you can unsubscribe it: Check How to remove listener for DocumentSnapshot events (Google Cloud FireStore)
The whole thing with a listener is that you don't want it to constantly check the database, like on an interval. Instead, a firestore listener opens up a websocket connection which, in contrast to http, allows two way communication - both hosts can be both server and client. This means that instead of constantly checking the database, the server tells you when there are updates, after which your snapshot callback code is run.
So like others has stated, attaching a listener consumes one read per document when the listener is attached, plus one per potential document update for as long as the listener is active.
My current application developed in Unity uses Firebase Realtime Database with database persistence enabled. This works great for offline use of the application (such as in areas of no signal).
However, if a new user runs the application for the first time without an internet connection - the application freezes. I am guessing, it's because it needs to pull down the database for the first time in order for persistence to work.
I am aware of threads such as: 'Detect if Firebase connection is lost/regained' that talk about handling database disconnection from Firebase.
However, is there anyway I can check to see if it is the users first time using the application (eg via presence of the persistent database?). I can then inform them they must go online for 'first time setup'?
In addition to #frank-van-puffelen's answer, I do not believe that Firebase RTDB should itself cause your game to lock up until a network connection occurs. If your game is playable on first launch without a network connect (ie: your logic itself doesn't require some initial state from the network), you may want to check the following issues:
Make sure you can handle null. If your game logic is in a Coroutine, Unity may decide to silently stop it rather than fully failing out.
If you're interacting with the database via Transactions, generally assume that it will run twice (once against your local cache then again when the cache is synced with the server if the value is different). This means that the first time you perform a change via a transaction, you'll likely have a null previous state.
If you can, prefer to listen to ValueChanged over GetValueAsync. You'll always get this callback on your main Unity thread, you'll always get the callback once on registration with the data in your local cache, and the data will be periodically updated as the server updates. Further, if you see #frank-van-puffelen answer elsewhere, if you're using GetValueAsync you may not get the data you expect (including a null if the user is offline). If your game is frozen because it's waiting on a ContinueWithOnMainThread (always prefer this to ContinueWith in Unity unless you have a reason not to) or an await statement, this could ValueChanged may work around this as well (I don't think this should be the case).
Double check your object lifetimes. There are a ton of reasons that an application may freeze, but when dealing with asynchronous logic definitely make sure you're aware of the differences between Unity's GameObject lifecycle and C#'s typical object lifecycle (see this post and my own on interacting with asynchronous logic with Unity and Firebase). If an objects OnDestroy is invoked before await, ContinueWith[OnMainThread], or ValueChanged is invoked, you're in danger of running into null references in your own code. This can happen if a scene changes, the frame after Destroy is called, or immediately following a DestroyImmediate.
Finally, many Firebase functions have an Async and synchronous variant (ex: CheckDependencies and CheckDependenciesAsync). I don't think there are any to call out for Realtime Database proper, but if you use the non async variant of a function (or if you spinlock on the task completing, including forgetting to yield in a coroutine), the game will definitely freeze for a bit. Remember that any cloud product is i/o bound by nature, and will typically run slower than your game's update loop (although Firebase does its best to be as fast as possible).
I hope this helps!
--Patrick
There is nothing in the Firebase Database API to detect whether its offline cache was populated.
But you can detect when you make a connection to the database, for example by listening to the .info/connected node. And then when that first is set to true, you can set a local flag in the local storage, for example in PlayerPrefs.
With this code in place, you can then detect if the flag is set in the PlayerPrefs, and if not, show a message to the user that they need to have a network connection for you to download the initial data.
Is there a way to automatically move expired documents to blob storage via change feed?
I Google but found no solution to automatically move expired documents to blob storage via the change feed option. Is it possible?
There is not built in functionality for something like that and the change feed would be of no use in this case.
The change feed processor (which is what the Azure Function trigger is using too) won't notify you for deleted documents so you can't listen for them.
Your best bet is to write some custom application that does scheduling archiving and deleted the archived document.
As statements in the Cosmos db TTL document: When you configure TTL, the system will automatically delete the expired items based on the TTL value, unlike a delete operation that is explicitly issued by the client application.
So,it is controlled by the cosmos db system,not client side.You could follow and vote up this feedback to push the progress of cosmos db.
To come back to this question, one way I've found that works is to make use of the in-built TTL (Let CosmosDB expire documents) and to have a backup script that queries for documents that are near the TTL, but with a safe window in case of latency - e.g. I have the window up to 24 hours.
The main reasoning for this is that issuing deletes as a query not only uses RUs, but quite a lot of them. Even when you slim your indexes down you can still have massive RU usage, whereas letting Cosmos TTL the documents itself induces no extra RU use.
A pattern that I came up with to help is to have my backup script enable the container-level TTL when it starts (doing an initial backup first to ensure no data loss occurs instantly) and to have a try-except-finally, with the finally removing/turning off the TTL to stop it potentially removing data in case my script is down for longer than the window. I'm not yet sure of the performance hit that might occur on large containers when it has to update indexes, but in a logical sense this approach seems to be effective.