I am using Firestore to store array of messages in a chat app. For each message send, I update my message array with "arrayUnion." On update, sometimes two items will be added (to the client DB, and quickly be overriden by server)
From the docs: "arrayUnion() adds elements to an array but only elements not already present". However, my case I see that my client Firebase DB instance will add duplicate data, and then the server will correct this upon successful write, and dedup.
db.update({
messages: firebase.firestore.FieldValue.arrayUnion(serverMessage)
})
For ~1 second, I have duplicate items in arrays. I'm not sure why my data is being duplicated (I have validated that I only have one listener on the client) --- but I also don't understand why the client Firebase db instance allows for the duplicate write. Here's a visual of the errors I'm seeing.
Related
In my Flutter/Dart mobile app I make use of Firebase RTDB persistence to enable offline use of the app.
My understanding is that to enable persistence you have to make the call, as per the following piece of code, before using any database references to eg. query the database. I use the following piece of code to enable persistence immediately after loading the app and it works fine:
FirebaseDatabase firebaseDatabase = FirebaseDatabase.instance;
bool _success = await firebaseDatabase.setPersistenceEnabled(true);
print(_success); // Prints true, so persistence is set 'on'.
When I logout of the app I attempt to turn persistence off with:
bool _success = await firebaseDatabase.setPersistenceEnabled(false);
print(_success); // Prints false, so persistence is still 'on', ie. the call failed.
I assume the reason persistence cannot be turned off is because there have been calls to db references prior to trying to switch it off.
This leads to three questions, I guess:
Should I be worried about turning it off at all, when I logout? The reason I attempt it is good house-keeping, mainly. I clean up shared preferences, close keepsyncd's, etc when logout is run. Also, though, the user can have multiple userids to login and I want to make sure that I am not retaining persisted data from their previous login id.
Related to 1, does setting persistence to false clear the cache of
data and potential queued calls to the db?
If the answers to 1 and 2 are 'yes', how can I switch persistence off given the code I'm using to do so keeps telling me it failed?
The typical way to handle this is to enable persistence once a user logs in.
Once disk persistence has been enabled and your app has used the database, it cannot be turned off. The documentation says this about it:
The returned Future will complete with true if the operation was successful or false if the persistence could not be set (because database references have already been created).
That last bit is clearly the case for you: you've been using the database already, which means that disk persistence is on.
To your specific questions:
Unfortunately the data in the local cache cannot be cleared up through the API at the moment. It is a valid feature request, but for now you'll have to assume that any data on the device can be seen by any user on that device (or device profile).
Disabling disk persistence keep the client from adding data to the cache. It does not clear any existing data in the cache.
One child node of my Firebase realtime database has become huge (aroung 20 GB) and I need to purge this and insert the the extracted data of last month from the backup into the Firebase realtime database using Python Admin SDK.
In the documentation, I see the following options:
set - Write or replace data to a defined path, like messages/users/
update - Update some of the keys for a defined path without replacing all of the data
push - Add to a list of data in the database. Every time you push a new node onto a list, your database generates a unique key, like messages/users//
transaction - Use transactions when working with complex data that could be corrupted by concurrent updates
However, I want to add/insert the data from the firebase backup. I have to insert because the app is used in production and I cannot afford the overwrite of data.
Is there any method available to insert/add the data and not overwrite the data?
Any help/support is greatly appreciated.
There is no way to do this in Firebase Realtime Database without reading the current value of each location.
The only operation that allows you to update data based on its existing value is a transaction. A Firebase transaction gives you the (likely) current value at a location, and you then return what the new value should become.
But if the data you're restoring is (largely) the same as the data you have in the database, you might be able to use an update() call with sufficiently deep paths.
Is there a way to programmatically determine from a DocumentClientException where StatusCode == HttpStatusCode.NotFound whether it was the document, the collection, or the database that was not found?
I'm trying to figure out whether I can implement on-demand collection provisioning and only call DocumentClient.CreateDocumentCollectionIfNotExistsAsync when I need to. I'm trying to avoid calling it before making every request (presumably this adds an extra network roundtrip to every request). Likewise, I'm trying to avoid calling it on error recovery when I know it won't help.
From experimentation with the local emulator, the only field I see varying in these three cases is DocumentClientException.Error.Message, and only when the database cannot be found. I generally try to avoid exception dispatching based on human-readable messages.
Wrong database name:
StatusCode: HttpStatusCode.NotFound
Error.Message: {\"Errors\":[\"Owner resource does not exist\"]}...
Correct database name, wrong collection name:
StatusCode: HttpStatusCode.NotFound
Error.Message: {\"Errors\":[\"Resource Not Found\"]}...
Correct database name, correct collection name, incorrect document ID:
StatusCode: HttpStatusCode.NotFound
Error.Message: {\"Errors\":[\"Resource Not Found\"]}...
I'm planning to use a database with its own offer. Since collections inside a database with its own offer are cheap, I'm trying to see whether I can segregate each tenant in my multi-tenant application into its own collection. Each tenant ends up having a different indexing and default TTL policy. The set of collections is not fixed and changes dynamically during runtime as new tenants sign up. I cannot predict when I will need to add a new collection. There's no new tenant notification: I just get a request that I need to handle by creating a document in a possibly non-existent collection. There's a process to garbage collect unused collections.
I'm using the NuGet package Microsoft.Azure.DocumentDB.Core Version 1.9.1 in a .NET Core 2.1 app targeting a SQL API Cosmos DB instance.
If you look at the Message property in detail, you should see following strings that informs whether 404 Not Found response was generated due to Document vs Collection.
ResourceType: Document
ResourceType: Collection
It's not ideal but you can try to regex this information out of error message.
After scouring the documentation, I recently learned that a shared realm (globally available to all users of my app) can only be queried with Realm.asyncOpen. For example, I have a /shared realm that has read-only access to any user. I tried querying it in the usual way, but it returned zero objects. But if I query it like this, it works:
Realm.asyncOpen(configuration: sharedConfig) { realm, error in
if let realm = realm {
// Realm successfully opened
self.announcements = realm.objects(Announcement.self)
print(self.announcements)
self.tableView.reloadData()
} else if let error = error {
print(error)
}
}
This method is visibly slower than a usual realm query since it appears to be fetching the data from the server instead of a local, already-synced realm.
Does this mean that the objects pulled down are never stored in the local copy of the realm, but are queried from the ROS each time I access them?
In other words, are shared realms pulled and not synced?
a shared realm (globally available to all users of my app) can only be queried with Realm.asyncOpen
This is incorrect. If a user only has read-only access to a Realm, it must be obtained with Realm.asyncOpen. That's explicitly what the documentation you linked to states.
This method is visibly slower than a usual realm query since it appears to be fetching the data from the server instead of a local, already-synced realm.
Almost correct. Yes data is fetched from the server, but not the whole Realm from scratch. Only the new data since the last time the Realm was synced with your local copy.
Does this mean that the objects pulled down are never stored in the local copy of the realm, but are queried from the ROS each time I access them?
This synced Realm is persisted locally and will be preserved across application launches.
In other words, are shared realms pulled and not synced?
No.
Taking a step back, let's explain what's happening here.
The reason why you get a "permission denied" error if you attempt to open a read-only synced Realm synchronously is that upon initialization, a local Realm file will be created, performing write operations to write the Realm's schema (i.e. create db tables, columns & metadata) immediately. However, since the user does not have write access to the Realm, the Realm Object Server (ROS) rejects the changes and triggers your global error handler notifying you that an illegal attempt to modify the file was made by your user.
The reason why this doesn't happen with asyncOpen is that it's an asynchronous operation and therefore doesn't need to give you a valid Realm immediately, so it doesn't need to "bootstrap" it by writing the schema to it. Instead, it requests the latest state of the Realm from ROS and vends it back to you once it's fully available in its latest state at the point in time at which the call was started.
That being said, if the local copy of the Realm already has its schema initialized (i.e. after a successful asyncOpen call), and the in-memory schema defined by either the default schema or the custom object types specified in Realm.Configuration hasn't changed, then no schema will be attempted to be written to the file.
This means that any time after a successful asyncOpen call, the Realm could be accessed synchronously without going through asyncOpen as long as you're ok with potentially not having the most up to date data from ROS.
So in your case, it appears as though you only want to use asyncOpen for the very first access to the Realm, so you could persist that state (using another Realm, or NSUserDefaults) and check for it to determine whether or not to open the Realm the asynchronously or synchronously.
I was testing some changes to my firebase and accidently submitted 100's of updates to a reference. Now when I try and delete the reference it then creates a new reference with different data.
I have tried deleting everything in the firebase but it will just keep creating a new reference.
In this specific example I used set() to add 5 random values to a user name Michael. The 5 random values were called 100's of times and now when I delete the Michael user to test again it already has a value queued up and recreates itself immediately. I looked at my upload usage and it showed a huge amount of data being uploaded at one point that coincides with this error.
Any idea how to remove these queued up changes?
Make sure to disconnect the client that is writing this data. I suspect somewhere you have a process running that is generating these writes.
If you can't stop the offending process for some reason, you could always modify your security rules to deny access to the client that's doing the writes (or if it's a server using a Firebase Secret to authenticate, you could revoke its secret).
I've had a similar issue - think it has to do with your session / caching.
Try logging out of firebase and back in - if the records are still there, make a backup of your security rules, then use:
{
"rules": {
".read": false,
".write": false
}
}
and delete them.